DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Effective Filing Date
Examiner acknowledges that the instant application claims domestic benefit to provisionally filed application 62737588, filed 09/27/2018. This provisionally filed application has been reviewed and provides sufficient support for the instant application’s claims.
Examiner acknowledges that the instant application is a 371 national stage entry to PCT/US2019/053539 filed 09/27/2019.
Examiner acknowledges that the instant application is a continuation of previously filed application 17280365 filed 03/26/2021.
As such, the effective filing date of the application is 09/27/2018.
Status of Claims
The amendment filed 04/01/2026 has been entered. Claims 1, 4-6, 9 and 12 have been amended. Claims 1-16 are now pending.
Response to Arguments
Examiner has fully considered applicant’s arguments with regards to the 35 USC 101 and 35 USC 103 rejections filed 10/02/2025. Examiner acknowledges errors made in the previously filed Non-Final Office Action. New rejections under 35 USC 101 and 35 USC 103 are set forth below. As such, applicants’ arguments are considered moot, and the previous non-final rejection Office action is hereby withdrawn. Of further note is that the previous examiner of this application is no longer the current examiner; please see the conclusion section for new contact information pertaining to the current examiner.
Claim Interpretation Under 35 USC § 112 (f)
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“command module” in claim 11.
“update module” in claims 11 and 13.
“notification module” in claim 12.
“Object-Oriented Partially Observable Monte-Carlo Planning module” in claim 15.
The specification does not appear to recite corresponding structure, material, or acts for performing the function, and merely recite what functions the “modules” are capable of performing.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11-13 and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claim limitations “command module,” “update module,” “notification module” and “Object-Oriented Partially Observable Monte-Carlo Planning module” invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed functions and to clearly link the structure, material, or acts to the functions.
The “command module” in claim 11 is described in paragraphs [0036]-[0038]. The specification describes that the “command module 206 can be communicatively coupled to at least one of the processors 202, and can be configured to receive a language command from a user.” Applicant’s specification further describes other functions that the module can be configured to perform. However, it is unclear if the command module is software or hardware. Applicants’ drawings do not appear to clarify this further.
The “update module” in claims 11 and 13 is described in paragraphs [0037]-[0038]. The specification describes that the “update module 208 can be communicatively coupled to at least one processor 202, and can be configured to update the belief associated with the at least one target object in the OO-POMDP model.” Applicant’s specification further describes other functions that the module can be configured to perform. However, it is unclear if the command module is software or hardware. Applicants’ drawings do not appear to clarify this further.
The “notification module” in claim 12 is described in paragraph [0045]. The specification describes that the “notification module 214 can be communicatively coupled to a processor 202 and to at least one sensor 212, and configured to notify the user upon finding each target object.” Applicant’s specification further describes other functions that the module can be configured to perform. However, it is unclear if the command module is software or hardware. Applicants’ drawings do not appear to clarify this further.
The “Object-Oriented Partially Observable Monte-Carlo Planning module” of claim 15 does not appear to be described anywhere in the specification. It is possible that the “module” as claimed is a typographical error for “model.”
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, it recites:
“A method of performing a multi-object search task with a mobile robot, the mobile robot having at least one processor executing computer readable instructions stored in at least one non-transitory computer readable storage medium to perform operations comprising the steps of:
receiving, from a user, a language command identifying at least one target object and at least one location corresponding to the at least one target object;
updating at least one belief associated with the at least one target object based on the language command, the at least one belief pertaining to a state and at least one observation space within an environment of the mobile robot, wherein the state is represented in terms of classes and objects and each object has at least one attribute and a semantic label;
and searching, using at least one sensor on the mobile robot while traversing the at least one observation space identified in the updated at least one belief, for the at least one target object.”
The limitations as drafted, under their broadest reasonable interpretation, covers performance
of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the step of “receiving, from a user, a language command identifying at least one target object and at least one location corresponding to the at least one target object” in the context of this claim, given the broadest reasonable interpretation encompasses being told a command regarding an object in a particular location. Similarly, the steps of “updating…” and “searching…” could be performed mentally. For example, the steps encapsulate a human being told to find an object in a particular room, the human having expected the object to be in a different room, then searching for the object in the described location. Classes, objects, and semantic labels can similarly be conceptualized in the human mind. For example, a class of “dishware” including the object “mug.” Semantic labeling works very similarly to how humans understand and process language in our day-to-day lives. For instance, understanding that in the command “Find the mug in the kitchen” that the object “mug” is the target object and “kitchen” is the target location. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, claim 1 only recites the following additional elements: a mobile robot, a processor, non-transitory computer readable storage medium, and a sensor. All of the components are recited at a high-level of generality (i.e., as a generic robot, a generic processor performing a generic computer function of receiving and providing information, generic non-transitory computer readable storage medium performing the generic computer function of data storage, and a generic sensor performing the generic function of gathering data from its environment) such that it amounts no more than mere instructions to apply the exception using a generic component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, a series of mental steps without meaningful limits on practicing the abstract idea cannot provide inventive concept. The claim is not patent eligible.
Regarding claim 9, it recites:
“A robotic system for conducting a multi-object search task, the robotic system comprising:
at least one processor configured to execute computer readable instructions stored in at least one non-transitory computer readable storage medium configured with a representation of the multi-object search task having at least one belief pertaining to a state and at least one observation space within an environment of the robotic system, wherein the state is represented in terms of classes and objects and each object has at least one attribute and a semantic label relating to at least one target object;
and at least one sensor responsive to the processor, the at least one sensor configured to detect and provide sensor data indicative of the at least one observation space.”
The limitations as drafted, under their broadest reasonable interpretation, covers performance
of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the elements of “the multi-object search task having at least one belief pertaining to a state and at least one observation space within an environment of the robotic system, wherein the state is represented in terms of classes and objects and each object has at least one attribute and a semantic label relating to at least one target object” in the context of this claim, given the broadest reasonable interpretation encompasses classes, objects, attributes and semantic labels which can be conceptualized in the human mind. For example, a class of “dishware” including the object “mug” with the attribute “ceramic” could be encapsulated. Semantic labeling works very similarly to how humans understand and process language in our day-to-day lives. For instance, understanding that in the command “find the mug in the kitchen” that the object “mug” is the target object and “kitchen” is the target location. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, claim 1 only recites the following additional elements: a robotic system, a processor, non-transitory computer readable storage medium, and a sensor. All of the components are recited at a high-level of generality (i.e., as a generic robotic system, a generic processor performing a generic computer function of receiving and providing information, generic non-transitory computer readable storage medium performing the generic computer function of data storage, and a generic sensor performing the generic function of gathering data from its environment) such that it amounts no more than mere instructions to apply the exception using a generic component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, a series of mental steps without meaningful limits on practicing the abstract idea cannot provide inventive concept. The claim is not patent eligible.
Dependent claims 2-8 and 10-16 are similarly rejected under 35 U.S.C. 101 because the
claimed invention is directed to an abstract idea without significantly more. The dependent claims have
been given the full two-part analysis including analyzing the additional limitations both individually and
in combination. The dependent claims, when analyzed individually, and in combination, are also held to
be patent ineligible under 35 U.S.C. 101. The additional recited limitations of the dependent claims fail
to establish that the claims do not recite an abstract idea because the additional recited limitations of
the dependent claims merely further narrow the abstract idea. These claims only recite the addition of generic computing components such as processors and memory. All of the components are recited at a high level of generality (i.e., a generic processor performing a generic computer function of receiving and providing information) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more
than the judicial exception. As discussed above with respect to integration of the abstract idea into a
practical application, the additional elements of generic computer and robotic devices to perform the claimed steps amounts to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are not patent eligible.
Examiner encourages the applicants to consider amending the independent claims to include a robotic control step describing how the robot is controlled to traverse its environment based on the claimed language commands and beliefs.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1,6, 9, 11 and 13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gupta et al. (US 20070022078 A1), hereinafter Gupta.
Regarding claim 1, Gupta discloses:
A method of performing a multi-object search task with a mobile robot, the mobile robot having at least one processor executing computer readable instructions stored in at least one non-transitory computer readable storage medium (see at least [0025]: “FIG. 1 shows a system 100 according to one embodiment of the present invention. Computer system 110 comprises an input module 112, a memory device 114, a storage device 118, a processor 122, and an output module 124.”) to perform operations comprising the steps of:
receiving, from a user, a language command identifying at least one target object and at least one location corresponding to the at least one target object (see at least Fig. 4, the command “Make Coffee” and the Meta-Rules involving locating the room the object is in);
updating at least one belief associated with the at least one target object based on the language command, the at least one belief pertaining to a state and at least one observation space within an environment of the mobile robot (see at least [0006]: “In particular, to accomplish an indoor task, an autonomous system such as a humanoid robot should have a plan. It is desirable that such plans be fundamentally based on "common sense" knowledge. For example, steps for executing common household or office tasks can be collected from non-expert human volunteers using distributed capture techniques. However, it is also desirable that the robot be able to derive or update such a plan dynamically, taking into consideration aspects of the actual indoor environment where the task is to be performed. For example, the robot should be aware of the locations of objects involved in the plan. Furthermore, the robot should know how to respond to situations that arise before or during execution of the plan, such as a baby crying. Also, the robot should have awareness of the consequences of actions, such as the sink filling when the tap is on. In other words, the robot should be able to augment rules about performing tasks with knowledge of the actual environment, and with reasoning skills to adapt the rules to the environment in an appropriate manner. This integrated ability is referred to as commonsense reasoning.”),
wherein the state is represented in terms of classes and objects and each object has at least one attribute and a semantic label (see at least Fig. 2 and paragraphs [0036]-[0038]: “Another category of class rules determines likely uses and locations of objects. Since the source information in the knowledge database is contributed by multiple human volunteers, it cannot be specific to a particular environment, e.g., a particular home. However, such information may suggest, for example, that a couch may be located in any of multiple places such as the living room, family room, or den. Accordingly, this information is formulated statistically based on relative frequency of occurrence. For example, couches may be twice as likely to be located in living rooms than in family rooms… Another category of class rules performs synonym search and replacement, for example, to match unknown verbs to other verbs that the autonomous machine can interpret. For example, if a robot recognizes the word "replace," but not "change" or "alter," the use of lexical utilities such as WordNet facilitate binding "change" or "alter" within user statements to "replace." Another category of class rules performs verb tense search and replacement. This class rule category might be used by a robot to provide verbal feedback to the user about the robot's current state and actions, as well as to target specific robotic functionality. For most verbs, the rule of adding "ed" or "ing" applies to convert to past or present participle tenses. However exceptions exist, such as the forms make and made. Such exceptions may be stored in a look-up table associated with the generic class rule. According to one embodiment, to convert verb references to present tense, applicable exceptions are checked, and then the rule is applied.”); and
searching, using at least one sensor on the mobile robot while traversing the at least one observation space identified in the updated at least one belief, for the at least one target object (see at least Fig. 4.)
Regarding claim 6, Gupta discloses:
The method of Claim 1, further comprising updating the at least one belief based on object-specific observations made by the at least one sensor (see at least the sensors described in [0026]. See further [0006]: “In particular, to accomplish an indoor task, an autonomous system such as a humanoid robot should have a plan. It is desirable that such plans be fundamentally based on "common sense" knowledge. For example, steps for executing common household or office tasks can be collected from non-expert human volunteers using distributed capture techniques. However, it is also desirable that the robot be able to derive or update such a plan dynamically, taking into consideration aspects of the actual indoor environment where the task is to be performed. For example, the robot should be aware of the locations of objects involved in the plan. Furthermore, the robot should know how to respond to situations that arise before or during execution of the plan, such as a baby crying. Also, the robot should have awareness of the consequences of actions, such as the sink filling when the tap is on. In other words, the robot should be able to augment rules about performing tasks with knowledge of the actual environment, and with reasoning skills to adapt the rules to the environment in an appropriate manner. This integrated ability is referred to as commonsense reasoning.”)
Regarding claim 9, Gupta discloses:
A robotic system for conducting a multi-object search task, the robotic system comprising:
at least one processor configured to execute computer readable instructions stored in at least one non-transitory computer readable storage medium (see at least [0025]: “FIG. 1 shows a system 100 according to one embodiment of the present invention. Computer system 110 comprises an input module 112, a memory device 114, a storage device 118, a processor 122, and an output module 124.”)
configured with a representation of the multi-object search task having at least one belief pertaining to a state and at least one observation space within an environment of the robotic system (see at least Fig. 4, the command “Make Coffee” and the Meta-Rules involving locating the room the object is in)
wherein the state is represented in terms of classes and objects and each object has at least one attribute and a semantic label relating to at least one target object (see at least [0006]: “In particular, to accomplish an indoor task, an autonomous system such as a humanoid robot should have a plan. It is desirable that such plans be fundamentally based on "common sense" knowledge. For example, steps for executing common household or office tasks can be collected from non-expert human volunteers using distributed capture techniques. However, it is also desirable that the robot be able to derive or update such a plan dynamically, taking into consideration aspects of the actual indoor environment where the task is to be performed. For example, the robot should be aware of the locations of objects involved in the plan. Furthermore, the robot should know how to respond to situations that arise before or during execution of the plan, such as a baby crying. Also, the robot should have awareness of the consequences of actions, such as the sink filling when the tap is on. In other words, the robot should be able to augment rules about performing tasks with knowledge of the actual environment, and with reasoning skills to adapt the rules to the environment in an appropriate manner. This integrated ability is referred to as commonsense reasoning.”); and
at least one sensor responsive to the processor, the at least one sensor configured to detect and provide sensor data indicative of the at least one observation space (see at least [0026]: “ Input module 112 may also receive digital images directly from an imaging device 130, for example, a digital camera 130a (e.g., robotic eyes), a video system 130b (e.g., closed circuit television), an image scanner, or the like.”)
Regarding claim 11, Gupta discloses:
The robotic system of Claim 9, wherein the at least one non-transitory computer readable storage medium comprises:
a command module configured to receive from a user a language command identifying the at least one target object and at least one location corresponding to the at least one target object (see at least Fig. 4, the command “Make Coffee” and the Meta-Rules involving locating the room the object is in); and
an update module configured to update the at least one belief, associated with the at least one target object, based on the language command (see at least [0006]: “In particular, to accomplish an indoor task, an autonomous system such as a humanoid robot should have a plan. It is desirable that such plans be fundamentally based on "common sense" knowledge. For example, steps for executing common household or office tasks can be collected from non-expert human volunteers using distributed capture techniques. However, it is also desirable that the robot be able to derive or update such a plan dynamically, taking into consideration aspects of the actual indoor environment where the task is to be performed. For example, the robot should be aware of the locations of objects involved in the plan. Furthermore, the robot should know how to respond to situations that arise before or during execution of the plan, such as a baby crying. Also, the robot should have awareness of the consequences of actions, such as the sink filling when the tap is on. In other words, the robot should be able to augment rules about performing tasks with knowledge of the actual environment, and with reasoning skills to adapt the rules to the environment in an appropriate manner. This integrated ability is referred to as commonsense reasoning.”)
Regarding claim 13, Gupta discloses:
The robotic system of Claim 11, wherein the update module is configured to update the at least one belief based upon sensor data (see at least the sensors described in [0026]. See further [0006]: “In particular, to accomplish an indoor task, an autonomous system such as a humanoid robot should have a plan. It is desirable that such plans be fundamentally based on "common sense" knowledge. For example, steps for executing common household or office tasks can be collected from non-expert human volunteers using distributed capture techniques. However, it is also desirable that the robot be able to derive or update such a plan dynamically, taking into consideration aspects of the actual indoor environment where the task is to be performed. For example, the robot should be aware of the locations of objects involved in the plan. Furthermore, the robot should know how to respond to situations that arise before or during execution of the plan, such as a baby crying. Also, the robot should have awareness of the consequences of actions, such as the sink filling when the tap is on. In other words, the robot should be able to augment rules about performing tasks with knowledge of the actual environment, and with reasoning skills to adapt the rules to the environment in an appropriate manner. This integrated ability is referred to as commonsense reasoning.”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 10 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Koseki et al. (US 20180197100 A1), hereinafter Koseki.
Regarding claim 2, Gupta discloses:
The method of Claim 1.
Gupta does not explicitly disclose, but Koseki, in the analogous endeavor of robotic action determination teaches:
wherein the multi-object search task is represented in an Object-Oriented Partially Observable Markov Decision Process model (see at least [0004]: “According to an embodiment of the present invention, a computer-implemented method for selecting an action is provided. The method comprises reading, into a memory, a Partially Observed Markov Decision Process (POMDP) model, the POMDP model having top-k action IDs for each of belief states, the top-k action IDs maximizing expected long-term cumulative rewards in each time-step, and k being an integer of two or more; in the execution-time process of the POMDP model, detecting a situation where an action identified by the best action ID among the top-k action IDs for a current belief state is unable to be selected due to constraint; and selecting and executing an action identified by the second best action ID among the top-k action IDs for the current belief state in response to a detection of the situation. The top-k action IDs may be top-k alpha vectors, each of the top-k alpha vectors having an associated action; or identifiers of top-k actions associated with alpha vectors.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the method as taught by Koseki. This is because as stated [0002]-[0003]: “It is a difficult problem to determine which action a system should take in a given situation in spoken dialog systems, which help a user accomplish a task using a spoken language, because automatic speech recognition is unreliable and, therefore, the state of the conversation can never be known with certainly. The task mentioned above may be an operation of a robot or an operation completed by a natural conversation dialog. The POMDP has been recently used for solving this problem. Many study reports describe the POMDP model.”
Regarding claim 10, Gupta discloses:
The robotic system of Claim 9.
Gupta does not disclose, but Koseki teaches: wherein the multi-object search task is represented in an Object-Oriented Partially Observable Markov Decision Process model (see at least [0004]: “According to an embodiment of the present invention, a computer-implemented method for selecting an action is provided. The method comprises reading, into a memory, a Partially Observed Markov Decision Process (POMDP) model, the POMDP model having top-k action IDs for each of belief states, the top-k action IDs maximizing expected long-term cumulative rewards in each time-step, and k being an integer of two or more; in the execution-time process of the POMDP model, detecting a situation where an action identified by the best action ID among the top-k action IDs for a current belief state is unable to be selected due to constraint; and selecting and executing an action identified by the second best action ID among the top-k action IDs for the current belief state in response to a detection of the situation. The top-k action IDs may be top-k alpha vectors, each of the top-k alpha vectors having an associated action; or identifiers of top-k actions associated with alpha vectors.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the method as taught by Koseki. This is because as stated [0002]-[0003]: “It is a difficult problem to determine which action a system should take in a given situation in spoken dialog systems, which help a user accomplish a task using a spoken language, because automatic speech recognition is unreliable and, therefore, the state of the conversation can never be known with certainly. The task mentioned above may be an operation of a robot or an operation completed by a natural conversation dialog. The POMDP has been recently used for solving this problem. Many study reports describe the POMDP model.”
Regarding claim 14, Gupta discloses:
The robotic system of Claim 12, wherein the multi-object search task is represented in an Object-Oriented Partially Observable Markov Decision Process model (see at least [0004]: “According to an embodiment of the present invention, a computer-implemented method for selecting an action is provided. The method comprises reading, into a memory, a Partially Observed Markov Decision Process (POMDP) model, the POMDP model having top-k action IDs for each of belief states, the top-k action IDs maximizing expected long-term cumulative rewards in each time-step, and k being an integer of two or more; in the execution-time process of the POMDP model, detecting a situation where an action identified by the best action ID among the top-k action IDs for a current belief state is unable to be selected due to constraint; and selecting and executing an action identified by the second best action ID among the top-k action IDs for the current belief state in response to a detection of the situation. The top-k action IDs may be top-k alpha vectors, each of the top-k alpha vectors having an associated action; or identifiers of top-k actions associated with alpha vectors.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the method as taught by Koseki. This is because as stated [0002]-[0003]: “It is a difficult problem to determine which action a system should take in a given situation in spoken dialog systems, which help a user accomplish a task using a spoken language, because automatic speech recognition is unreliable and, therefore, the state of the conversation can never be known with certainly. The task mentioned above may be an operation of a robot or an operation completed by a natural conversation dialog. The POMDP has been recently used for solving this problem. Many study reports describe the POMDP model.”
Claims 3 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Shinn et al. (US 20180314942 A1), hereinafter Shinn.
Regarding claim 3, Shinn discloses:
The method of Claim 1.
Gupta does not explicitly disclose, but Shinn, in the analogous field of artificial intelligence decision making teaches wherein the method further comprises the step of updating the at least one belief based upon a new language command (see at least [0119]-[0120]: “As another example, when NPC1 talks to NPC2, the information of NPC1 affects the belief of NPC2. NPC2 updates its belief and generates a new plan to reach a new goal. At the same time, NPC1 updates its belief that it provided information to NPC2. In some embodiments, beliefs are used to represent the informational state of the agent. In some embodiments, the informational state of an agent is the agent's understanding or beliefs about the world and includes its beliefs about itself and other agents. In some embodiments, beliefs can include inference rules and allow forward chaining that results in new beliefs. In some embodiments, a belief system is used to represent an agent's understanding of the world. A belief may be held by the agent to be true but the belief itself may not necessarily be true and may change in the future. For example, NPC1 has a true belief that NPC2 is awake. However, at a later time, NPC2 is now asleep. Absent other input, with respect to NPC1, NPC1 still believes that NPC2 is awake even though the belief is no longer true.
In some embodiments, a player and a non-player character (NPC) agent can communicate using natural language. In some embodiments, NPCs can share information with one another. In various embodiments, the conversation between a player and an NPC can change the game. For example, a conversation between a player and an NPC changes the game environment including the NPC's understanding of the game world.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the belief updates as taught by Shinn. This is because as stated in [0003] of Shinn: “Many computer-based games use non-playing characters (NPCs) to simulate a dynamic environment. Typically, this is implemented using a decision tree and/or state machine. For example, NPCs and game events are often scripted to cycle between one of several states and are therefore limited in their range of variations. Moreover, multiple NPCs may share the same game logic and as a result will often appear to game players as duplicates of one another. In the case where there are multiple different NPCs, the different NPCs often have only superficial differences between one another and/or are one of a limited number of different variations of game logic. Implementing an NPC that acts independent from other NPCs using traditional artificial intelligence techniques is both processor and memory intensive. Therefore, there exists a need for a framework to support autonomous NPCs that utilizes both artificial intelligence (AI) and is scalable. Unlike traditional non-autonomous NPCs, autonomous AI agent characters may move and act independently from one another. Moreover, autonomous AI agent characters may be incorporated into a computer-based game such that their actions both respond to and also affect the game environment.” Shinn further states in [0070]: “In various embodiments, the process of FIG. 7 may be utilized to implement fully autonomous behavior for agents such as robots and non-player characters (NPCs) in a game.”
Regarding claim 4, Gupta discloses:
The method of Claim 3.
Gupta does not explicitly disclose, but Shinn teaches wherein the method further comprised the step of adjusting a course of travel based upon the updated at least one belief. (see at least [0027]: “In some embodiments, the autonomous AI NPCs operate consistent to a belief, desire, and intent model. An autonomous AI character receives sensor inputs. For example, the autonomous AI character detects movement, voices, and obstacles in the game environment. Based on the sensory input, the character updates its set of beliefs. For example, an autonomous AI character constructs a set of beliefs that are based on facts, including observed facts from its sensors, that correspond to its understanding of the game world. Facts may include its position and positions of other NPCs, objects, obstacles, etc. in the game world. Its beliefs may also include it's understanding of the beliefs of other NPCs. In addition to a set of beliefs, the autonomous AI character identifies one or more goals. Goals may include game objectives such as traveling from one location to another, speaking with the game player to inform her or him of relevant information, refueling a vehicle to increase its potential range, etc.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the belief updates as taught by Shinn. This is because as stated in [0003] of Shinn: “Many computer-based games use non-playing characters (NPCs) to simulate a dynamic environment. Typically, this is implemented using a decision tree and/or state machine. For example, NPCs and game events are often scripted to cycle between one of several states and are therefore limited in their range of variations. Moreover, multiple NPCs may share the same game logic and as a result will often appear to game players as duplicates of one another. In the case where there are multiple different NPCs, the different NPCs often have only superficial differences between one another and/or are one of a limited number of different variations of game logic. Implementing an NPC that acts independent from other NPCs using traditional artificial intelligence techniques is both processor and memory intensive. Therefore, there exists a need for a framework to support autonomous NPCs that utilizes both artificial intelligence (AI) and is scalable. Unlike traditional non-autonomous NPCs, autonomous AI agent characters may move and act independently from one another. Moreover, autonomous AI agent characters may be incorporated into a computer-based game such that their actions both respond to and also affect the game environment.” Shinn further states in [0070]: “In various embodiments, the process of FIG. 7 may be utilized to implement fully autonomous behavior for agents such as robots and non-player characters (NPCs) in a game.”
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Tesauro et al. (US 9047423 B2), hereinafter Tesauro.
Regarding claim 5, Gupta discloses:
The method of Claim 1.
Gupta does not explicitly disclose, but Tesauro, in the analogous field of endeavor of artificial intelligence decision making teaches:
wherein the updating of the at least one belief includes utilizing an Object-Oriented Partially Observable Monte-Carlo Planning process to update the at least one belief on a per object distribution basis (see at least col. 8 line 48-col. 9 line 2: “FIGS. 3A-3B show a method and system for Monte-Carlo planning of actions in the real-world environment. In FIG. 3A, at step 202, the computing system starts to operate. At step 204, the computing system builds and/or initializes a data structure corresponding to a root node of an MCTS-type data structure representing the real-world environment, where the state of the root node of the tree is initialized according to the current state of the live scheduling task, e.g., in one embodiment, a real-world job scheduling environment. The computing system may also be configured to create and initialize a number of intermediate nodes and/or leaf nodes of the MCTS-type data structure representing subsequent base states that may be reached starting from the root-node state in the real-world environment. The computing system may also be configured to create and initialize additional intermediate nodes and leaf nodes while running method steps 208-218 (e.g., created and initialized during execution steps of the trials simulated at 208). The data structure alternates between action nodes, where the computing system is configured to assign jobs to idle servers in the example presented above, and successor state nodes, where a job is completed and a server device becomes available.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the method of Tesauro because as stated in col. 3 line 66 – col. 4 line 5: “Hence, it would be desirable to provide a system and method implementing improved Monte-Carlo planning that reduces the nominal search complexity of standard Monte-Carlo Tree Search, and effectively exploits smooth dependence of expected cumulative reward on some or all of the observable state variables in a given real-word domain.”
Regarding claim 15, Gupta discloses:
The robotic system of Claim 14.
Gupta does not explicitly disclose, but Tesauro, in the analogous field of endeavor of artificial intelligence decision making teaches:
wherein the at least one non-transitory computer readable storage medium further comprises an Object-Oriented Partially Observable Monte-Carlo Planning module configured to update the at least one belief on a per object distribution basis (see at least col. 8 line 48-col. 9 line 2: “FIGS. 3A-3B show a method and system for Monte-Carlo planning of actions in the real-world environment. In FIG. 3A, at step 202, the computing system starts to operate. At step 204, the computing system builds and/or initializes a data structure corresponding to a root node of an MCTS-type data structure representing the real-world environment, where the state of the root node of the tree is initialized according to the current state of the live scheduling task, e.g., in one embodiment, a real-world job scheduling environment. The computing system may also be configured to create and initialize a number of intermediate nodes and/or leaf nodes of the MCTS-type data structure representing subsequent base states that may be reached starting from the root-node state in the real-world environment. The computing system may also be configured to create and initialize additional intermediate nodes and leaf nodes while running method steps 208-218 (e.g., created and initialized during execution steps of the trials simulated at 208). The data structure alternates between action nodes, where the computing system is configured to assign jobs to idle servers in the example presented above, and successor state nodes, where a job is completed and a server device becomes available.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the method of Tesauro because as stated in col. 3 line 66 – col. 4 line 5: “Hence, it would be desirable to provide a system and method implementing improved Monte-Carlo planning that reduces the nominal search complexity of standard Monte-Carlo Tree Search, and effectively exploits smooth dependence of expected cumulative reward on some or all of the observable state variables in a given real-word domain.”
Claims 7 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Chen (US 20180079081 A1), hereinafter Chen.
Regarding claim 7, Gupta discloses:
The method of Claim 1.
Gupta does not explicitly disclose, but Chen, in the analogous field of endeavor of robotic control teaches wherein the method further comprises notifying the user upon finding the at least one target object by providing an audible, visual, audiovisual, and/or electronic indication (see at least [0050]: “In that regard, in block 310, the robot controller may identify the object as a “hot” object. The robot controller may be designed to take separate actions with regard to “hot” objects as opposed to other objects for the robot to track. In some embodiments, the robot controller may be designed to notify the user each time a “hot” object is manipulated or moved from a current location. For example, the robot controller may transmit a message to a mobile device of the user indicating that the “hot” object has been moved along with a current location of the “hot” object. In some embodiments, the robot controller may cause an output device of the robot to output data indicating that the “hot” object has been moved and/or indicating a current location of the “hot” object.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the method of Chen because as stated in [0004]-[0005] of Chen: “There are some systems that have been designed to keep track of personal objects. For example, one such system includes a receiver to be connected to each object to be tracked. The system also includes a remote control that can wirelessly connect to the receiver. A user may use the remote control to transmit a signal to the receiver causing it to make a sound. The user may find the object by following the sound generated by the receiver. This type of system, however, may be undesirable as it requires the user to place the receiver on each object to track, making the objects relatively bulky. The systems also require the user to keep track of the remote electronic device. Thus, there is a need in the art for systems and methods for providing a robot that can take an inventory of an area and provide information regarding current locations of objects.”
Regarding claim 12, Gupta discloses:
The robotic system of Claim 11.
Gupta does not explicitly disclose, but Chen, in the analogous field of endeavor of robotic control teaches wherein the at least one non- transitory computer readable storage medium further comprises a notification module communicatively coupled to the at least one processor and configured to notify the user upon finding the at least one target object by providing an audible, visual, audiovisual, and/or electronic indication (see at least [0050]: “In that regard, in block 310, the robot controller may identify the object as a “hot” object. The robot controller may be designed to take separate actions with regard to “hot” objects as opposed to other objects for the robot to track. In some embodiments, the robot controller may be designed to notify the user each time a “hot” object is manipulated or moved from a current location. For example, the robot controller may transmit a message to a mobile device of the user indicating that the “hot” object has been moved along with a current location of the “hot” object. In some embodiments, the robot controller may cause an output device of the robot to output data indicating that the “hot” object has been moved and/or indicating a current location of the “hot” object.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the method of Chen because as stated in [0004]-[0005] of Chen: “There are some systems that have been designed to keep track of personal objects. For example, one such system includes a receiver to be connected to each object to be tracked. The system also includes a remote control that can wirelessly connect to the receiver. A user may use the remote control to transmit a signal to the receiver causing it to make a sound. The user may find the object by following the sound generated by the receiver. This type of system, however, may be undesirable as it requires the user to place the receiver on each object to track, making the objects relatively bulky. The systems also require the user to keep track of the remote electronic device. Thus, there is a need in the art for systems and methods for providing a robot that can take an inventory of an area and provide information regarding current locations of objects.”
Claims 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Pogorelik (US 20190050646 A1), hereinafter Pogorelik.
Regarding claim 8, Gupta discloses:
The method of Claim 1.
Gupta does not explicitly disclose, but Pogorelik, in the analogous field of object tracking and robotic control teaches wherein the method further comprises the step of generating a map of the environment (see at least [0012]: “In use, the autonomous device 102 may detect and identify one or more objects located in an environment surrounding the autonomous device 102 and generate an environment map that includes locations of the detected objects. To do so, the autonomous device 102 may detect a presence of an object in the environment based on environmental data collected from one or more input devices (e.g., a camera 134 and/or a light detection and ranging (LIDAR) sensor 136) and perform data analysis (e.g., an object recognition technique and/or simultaneous localization and mapping (SLAM)) to identify the detected object. It should be appreciated that, in some embodiments, deep learning and machine learning techniques may also be used to identify one or more objects detected in the environment.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Gupta with the object tracking of Pogorelik. This is because Pogorelik’s invention is directed to solving the problem described in the Background section of Pogorelik’s disclosure: “User interaction often requires a system to be familiar with an object of interest and allows users to use a semantic name associated with each object. When a system detects an unfamiliar object, an object recognition technique may be used to identify the unfamiliar object. However, object recognition is cost and performance expensive. As such, the system typically includes a limited number of objects that may be recognizable using the object recognition technique. Additionally, when the unfamiliar object is detected, the system may interrupt work flow and use a speech-based human interaction for resolving the issue. However, it may be difficult and lengthy to describe the unfamiliar object.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ELIZABETH NELESKI whose telephone number is (571)272-6064. The examiner can normally be reached 10 - 6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THOMAS WORDEN can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.R.N./Examiner, Art Unit 3658
/THOMAS E WORDEN/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights