Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 101
1. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Independent claim 1 recites steps that assigning a first label to a first object, receiving a response from a query and assigning a second label to a second object. All of the recited steps are processes that, under its broadest reasonable interpretation, cover the limitations under the organized human activity.
The claim features under its broadest reasonable interpretation, are certain methods of organizing human activity performed by generic computer components. For example, but for the “assigning” [human behavior: allocating, giving], “sending” [human behavior: dispatching, conveying], and “receiving” [human behavior: obtaining, accepting], in the context of this claim encompasses methods of organized human activity. If the claim limitations, under its broadest reasonable interpretation, covers fundamental economic practice, commercial or legal interaction or managing personal behavior or relationships or interactions between people but for the recitation of generic computer components, then it falls within the "method of organized human activity" grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
"[A]fter determining that a claim is directed to a judicial exception, 'we then ask, [w]hat else is there in the claims before us?"' MPEP 2106.05 (emphasis in MPEP) citing Mayo, 566 U.S. at 78. "What is needed is an inventive concept in the non-abstract application realm." SAP Inc. v. lnvestPic, LLV, Appeal No. 2017-2081 (Fed. Cir. 2018). For step two, the examiner must "determine whether the claims do significantly more than simply describe [the] abstract method" and thus transform the abstract idea into patent-eligible subject matter. Ultramercial, Inc. v. Hutu, LLC, 772 F.3d 709 (Fed. Cir. 2014).
A primary consideration when determining whether a claim recites "significantly more" than abstract idea is whether the additional element(s) are well-understood, routine, conventional activities previously known to the industry. See MPEP 2106.0S{d). "If the additional element (or combination of elements) is a specific limitation other than what is well- understood, routine and conventional in the field, for instance because it is an unconventional step that confines the claim to a particular useful application of the judicial exception, then this consideration favors eligibility. If, however, the additional element {or combination of elements) is no more than well-understood, routine, conventional activities previously known to the industry, which is recited at a high level of generality, then this consideration does not favor eligibility." Id.
The Federal Circuit has held that "[w]hether something is well-understood, routine, and conventional to a skilled artisan at the time of the patent is a factual determination." Bahr, Robert (April 19, 2018). Changes in Examination Procedure Pertaining to Subject Matter Eligibility, Recent Subject Matter Eligibility Decision (Berkheimer v. HP, Inc.) citing Berkheimer at 1369. "As set forth in MPEP 2106.05(d)(I), an examiner should conclude that an element (or combination of elements) represents well-understood, routine, conventional activity only when the examiner can readily conclude that the element(s) is widely prevalent or in common use in the relevant industry. This memo [] clarifies that such a conclusion must be based upon a factual determination that is supported as discussed in section III [of the memo]." Berkheimer Memo at 3 (emphasis in memo).
Generally, "[i]f a patent uses generic computer components to implement an invention, it fails to recite an inventive concept under Alice step two." West View Research v. Audi, CAFC Appeal Nos. 2016-1947-51 (Fed. Cir. 04/19/2017) citing Mortg. Grader, Inc. v. First Choice Loan Servs. Inc., 811 F.3d 1314, 1324-25 (Fed. Cir. 2016) (explaining that "generic computer components such as an 'interface,' 'network,' and 'database' ... do not satisfy the inventive concept requirement"; but see Bascom (finding that an inventive concept may be found in the non-conventional and non-generic arrangement of the generic computer components, i.e., the installation of a filtering tool at a specific location, remote from the end- users, with customizable filtering features specific to each end user).
In accordance with the above guidance, the examiner has searched the claim(s) to determine whether there are any "additional elements" in the claims that constitute "inventive concept," thereby rendering the claims eligible for patenting even if they are directed to an abstract idea. Alice, 134 S. Ct. 2347 (2014). Those "additional features" must be more than "well understood, routine, conventional activity." See Alice. To note, "under the Mayo/Alice framework, a claim directed to a newly discovered ... abstract idea [] cannot rely on the novelty of that discovery for the inventive concept necessary for patent eligibility." Genetic Techs. Ltd v. Merial LLC, 818 F.3d 1369, 1376 (Fed. Cir. 2016); Diamond v. Diehr, 450 U.S. 175, 188-89 (1981).
As an example, the Federal Circuit has indicated that "inventive concept" can be found where the claims indicate the technological steps that are undertaken to overcome the stated problem(s) identified in Applicant's originally-filed Specification. See Trading Techs. Inc. v. CQG, Inc., No. 2016-1616 (Fed. Cir. 2017); but see IV v. Erie Indemnity, No. 2016-1128 (Fed. Cir. March 7, 2017) ("The claims are not focused on how usage of the XML tags alters the database in a way that leads to an improvement in technology of computer databases, as in Enfish.") (emphasis in original) and IV. v. Capital One, Nos. 2016-1077 (Fed. Cir. March 7, 2017) ("Indeed, the claim language here provides only a result-oriented solution, with insufficient detail for how a computer accomplishes it. Our law demands more. See Elec. Power Grp., 830 F.3d 1356 (Fed. Cir. 2016) (cautioning against claims 'so result focused, so functional, as to effectively cover any solution to an identified problem.')"). Furthermore, "[a]bstraction is avoided or overcome when a proposed new application or computer-implemented function is not simply the generalized use of a computer as a tool to conduct a known or obvious process, but instead is an improvement to the capability of the system as a whole." Trading Techs. Int'l, Inc. v. CQG, Inc., No. 2016-1616 (Fed. Cir. 2017) (emphasis added).
In the search for inventive concept, the Berkheimer Memo describes "an additional element (or combination of elements) is not well-understood, routine or conventional unless the examiner finds, and expressly supports a rejection in writing with, one or more of the following:
A citation to an express statement in the specification or to a statement made by an applicant during prosecution that demonstrates the well-understood, routine, conventional nature of the additional element(s).
A citation to one or more of the court decisions discussed in the MPEP as noting the well-understood, routine, conventional nature of the additional element(s).
A citation to a publication that demonstrates the well-understood, routine, conventional nature of the additional element(s).
A statement that the examiner is taking official notice of the well-understood, routine, conventional nature of the additional element(s).
See Berkheimer Memo at 3-4.
Accordingly, the examiner refers to the following generically-recited computer elements with their associated functions (and associated factual finding(s)), which are considered, individually and in combination, to be routine, conventional, and well-understood:
“a method of operation of a robotic system, the robotic system comprising a robot, an object recognition subsystem, and an interface to a large language model (LLM), the robot operating in an environment, the environment comprising a plurality of objects, the plurality of objects including a first object and a second object, the method comprising”
As set forth in MPEP § 2106.0S(d)(I), an examiner should conclude that an element (or combination of elements) represents well-understood, routine, conventional activity only when the examiner can readily conclude that the element(s) is widely prevalent or in common use in the relevant industry. The Berkhiemer memo clarifies that such a conclusion must be based upon a factual determination that is supported as discussed in section III the memo. As seen in paragraphs ([28, 99, 104, 111]) of the instant Specification and Symantec.. 838 F.3d at 1.321, 110 USPQ2d at. 1362, the elements are viewed to be well-understood, routine and conventional.
In sum, the Examiner finds that the claims "are directed to the use of conventional or generic technology in a nascent but well-known environment, without any claim that the invention reflects an inventive solution to any problem presented by combining the two." In re TLI Communications LLC, No. 2015-1372 (May 17, 2016). Similar to the claims in SAP v. lnvestPic, "[t]he claims here are ineligible because their innovation is an innovation in ineligible subject matter." Appeal No. 2017-2081 (Fed. Cir. 2018). In other words, "the advance lies entirely in the realm of abstract ideas, with no plausibly alleged innovation in the non-abstract application realm." Id. Accordingly, when considered individually and in ordered combination, the examiner finds the claims to be directed to in-eligible subject matter.
Next, it is determined whether the claim integrates the judicial expectation into a practical application by identifying whether “any additional elements recited in the claim beyond the judicial exception(s)” and evaluate those elements to determine whether the integrate the judicial exception into a recognized practical application.
In this case, the additional elements do not integrate the judicial application into a practical application. The claim does not recite (i) an improvement to the functionality of a computer or other technology or technical field ; (ii) a "particular machine" to apply or use the judicial exception; (iii) a particular transformation of an article to a different thing or state; or (iv) any other meaningful limitation.
The additional elements beyond the judicial exception are by a robot, an object recognition subsystem, an interface to a large language model. Using a computing device to identify and determine a value and disposition of an object is merely applying the judicial exception using a generic computing component. Additionally, the claim identifies and determines a value and disposition of an object - the claim does not improve the functioning of the computing device, or other technology or field.
The claims do not recite specific limitations (alone or when considered as an ordered combination) that were not well understood, routine, and conventional. As set forth in the Specification, the disclosed subject matter can be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter.
Dependent claims 2-20 include further recited limitations, do not integrate the abstract idea into a practical application, and the additional elements taken individually and in combination, do not contribute to an inventive concept, In other words, the dependent claims are directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 103
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 1, 10-11, 14, 16-17, 19 are rejected under 35 U.S.C. 103 as being unpatentable over submitted prior art Hausman et al. (2023/0311335).
As to claim 1, Hausman teaches a method of operation of a robotic system, the robotic system comprising a robot (robot 110 in Fig. 1A), an object recognition subsystem ([0058] - the current vision data instance 180, using vision component 111), and an interface to a large language model (LLM) (Figs. 2A & 2B – LLM 150), the robot operating in an environment ([0005-0006, 0014]), the environment comprising a plurality of objects, the plurality of objects including a first object and a second object (Figs. 2A & 2B and [0062, 0065] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops”), the method comprising:
assigning, by the object recognition subsystem ([0058] - the current vision data instance 180, using vision component 111), a first label to the first object ([0062] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops”);
sending, by the interface, a query to the LLM, the query comprising the first label ([0062], [0074] - generates an LLM prompt 205B based on the FF NL input 105 (“bring me a snack from the table”) and further based on the selected skill descriptor 201B (“go to the table”), for robotic skill A, based on robotic skill A being selected and provided for implementation. The LLM engine 130 can generate the LLM prompt 205B such that it conforms strictly to the FF NL input 105 and the selected skill descriptor 201B or can generate the LLM prompt 205B such that it is based on, the FF NL input 105 and the selected skill descriptor 201B. For example, as illustrated by LLM prompt 205B1, a non-limiting example of LLM prompt 205B, the LLM prompt can be “How would you bring me a snack from the table? I would 1. Go to the table. 2.”. Such LLM prompt 205B1 includes “How would you” as a prefix and “2.” as a suffix, either or both of which can encourage prediction, in LLM output, of step(s) that are relevant to achieving the high-level task specified by the FF NL input 105. Further, such LLM prompt 205B1 includes the selected skill descriptor 201B of “go to the table”, preceded by “1.”); and
assigning, by the object recognition subsystem, the second label to the second object ([0062] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops”).
Hausman does not explicitly discuss receiving, the interface, a response from the LLM, the response in reply to the query, the response comprising a second label.
However, Hausman teaches the LLM engine 130 can generate the LLM prompt 205A to incorporate one or more of such descriptors. For instance, the LLM prompt 205A can be “A pear, keys, a human, a table, a sink, and countertops are nearby. How would you bring the human a smack from the table. I would 1.” ([0062]); the LLM engine 130 can optionally generate the LLM prompt 205B further based on one or more of scene descriptor(s) 202B of a current environment of the robot 110 (Fig. 2B and [0075]).
Hence, it would have been obvious that the LLM engine can generate the LLM prompt further with more descriptors or a second label in order to assign to the second object in the environment such as a table, a sink, a trashcan, a bottle, a banana, a pear, keys… for additional skills in various implementations.
As to claims 10 and 16, Hausman teaches the method of claims 1 and 14, wherein the sending, by the interface, a query to the LLM includes formulating a natural language statement comprising the first label (at least Fig. 2B and [0074] – LLM prompt 205B, FF NL input 105 (“bring me a snack from the table”); Fig. 2B and [0075] – one or more of scene descriptor(s) 202B of a current environment of the robot 110).
As to claims 11 and 17, Hausman teaches the method of claims 10 and 16, wherein the formulating a natural language statement includes structuring the natural language statement to cause the response from the LLM to follow a defined structure (at least [0063] - " For example, the prior LLM prompt could be “explain how you would bring me a snack from the table”, and the explanation 204A can be generated based on the highest probability decoding from the prior LLM output. For instance, the explanation 204A can be “I would find the table, then find a snack on the table, then bring it to you”. The explanation 204A can be prepended to the LLM prompt 205A, replace term(s) of the FF NL input 105 in the LLM prompt 205A, or otherwise incorporated into the LLM prompt 205A. ").
As to claim 14, Hausman teaches the method of claim 1, wherein the assigning, by the object recognition subsystem, a first label to the first object includes:
identifying the first object ([0058] - the current vision data instance 180, using vision component 111. The vision data instance 180 captures a pear 184A and keys 184B that are both present on the round table represented by feature 194); and
assigning a natural language label to the first object ([0062] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops”).
As to claim 19, Hausman teaches the method of claim 1 comprising assigning, by the object recognition subsystem ([0058] - the current vision data instance 180, using vision component 111), a third label to the second object ([0062] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops”). Hausman does not explicitly discuss comparing the second label with the third label. However, Hausman teaches a task-grounding measure for a robotic skill, an NL skill description of the robotic skill can be compared to the LLM output to generate the task-grounding measure. For example, the task-grounding measure can reflect the probability of the NL skill description in the probability distribution that is modeled by the LLM output. For instance, and continuing with the working example, the task-grounding measure for a first robotic skill having an NL skill description of “pick up a sponge” can reflect a higher probability than does the task-grounding measure for a second robotic skill having an NL skill description of “pick up a banana”. Put another way, the probability distribution of the LLM output can characterize a higher probability for “pick up a sponge” than for “pick up a banana”. Also, for instance, and continuing with the working example, the task-grounding measure for a third robotic skill having an NL skill description of “pick up a squeegee” can reflect a probability that is similar to that of the first robotic skill ([0009]); and determining to implement the robotic skill in lieu of the additional robotic skill based on comparison of the overall measure to the additional overall measure ([0114]) ; and determining to implement the robotic skill based on the comparing. In some versions of those implementations, the overall measure is a weighted or non-weighted combination of the task-grounding measure and the world-grounding measure ([0118]). It would have been obvious to comparing the second label with the third label in order to have the LLM output to generate the task-grounding measure can reflect the probability of the NL skill description in the probability distribution that is modeled by the LLM output that can reflect a higher probability than does the task-grounding measure for a second robotic skill having an NL skill description of a second task label.
3. Claims 2-4, 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over submitted prior art Hausman et al. (2023/0311335) in view of Calenzo et al. (2019/0065888).
As to claim 2, Hausman teaches the method of claim 1, the object recognition subsystem comprising a plurality of sensors and sensor data processor ([0101] - The robot 520 includes a robot control system 560, one or more operational components 540a-540n, and one or more sensors 542a-542m. The sensors 542a-542m may include, for example, vision sensors, …, proximity sensors, and so forth. While sensors 542a-m are depicted as being integral with robot 520, this is not meant to be limiting. In some implementations, sensors 542a-m may be located external to robot 520, e.g., as standalone units), the method further comprising: scanning the environment, by the plurality of sensors, to generate sensor data; and the vision sensor data and/or determinations made from processing the vision sensor data (e.g., object detections ([0010]); [0123] - The current environmental state data includes sensor data captured by one or more sensor components of the robot in a current environment of the robot; [0062] – at least the scene descriptor(s) 202A include NL descriptor(s) of object(S) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning models).
Hausman does not explicitly discuss detecting by the sensor data the presence of the first object and the second object based at least in part on the sensor data.
Calenzo teaches referring to FIG. 4, which represents an exemplary embodiment of a device 10 for detecting the presence or the absence and motion of at least one object in the cubic-shaped housing 8 ([0065]); The device 10 for detecting presence and motion of an object includes the sensor 6 and a second sensor 11 identical to the sensor 6 configured to measure several distances so as to establish in real time a three-dimensional scene (i.e. a distance) ([0066]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Calenzo into the teachings of Hausman for the purpose of detecting the presence or the absence and motion of at least one object in the housing.
As to claim 3, Hausman teaches the method of claim 2, wherein the assigning, by the object recognition subsystem, a first label to the first object includes: identifying the first object based at least in part on the sensor data ([0062] – at least the scene descriptor(s) 202A include NL descriptor(s) of object(S) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning models); and assigning a natural language label to the first object ([0062] – NL descriptor(s) of object(S) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning models; the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops” and the LLM engine 130 can generate the LLM prompt 205A can be “A pear, keys, a human, a table, a sink, and countertops are nearby. How would you bring the human a snack from the table. I would 1.”; [0074-0075] and Fig. 2B).
As to claim 4, Hausman teaches the method of claim 3, wherein the sending a query to the LLM incudes formulating a natural language statement, the natural language statement comprising the natural language label assigned to the first object ([0062], [0074] - generates an LLM prompt 205B based on the FF NL input 105 (“bring me a snack from the table”) and further based on the selected skill descriptor 201B (“go to the table”), for robotic skill A, based on robotic skill A being selected and provided for implementation. The LLM engine 130 can generate the LLM prompt 205B such that it conforms strictly to the FF NL input 105 and the selected skill descriptor 201B or can generate the LLM prompt 205B such that it is based on, the FF NL input 105 and the selected skill descriptor 201B. For example, as illustrated by LLM prompt 205B1, a non-limiting example of LLM prompt 205B, the LLM prompt can be “How would you bring me a snack from the table? I would 1. Go to the table. 2.”. Such LLM prompt 205B1 includes “How would you” as a prefix and “2.” as a suffix, either or both of which can encourage prediction, in LLM output, of step(s) that are relevant to achieving the high-level task specified by the FF NL input 105. Further, such LLM prompt 205B1 includes the selected skill descriptor 201B of “go to the table”, preceded by “1.”).
As to claim 6, Hausman teaches the method of claim 2, wherein the scanning the environment, by the plurality of sensors, to generate sensor data includes generating at least one of image data, video data, audio data, or haptic data (0055, 0101]).
As to claim 7, Calenzo teaches the method of claim 2, wherein the detecting, by the sensor data processor, the presence of the first object and the second object in real time ([0065-0066]).
4. Claims 5, 8 are rejected under 35 U.S.C. 103 as being unpatentable over submitted prior art Hausman et al. (2023/0311335) and Calenzo et al. (2019/0065888) in view of Brantner (2022/0014476).
As to claim 5, Hausman and Calenzo do not explicitly discuss the method of claim 3, further comprising determining a degree of confidence in identifying of the first object exceeds a determined confidence threshold includes determining a probability.
Brantner teaches determining that the probability of the object coming loose exceeds a threshold associated with the confidence requirement ([0200]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Brantner into the teachings of Hausman and Calenzo for the purpose of determining that the level does not satisfy a confidence requirement may be carried out by determining based on the sensor input that the object has a particular probability of coming loose from the grasp of the autonomous device.
As to claim 8, Hausman teaches the method of claim 2 comprising assigning, by the object recognition subsystem ([0058] - the current vision data instance 180, using vision component 111), a third label to the second object ([0062] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops”) includes: identifying the second object based at least in part on the sensor data ([0062] – at least the scene descriptor(s) 202A include NL descriptor(s) of object(S) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning models); and Brantner teaches determining a degree of confidence in the identifying of the second object fails to exceed a determined confidence threshold ([0200] - determining that the level does not satisfy the confidence requirement may be carried out by determining that the probability of the object coming loose exceeds a threshold associated with the confidence requirement).
5. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Hausman, Calenzo, and Brantner in view of Ji et al. (2020/0365148).
As to claim 9, Hausman, Calenzo, and Brantner do not explicitly discuss the method of claim 8, wherein the assigning, by the object recognition subsystem, the second label to the second object includes updating the degree of confidence in the identifying of the second object.
Ji teaches the domain-specific confidence weight can be a predetermined value, and can be updated based on machine learning and user experience ([0063]) and AMCA 340 updates the weather domain confidence score by adding the weather-domain confidence weights of “0.8” of the subsequent/second keyword “degrees” ([0113]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Ji into the teachings of Hausman, Calenzo, and Brantner for the purpose of updating a domain-specific confidence score to meet the domain-specific acceptance criteria by adding an enhanced (i.e., intensified or increased) domain-specific confidence weight to the domain-specific confidence score.
6. Claims 12-13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over submitted prior art Hausman et al. (2023/0311335) in view of Xu et al. (2023/0103305).
As to claims 12 and 18, Hausman teaches the method of claim 1, wherein the receiving, by the interface, a response from the LLM includes: receiving a natural language statement comprising a natural language label ([0062] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops” and the LLM engine 130 can generate the LLM prompt 205A to incorporate one or more of such descriptors).
Hausman does not explicitly discuss parsing the natural language statement to extract the natural language label.
However, Hausman teaches ([0065] - The task grounding engine 132 generates task-grounding measures 208A and generates the task-grounding measures 208A based on the LLM output 206A and skill descriptions 207. Each of the skill descriptions 207 is descriptive of a corresponding skill that the robot 110 is configured to perform. For example, “go to the table” can be descriptive of a “navigate to table” skill that the robot can perform by utilizing a trained navigation policy with a navigation target of “table” (or of a location corresponding to a “table”)… Such additional skills can correspond to alternative objects and/or can be for different types of robotic action(s) (e.g., “place”, “push”, “open”, “close”); and [0071] - The selection engine 136 considers both the world-grounding measures 211A and the task-grounding measures 208A in selecting robotic skill A (“go to the table”) and sends an indication 213A of the selected robotic skill A to the implementation engine 136. In response, the implementation engine 136 controls the robot 110 based on the selected robotic skill A. For example, the implementation engine 136 can control the robot using a navigation policy with a navigation target of “table” (or of a location corresponding to a “table”)).
Xu teaches utilizing a natural language model to extract entity labels from a description of the digital image ([0014]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Xu into the teachings of Hausman for the purpose of matching system obtains an ungrounded label graph for a digital image.
As to claim 13, Hausman teaches the method of claim 12, wherein the assigning, by the object recognition subsystem, a second label to the second object includes assigning the natural language label to the second object ([0062] - The scene descriptor(s) 202A can include NL descriptor(s) of object(s) currently or recently detected in the environment with the robot 110, such as descriptor(s) of object(s) determined based on processing image(s) or other vision data using object detection and classification machine learning model(s). For example, the scene descriptor(s) 202A can include “pear”, “keys”, “human”, “table”, “sink”, and “countertops”).
7. Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over submitted prior art Hausman et al. (2023/0311335) in view of Brantner (2022/0014476).
As to claim 15, Hausman does not explicitly discuss the method of claim 14, further comprising determining a degree of confidence in identifying of the first object exceeds a determined confidence threshold includes determining a probability.
Brantner teaches determining that the probability of the object coming loose exceeds a threshold associated with the confidence requirement ([0200]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Brantner into the teachings of Hausman for the purpose of determining that the level does not satisfy a confidence requirement may be carried out by determining based on the sensor input that the object has a particular probability of coming loose from the grasp of the autonomous device.
8. Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Hausman in view of Ji et al. (2020/0365148).
As to claim 20, Hausman does not explicitly discuss the method of claim 19, wherein the assigning, by the object recognition subsystem, the second label to the second object includes updating the degree of confidence.
Ji teaches the domain-specific confidence weight can be a predetermined value, and can be updated based on machine learning and user experience ([0063]) and AMCA 340 updates the weather domain confidence score by adding the weather-domain confidence weights of “0.8” of the subsequent/second keyword “degrees” ([0113]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Ji into the teachings of Hausman for the purpose of updating a domain-specific confidence score to meet the domain-specific acceptance criteria by adding an enhanced (i.e., intensified or increased) domain-specific confidence weight to the domain-specific confidence score.
Conclusion
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUYNH H NGUYEN whose telephone number is (571)272-7489. The examiner can normally be reached Monday-Thursday 7:30AM-5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached on 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUYNH H NGUYEN/Primary Examiner, Art Unit 2693