DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Response to Amendment
Claims 1-3, 5, and 8 have been amended. Claim 7 has been canceled, and no claims have been added. The 35 U.S.C. 101 rejection has been withdrawn in view of amendment.
Response to Argument
Examiner has considered Applicant’s arguments filed 03/05/2026 and finds them unconvincing.
Applicant contends that Haughton fails to disclose “a scene modification which is a virtual change of the scene”. Examiner finds this assertion unconvincing, and believes that Applicant is narrowing the scope of the claimed limitation beyond reasonable interpretation. Adding that the scene modification is a virtual change of the scene does not mean that a physical change cannot also occur, but merely requires that a virtual change occurs, not that it occurs without a physical change also occurring.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-6 and 8-10 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Haughton et al. (GB2619519, referred to as Haughton).
Regarding claim 1: Haughton discloses: A method for assisting an operator in operating a telerobot system, the telerobot system comprising a robot, at least one sensor, a processing unit and an operator interface, the method comprising the steps of physically sensing a working area of the robot by the at least one sensor; detecting, by the processing unit, objects and estimating at least one of an object property, object state and object relation to another object; ([pg. 8, lines 16-28] the obtained scene representation model has been pre-trained by optimising the scene representation model so as to minimise a loss between a provided estimate of a value of the physical property of at least one object of the scene and the predicted value of the physical property of the at least one object. This may help provide a relatively accurate scene representation relatively quickly, for example as compared to starting with no information about the physical property value of any of the portions of the scene. These estimates may be based, for example, on typical ranges of stiffnesses of household objects, for example. Optionally, the estimate is provided by applying a pre-trained object detector to a captured image to identify the at least one object, and inferring the estimate from the identity of the at least one object. This may provide more accurate initial estimates of the physical properties. This may in turn reduce the time (and interactions) taken to generate a relatively accurate scene representation model.) estimating, by the processing unit, for at least one object, an actual scene uncertainty related to the object; predicting, by the processing unit, a modified scene uncertainty related to the object assuming a scene modification which is a virtual change of the scene; and determining, by the processing unit, whether the modified scene uncertainty has improved compared to the actual scene uncertainty and, if an improvement is determined, communicating the corresponding scene modification using the operator interface. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to
physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output
interface 606 may be used, for example, to output control instructions to cause the robot
208 to physically contact the at least one object 204 and/or to cause the robot 208 to
perform a tasks based on the updated scene representation model 438, for example.) and autonomously executing, by the robot, the scene modification corresponding to a determined improvement of the modified scene uncertainty, or executing, by the robot, control inputs received by a teleoperation interface in response to communicating the corresponding scene modification. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to
physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output
interface 606 may be used, for example, to output control instructions to cause the robot
208 to physically contact the at least one object 204 and/or to cause the robot 208 to
perform a tasks based on the updated scene representation model 438, for example.)
Regarding claim 2: Haughton discloses: The method according to claim 1,
Haughton further discloses: wherein the method further comprises a step of selecting, by the processing unit, one or more scene modifications based on a predicted future operation of the operator. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output interface 606 may be used, for example, to output control instructions to cause the robot 208 to physically contact the at least one object 204 and/or to cause the robot 208 to perform a tasks based on the updated scene representation model 438, for example.)
Regarding claim 3: Haughton discloses: The method according to claim 1,
Haughton further discloses: wherein a task is divided into subtasks and a sequence of the subtasks resulting in an improved uncertainty is communicated to the operator. ([pg. 28, lines 9-28] Updating the scene representation model 432 based on a mass or an estimate of 10 the mass of an object 204 may be useful assisting the robot 208 to carry out tasks. For example, a task may be defined to tidy up empty cups. However, a cup that is full of water may look identical to an empty cup. But a full cup will have a greater mass than an empty cup. Accordingly, the robot 208 may physically contact a cup (such as by performing a lift or a minor lateral push) to derive an estimate of the mass of the cup, 15 and update the scene representation model 432 based on the derived mass of the cup. The robot may then perform the defined task based on the updated scene representation model 432. For example, if the mass of the cup is low, the robot 208 may infer that the cup is empty and hence tidy up the cup, but if the mass is high the then the robot 208 may infer the cup is full and hence not tidy up the cup. As another example, similarly, 20 a task may be defined to sort boxes which are identical in appearance, but which have different masses, by mass. Accordingly, the robot may perform a lateral push or a lift on each box to determine an estimate of the mass, update the scene representation model 432 based on the masses, and carry out the task based on the updated scene
representation model 432. 25 In some examples, the physical contact of the robot with the at least one object 204 may comprise physical contact of a measurement probe 212 of the robot 208 with the at least one object 204, and the obtained value may be derived based on an output of the measurement probe when contacting the at least one object 204.)
Regarding claim 4: Haughton discloses: The method according to claim 1,
Haughton further discloses: wherein communicating scene modification includes outputting information on the determined uncertainty improvement. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output interface 606 may be used, for example, to output control instructions to cause the robot 208 to physically contact the at least one object 204 and/or to cause the robot 208 to perform a tasks based on the updated scene representation model 438, for example.)
Regarding claim 5: Haughton discloses: The method according to claim 1,
Haughton further discloses: wherein a cost for executing the scene modification is estimated and communicated to the operator. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output interface 606 may be used, for example, to output control instructions to cause the robot 208 to physically contact the at least one object 204 and/or to cause the robot 208 to
perform a tasks based on the updated scene representation model 438, for example.)
Regarding claim 6: Haughton discloses: The method according to claim 1,
Haughton further discloses: wherein an operator request is received specifying an actual scene uncertainty for which a modified scene uncertainty which is improved compared to the actual scene uncertainty shall be determined. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output interface 606 may be used, for example, to output control instructions to cause the robot 208 to physically contact the at least one object 204 and/or to cause the robot 208 to
perform a tasks based on the updated scene representation model 438, for example.)
Regarding claim 8: Haughton discloses: The method according to claim 1,
Haughton further discloses: wherein the operator interface outputs a scene representation an visualizes the actual scene uncertainty or the modified scene uncertainty. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output interface 606 may be used, for example, to output control instructions to cause the robot 208 to physically contact the at least one object 204 and/or to cause the robot 208 to perform a tasks based on the updated scene representation model 438, for example.)
Regarding claim 9: Haughton discloses: The method according to claim 1,
Haughton further discloses: wherein the scene uncertainty is an uncertainty of one of: pose of an object, object existence, object detection, object classification, classification of object instance, object state, relation between the object and at least one other object, prediction of dynamical object movements, traversability, graspability and at least one physical property of the object. ([pg. 4, lines 5-14] the method comprises selecting the at least one object from among a plurality of the one or more objects based on an uncertainty of the predicted value of the physical property of each of the plurality of objects; controlling the robot to physically contact the selected object; and deriving the value of the physical property of the selected object from the physical contact, thereby to obtain the value. This may allow for a robot to select autonomously (e.g. without input from a user) the object to physically contact that may result in the largest decrease in uncertainty in the scene representation model, and hence provide for the largest gain in accuracy and/or reliability of the scene representation model. This may provide for an accurate and/or reliable scene representation model to be autonomously created. [pg. 31, lines 1-6] The input interface 604 may be configured, for example, to obtain the scene representation model 432 (which may be stored in memory 604) and/or to obtain the value of the physical property of the at least one object, for example. The output interface 606 may be used, for example, to output control instructions to cause the robot 208 to physically contact the at least one object 204 and/or to cause the robot 208 to perform a tasks based on the updated scene representation model 438, for example.)
Regarding claim 10: Rejected using the same rationale as claim 1.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ATTICUS A CAMERON whose telephone number is 703-756-4535. The examiner can normally be reached M-F 8:30 am - 4:30 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Worden can be reached on 571-272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ATTICUS A CAMERON/ /JASON HOLLOWAY/ Primary Examiner, Art Unit 3658 Examiner, Art Unit 3658A
Prosecution Insights