Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This is a non-final Office Action on the merits. Claims 1-20 are currently pending and are addressed below.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-6 and 8-11 of copending Application No. 18991647 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because they comprise substantially the same claim limitations, differing essentially only by statutory category.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-7, and 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Bollini et al. (US 2022/0241980) in view of Livatino et al. (Intuitive Robot Teleoperation Through Multi-Sensor Informed Mixed Reality Visual Aids).
Regarding claim 1:
Bollini teaches control system for controlling operation of a robot in an environment, comprising:
a plurality of sensors configured to convert information from the environment and the robot into sensor data (see at least sensor system 130, Fig. 1B, ¶0007-0008, abstract)
a plurality of actuators configured to cause movement of the robot (actuators, motors M, see at least ¶0036, ¶0041);
a cognitive architecture control system, communicatively coupled to the robot, configured to control the robot (The Examiner notes that, absent any explicit limitations, Bollini’s teaching of building an environmental model for the robot to understand its surroundings meets the broadest reasonable interpretation of a cognitive architecture control subsystem.), and to:
receive the sensor data from the plurality of sensors (see at least sensor system 130, Fig. 1B, ¶0007-0008, abstract.);
generate a model of the environment from the sensor data (see at least sensor system 130, Fig. 3F, 4, ¶0007, ¶0062); and
output autonomous actuator data to the plurality of actuators of the robot, based on at least one instruction (see at least ¶0041-0043).; and
a graphical user interface configured to display a graphical representation of the robot-egocentric model to enable a human operator to select the at least one instruction from a pre-determined instruction set, based on the robot-egocentric model, to control the robot semi-autonomously (see at least Fig. 3D, E, F, 4, S406, ¶0027-0029, ¶0060, menu of behaviors, ¶0062, receive input selection of behavior, see at least ¶0005-0007, ¶0055-0063).
Bollini does not explicitly teach that the displayed model is an ego-centric model.
However, Bollini does further teach wherein the model may be displayed in an augmented reality representation, which at least suggests an ego-centric representation, and further discloses an ego-centric display in at least Fig. 3C.
Therefore, It would have been obvious to one of ordinary skill in the art before the time of filing of the invention to modify the robotic control system and method as taught by Bollini by implementing the model in an ego-centric view in order to provide the expected benefit of providing additional modelled data in an intuitive manner for a user to operate the robot.
Alternately, or in addition, Livatino teaches a system and method of robotic teleoperation, including presenting a user interface with an ego-centric model of the robot environment (see at least video-synthetic images, p. 25797-25799, Extended camera view p. 25800).
It would have been obvious to one of ordinary skill in the art at the time of filing of the invention to modify the robotic control system and method as taught by Bollini with the ego-centric model as taught by Livatino in order to provide sensor-information presentation that appears coherent and intuitive, making it easier for an operator to catch and comprehend information meaning. This increases situational awareness and speeds up decision-making (Livatino, abstract).
Regarding claim 3:
Bollini further teaches wherein the graphical user interface enables the human operator to select, through the graphical user interface, at least one detected object within the robot-egocentric model, wherein the pre-determined instruction set is based on the selected at least one detected object (see at least ¶0059-0060).
Regarding claim 4:
Bollini further teaches wherein the pre-determined instruction set is based on a context of the at least one detected object and the robot-egocentric model (see at least ¶0059-0060).
Regarding claim 5:
Bollini further teaches wherein the pre-determined instruction set is displayed as a menu (see at least ¶0060).
Bollini does not explicitly teach a drop-down menu. However, the Examiner notes that the pop up menu as taught by Bollini is a known, functional equivalent to the well-known drop-down menu, therefore, it would have been an obvious matter of design choice to substitute any known menu type as desired.
Regarding claim 6:
Bollini teaches wherein the cognitive architecture control system includes a feature extraction module which is configured to receive at least one sensor data stream from the robot and convert the at least one sensor data stream to features, wherein the features are semantically meaningful information, and wherein the features are used to generate the robot-egocentric model (see at least ¶0045).
Regarding claim 10:
Bollini further teaches wherein the cognitive architecture control system includes a concrete state representation updater which provides a state representation of a current state of the environment as understood by the cognitive architecture control system (see at least ¶0035).
Regarding claims 11, 13-16, and 20, Bollini teaches a computer program product comprising a non-transitory processor-readable storage medium storing processor-executable instructions and/or data that, when executed by at least one processor of a robot control system, cause the robot control system to perform a control as in claims 1, 2-6, and 10 above.
Claim Rejections - 35 USC § 103
Claims 2, 10, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Bollini and Livatino and further in view of Gu et al. (US 2009/0089700).
Regarding claims 2 and 12:
Bollini and Livatino teaches the limitations as in claim 1 above.
The combination does not explicitly teach switching between physical and simulated robots.
Gu teaches a system and method of integrating real and virtual robotic systems including wherein the control system includes a control-determining unit and the method further comprises switching control between a physical robot in a physical environment and a simulated robot in a simulated environment, wherein:
the plurality of sensors of the physical robot comprise a plurality of physical sensors generating physical sensor data from the physical environment (real sensors 118), and the plurality of actuators of the physical robot comprises a plurality of physical actuators receiving physical actuator data (real actuators 116);
the plurality of sensors of the simulated robot comprises a plurality of simulated sensors generating simulated sensor data from the simulated environment (virtual sensors 128), and the plurality of actuators of the simulated robot comprises a plurality of simulated actuators receiving simulated actuator data (virtual actuator, 130, see at least abstract, ¶0011-0012, Fig. 5, ¶0045-0054).
It would have been obvious to one of ordinary skill in the art at the time of filing of the invention to modify the robotic control system and method as taught by Bollini and Livatino with the simulation technique as taught by Gu in order to provide the expected result of simplifying field testing of a robot system, and validating commands in simulation prior to real-world execution.
Regarding claims 10 and 20:
Gu further teaches testing, by the cognitive architecture control subsystem, actuator data within the robot-egocentric model to determine the effects of an actuator data driven action before sending the actuator data to the robot (see at least Fig. 1, ¶0033-39).
Claim Rejections - 35 USC § 103
Claims 7-8 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Bollini Livatino in view of Sarvadevabhatla (US 2011/0004341)
Regarding claims 7-8 and 17-18:
Bollini in view of Livatino teaches the limitations as in claim 6 above.
The combination is silent as to feature extraction.
Sarvadevabhatla teaches a system and method of robotic control and environmental mapping including a feature extraction module including a plurality of specialized submodules to each extract a feature from the at least one sensor data stream and further comprising an attention module which is configured to turn on and off at least one of the specialized submodules (see at least abstract, ¶0003, ¶0028-0031, ¶0046-0069).
It would have been obvious to one of ordinary skill in the art at the time of filing of the invention to modify the robot control system and method as taught by Bollini and Livatino with the attention and feature extraction modules as taught by Sarvadevabhatla in order to provide the well-known machine vision technique of utilizing attention modules for directing the image processing resources to the most salient features for a particular task.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN J RINK whose telephone number is (571)272-4863. The examiner can normally be reached M-F 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anna Momper can be reached on (571) 270-5788. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ryan Rink/ Primary Examiner, Art Unit 3619
Prosecution Insights