DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
Claim Rejections - 35 USC § 102
Applicant’s arguments with respect to the independent claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
The rejections of the claims under 35 USC § 102 are withdrawn. The claims now stand rejected under Raaj et al. in view of Konolige et al. (US 20160016311 A1). See the updated rejection below.
It is unclear if Applicant is arguing that Raaj does not teach the limitation “at least one of the at least two sources comprising force feedback information”. Examiner notes that the present claim language remains particularly broad and that Raaj still discloses this limitation as illustrated in the new/updated rejection.
Claim Interpretation
The phrase “instructing the robot to perform a manipulation” has been interpreted as meaning a manipulation attempt is performed and not simply that instructions which might be used therefor are generated.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3, 5, and 23 – 24 are rejected under 35 U.S.C. 103 as being unpatentable over Raaj et al. (US 12521894 B1) further in view of Konolige et al. (US 20160016311 A1).
Regarding Claim 1, Raaj teaches:
A computer-implemented method comprising:
receiving, by a computing system (See at least computing components 178 and Figure 4) of a robot (See at least robot 100), (i) robot data reflecting at least a portion of the robot (See at least Column 8, Lines 4 – 6, “In at least some cases, the robot 100 monitors itself and/or its environment in real-time or in near real-time”), and (ii) object data reflecting at least a portion of an object (See at least Column 10, Lines 11 – 12, “The object estimator 212 can generate a current estimate of an object (e.g., a tote) within an environment”), the object data determined based on information from at least two sources (The nature of “two sources” is not further described or defined and consequently may be effectively two different things. Two alternative teachings which both read on the limitation are herein provided:
The “sources” are anything: See at least Column 10, Lines 8 – 10, “the object estimator 212 can receive information from the robot kinematic estimator 208 and the robot position estimator 210 directly” or
The “sources” are two different sensors: See at least Column 8, Lines 4 – 5, “The robot 100 can include one or more sensors in a sensor system”)
, at least one of the at least two sources comprising force feedback information (See at least Column 7, Lines 50 – 59, “At one, some, or all of the actuators and/or at one or more other suitable locations, the robot 100 can include among the sensor components 189, sensors that measure properties of the corresponding joints. Such properties can include … applied force (e.g., torque), … mass, … The measurements of these properties can be direct or indirect. As an example of direct sensing, the robot 100 may sense a torque acting on a given joint via a torque sensor operably associated with the joint”. Examiner notes that a manipulated object will reflect on the perceived sensor data of joints such and thus qualifies as “object data”);
determining, by the computing system, based on the robot data and the object data, a set of states of the object (Examiner notes that the nature of “states” is not further described or defined and can be interpreted potentially as the data items already disclosed above. However, in the interest of compact prosecution, the “states of the object” are presently mapped to “object reference”.
See again Column 10, Lines 8 – 10, as well as at least Column 10, Lines 33 – 37, “In at least some cases, the object estimator 212 is configured to update the world state 213 with object references, support structure references, and/or other information related to objects in an environment in which the robot 100 operates” and Column 19, Lines 42 – 45, “The object reference can be a corrected object reference, a first object reference, a second object reference, a new object reference, or an object reference of another suitable type”), each state in the set of states associated with a distinct time at which the object is at least partially supported by the robot, wherein the set of states includes at least three states associated with three distinct times (See again Column 19, Lines 42 – 45, as well as Columns 17 and 18 in general (the disclosure providing a very broad spectrum of teaching that all read on the limitation). As provided in Column 17, an initial object reference is generated and the methods may include “generating a corrected object reference more than once” (Lines 59 – 60) which inherently requires at minimum two other object references in addition to the initial/base reference at different times. In other words, at least three states at three distinct times), … ;
instructing, by the computing system, the robot to perform a manipulation of the object based, at least in part, on at least the one state in the set of states (See at least abstract, “the method includes controlling the mobile robot based at least partially on the object reference”);
determining based, at least in part, on a change to the at least one state, an adjustment to the manipulation of the object (See again Column 10, Lines 8 – 10, as well as at least Column 10, Lines 33 – 37, “In at least some cases, the object estimator 212 is configured to update the world state 213 with object references, support structure references, and/or other information related to objects in an environment in which the robot 100 operates” and Column 19, Lines 42 – 45, “The object reference can be a corrected object reference, a first object reference, a second object reference, a new object reference, or an object reference of another suitable type”, as well as if further clarity is required with respect to “change”, Column 17, Lines 16 – 18, “In addition or alternatively, shifting between the end effectors 122a, 122b and the object 300 can occur after the robot 100 grasps the object 300”); and
instructing the robot to implement the adjustment to the manipulation of the object (See again preceding clause. As necessary, see also Column 8, Lines 4 – 8, “In at least some cases, the robot 100 monitors itself and/or its environment in real-time or in near real-time. Moreover, the robot 100 may use acquired sensor data as a basis for decision-making via the computing components 178”).
Raaj does not teach, but in combination with Konolige (recited below) teaches:
…
wherein at least one state in the set of states comprises a mass of the object determined, at least in part, from the force feedback information (See at least [0028] In additional examples, a force-torque sensor mounted on the robotic arm may be used to help determine box properties. The force-torque sensor may be positioned above a suction gripper in order to determine force and/or torque caused by an object held by the gripper. Based on this data, the force-torque sensor may be used to determine parameters of a picked box, such as mass, center of gravity, center of mass, how static the center of mass is, and/or inertia matrix” and [0030] “some or all of the trajectory planning may be done on the fly (e.g., in real-time) as object measurements are determined. For instance, in one example, a conservative trajectory may be planned and used during a first period that allows for safe movement of objects at the extreme end of possible measurements (e.g., a very heavy box or a very long box). As object measurements become available, the trajectory may be modified to a more aggressive trajectory that allows for faster object transportation (e.g., of lighter or smaller boxes)”);
…
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to utilize the mass of the object in planning and modifying/adjusting robotic manipulations as taught by Konolige in the system of Raaj with a reasonable expectation of success. As is well understood by anyone with experience carrying objects, the mass of an object is of clear importance in manipulating the object. The use of object mass data in object manipulation will consequently result in more effective robotic manipulation attempts.
Regarding Claim 3, the combination of Raaj and Konolige teaches:
The method of claim 1,
Raaj further teaches:
wherein at least one state in the set of states comprises a position and an orientation of the object (See at least Column 2, Line 64 – Column 3, Line 2, “An object pose is one example of an object reference potentially useful for this purpose. An object pose includes information about how an object is situated in an environment. This can include information about the position and orientation of an object relative to a known reference, another object, the robot itself, the world, etc.” and Column 19, Lines 48 – 50, “The object reference can correspond to a pose (e.g., a six degree of freedom pose) of the object 300”).
Regarding Claim 5, the combination of Raaj and Konolige teaches:
The method of claim 1,
Raaj further teaches:
wherein at least one state in the set of states comprises a six-dimensional pose of the object (See at least Column 2, Line 64 – Column 3, Line 2, “An object pose is one example of an object reference potentially useful for this purpose. An object pose includes information about how an object is situated in an environment. This can include information about the position and orientation of an object relative to a known reference, another object, the robot itself, the world, etc.” and Column 19, Lines 48 – 50, “The object reference can correspond to a pose (e.g., a six degree of freedom pose) of the object 300”).
Regarding Claims 23 and 24, the claims are directed to effectively the same subject matter as Claim 1 with respect to the application of prior art. The only distinction from Claim 1 appears to be the specific recitation of generic computer components/structures, specifically “data processing hardware”, “memory hardware”, and “instructions”. Raaj readily teaches these additional features which are common knowledge. See at least “processor 179”, “memory 180”, persistent storage 181”, the associated data thereof, and Column 8, Lines 33 – 34 “Software features of the robot 100 may take the form of computer-executable instructions” as necessary.
The claims are therefore rejected under the same logic as Claims 1 above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW C GAMMON whose telephone number is (571)272-4919. The examiner can normally be reached M - F 10:00 - 6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ADAM MOTT can be reached on (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW C GAMMON/Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights