DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 10-18 as originally filed are pending and have been considered as follows.
Priority
1. Acknowledgement is made that the present application is a national phase conversion of PCT/EP2023/062224 filed on 05/09/2023, which claims priority to EP22187200.5 filed on 07/27/2022 and EP22174203.4 filed on 05/15/2022.
Information Disclosure Statement
2. The information disclosure statement (IDS) filed on 11/18/2024 is being considered by the examiner.
Specification
3. The disclosure is objected to because of the following informalities:
Paragraph [0095], “external location marking 36” should read “external location marking 66” to be consistent with Fig. 5 and the rest of the specification.
Appropriate correction is required.
Claim Objections
4. Claims 17 and 18 are objected to because of the following informalities:
Claim 17 is a method claim and should be re-written in independent format with other limitations of claim 10.
• In claim 18, “an external location marking (36)” should be replaced with “an external location marking (66)” since paragraph [0094] and Fig. 5 indicates “an external location marking 66”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 10-13 and 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Halvorsen et al. (US 20180326507, hereinafter Halvorsen) in view of Hertzman (US 7474760, hereinafter Hertzman).
Regarding claim 10, Halvorsen teaches a construction robot (10) for carrying out construction work on a construction site object (see at least Fig. 1 and Abstract: “A robotic drilling apparatus is described which has been adapted for drilling holes in ceilings and walls on a construction site.”), comprising:
a mobile platform (12) (see at least Fig. 1 and [0068]: “A base 116 of the substructure 112 may be comprise driven wheels 117 to enable it to be manoeuvred around a construction site easily.”);
a manipulator (18) which is movable relative to the mobile platform (12) (see at least [0065]: “In this embodiment, there is provided a robotic drilling apparatus 100 having a robotic arm 110, with one end (a base end 110a) mounted to a substructure 112 and a moveable end 110b of the robotic arm 110 connected to a mount 120 for holding a drilling device 122.”);
an optical sensor system (62) which is at least partially disposed on the mobile platform (12) (see at least Fig. 1 and [0090]: “A vision system 136 may also be attached adjacent to the support structure 130.”); and
a control unit (38) which is configured to determine, using the optical sensor system (62), a position and/or an orientation of a tool disposed on the manipulator (18) (see at least Fig. 8b and [0140]: “The robot server 803 is able to calculate the precise position of the robotic drilling apparatus 100 and its drilling device 122, and can keep track of all the robotic drilling apparatus' movements. It can feedback statuses of its current position to the operator via the computing device 801.”; [0141]: “The current position may be calculated based on inputs from a laser total station 810 and a vision system 811 (preferably located on the robotic drilling apparatus 100). The total laser station 810 may be provided in the vicinity of the robotic drilling apparatus 100 on the construction site floor (or wall), close to where the drilling is to occur.”).
Halvorsen fails to explicitly teach determine, using the optical sensor system (62), a position of a tool relative to the mobile platform (12).
However, Hertzman teaches a method and apparatus that contactless measurement of a tool that determine, using the optical sensor system (62), a position of a tool relative to the mobile platform (12) (see at least Figs. 1, 3, 4, Col. 8, lines 47-51, and Col. 9, lines 4-14: position and/or orientation of the tool relative to the body of the machine is determined using a camera system.).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halvorsen to incorporate the teachings of Hertzman and provide a means to determine, using the optical sensor system, a position of a tool relative to the mobile platform, with a reasonable expectation of success, in order have full picture of the tool’s position by tracking the tool with respect to the body of the robot.
Regarding claim 11, modified Halvorsen teaches the limitations of claim 10. Halvorsen further teaches a non-optical sensor system (64) (see at least Fig. 1, 8b, [0138]: “The robot server 803 may also receive data from an inertial measurement unit (IMU) 812 as shown in the similar arrangement of FIG. 8b.”),
wherein the control unit (38) is configured to determine, using the non-optical sensor system (64), a position and/or an orientation of the tool disposed on the manipulator (18) (see at least [0047]: “The robotic drilling apparatus may also include an inertial measurement unit (IMU) which provides feedback measurements to the robot server. The feedback measurements may be used to generate a compensation in terms of a displacement and rotation of the drill to account for an amount of sway detected by the IMU.”; [0142]: “In some situations, where efficiency is a priority, an inertial measurement unit (IMU) 812 can deliver position and orientation data such as yaw and distance travelled, to the robot server 803.”).
Halvorsen fails to explicitly teach determine a position of a tool relative to the mobile platform (12).
However, Hertzman teaches a method and apparatus that contactless measurement of a tool that determine a position of a tool relative to the mobile platform (12) (see at least Figs. 1, 3, 4, Col. 8, lines 47-51, and Col. 9, lines 4-14: position and/or orientation of the tool relative to the body of the machine is determined using a camera system.).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halvorsen to incorporate the teachings of Hertzman and provide a means to determine a position of a tool relative to the mobile platform, with a reasonable expectation of success, in order have full picture of the tool’s position by tracking the tool with respect to the body of the robot.
Regarding claim 12, modified Halvorsen teaches the limitations of claim 10. Halvorsen further teaches an orientation sensor, wherein an orientation of the construction robot (10) is determinable by the orientation sensor (see at least [0142]: “In some situations, where efficiency is a priority, an inertial measurement unit (IMU) 812 can deliver position and orientation data such as yaw and distance travelled, to the robot server 803.”).
Regarding claim 13, modified Halvorsen teaches the limitations of claim 10. Halvorsen further teaches wherein the optical sensor system (62) comprises a location marking (36) (see at least Fig. 1, 14a, and [0047]: “One or more reflector prisms may also be mounted on the robotic arm and/or lifting mechanism, and measurements from a total laser station may be used (in addition to or in place of the IMU measurements) to provide position feedback measurements to the robot server for a sway compensation to be determined.”; [0202]: “Another method may use the fitting of a prism close to the movable end of the robotic arm so that the exact position of the drill 122 can be determined before drilling is commenced. A feedback system could be used to tell the robot server 803 when displacement of the drill tip caused by the sway has exceeded a particular threshold, and the robot server 803 can then compensate with a suitable displacement or rotation to the drill 120.”).
Regarding claim 15, modified Halvorsen teaches the limitations of claim 10. Halvorsen further teaches wherein the manipulator (18) has at least three degrees of freedom (see at least [0021]: “The robotic arm may be one having six degrees of freedom to allow it to move a drilling device freely within a three dimensional space as well as controlling the orientation of the drilling device within that space.”).
Regarding claim 16, modified Halvorsen teaches the limitations of claim 10. Halvorsen further teaches wherein the tool is a marking tool or a drilling tool or a chiseling tool or a grinding tool or a cutting tool (see at least [0065]: “In this embodiment, there is provided a robotic drilling apparatus 100 having a robotic arm 110, with one end (a base end 110a) mounted to a substructure 112 and a moveable end 110b of the robotic arm 110 connected to a mount 120 for holding a drilling device 122.”).
Claim Rejections - 35 USC § 103
7. Claim 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Halvorsen et al. (US 20180326507, hereinafter Halvorsen) and Hertzman (US 7474760, hereinafter Hertzman) in view of Malackowski et al. (US 20220117682, hereinafter Malackowski).
Regarding claim 14, modified Halvorsen teaches the limitations of claim 13. Halvorsen further teaches wherein the location marking (36) is disposed on the manipulator (18) (see at least Fig. 1, 14a, and [00194]: “It can be measured easily by having a reflector prism fitted to the base of the robotic arm 110 or some other part of the apparatus with a fixed, known relationship between the position of the prism and the base of the robotic arm.”: [0207]: “FIG. 14a is a perspective view of an exemplary prism cap 1401 that can be fitted over the top of a prism mounted on the drilling apparatus 100 used with the total station measuring system. FIG. 14b is a side elevation of the prism cap 1401.”).
Halvorsen fails to explicitly teach wherein the location marking (36) is disposed on an end effector (16) of the manipulator (18).
However, Malackowski teaches a system and method for robotic obstacle avoidance that comprises a location marking (36) disposed on an end effector (16) of a manipulator (18) (see at least Fig. 1, [0062-0064], and [0073]: trackers 38 is disposed on an end effector of a robot arm (manipulator).).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halvorsen to incorporate the teachings of Malackowski and provide a location marking disposed on an end effector of a manipulator, with a reasonable expectation of success, in order track and monitor a specific position of the manipulator.
Claim Rejections - 35 USC § 103
8. Claim 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Halvorsen et al. (US 20180326507, hereinafter Halvorsen) and Hertzman (US 7474760, hereinafter Hertzman) in view of Bradski et al. (US 9238304, hereinafter Bradski).
Regarding claim 17, modified Halvorsen teaches the limitations of claim 10. Halvorsen further teaches the steps of:
bringing the manipulator (18) into at least one situation (I, II, III), and when the at least one situation (I, II, III) has been reached, determining a position and/or an orientation of a tool disposed on the manipulator (18) (see at least [0140]: “The robot server 803 is able to calculate the precise position of the robotic drilling apparatus 100 and its drilling device 122, and can keep track of all the robotic drilling apparatus' movements. It can feedback statuses of its current position to the operator via the computing device 801.”; [0141]: “The current position may be calculated based on inputs from a laser total station 810 and a vision system 811 (preferably located on the robotic drilling apparatus 100). The total laser station 810 may be provided in the vicinity of the robotic drilling apparatus 100 on the construction site floor (or wall), close to where the drilling is to occur.”; [0186]: “The operator could line up a visible laser line parallel to the drill line. In this way the vision system 811 of the robot can pick up on this line and the robot server will automatically calculate the relative rotation of the drilling apparatus 100 to the drill line.”).
Halvorsen fails to explicitly teach determining a position and/or an orientation of a tool disposed on the manipulator (18) relative to the mobile platform (12) using the optical sensor system (62).
However, Hertzman teaches a method and apparatus that contactless measurement of a tool that determines a position and/or an orientation of a tool disposed on a manipulator (18) relative to a mobile platform (12) using an optical sensor system (62) (see at least Figs. 1, 3, 4, Col. 8, lines 47-51, and Col. 9, lines 4-14: position and/or orientation of the tool relative to the body of the machine is determined using a camera system.).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halvorsen to incorporate the teachings of Hertzman and provide a means to determines a position and/or an orientation of a tool disposed on a manipulator relative to a mobile platform using an optical sensor system, with a reasonable expectation of success, in order have full picture of the tool’s position by tracking the tool with respect to the body of the robot.
The combination of Halvorsen and Hertzman fails to explicitly teach generating calibration data for calibrating adjustment movements (1012) of the manipulator (18).
However, Bradski teaches a system and method for recognizing information on three-dimensional objects that generates calibration data for calibrating adjustment movements (1012) of a manipulator (18) (see at least Col. 12, line33-44: “In some systems, calibration techniques may allow for calibration of grippers or other tools positioned at the end of a robotic device, such as the end effector of a robotic arm. These tools may include, for example, welding irons, screwdrivers, glue guns, tiny paint brushes, and drills, among other possibilities. In some examples, accuracy may be required for processes involving these tools (e.g., surgery). Example systems may be calibrated in order to localize where the tip of the tool is, including applications where repeated use of the tool or accidental collisions between the tool and another object can cause the tip to be dislodged, bent, or otherwise manipulated.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halvorsen and Hertzman to incorporate the teachings of Bradski and provide a means to generate calibration data for calibrating adjustment movements of a manipulator, with a reasonable expectation of success, in order enhance accuracy and localize where the tip of the tool is.
Claim Rejections - 35 USC § 103
9. Claim 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Halvorsen et al. (US 20180326507, hereinafter Halvorsen) and Hertzman (US 7474760, hereinafter Hertzman) and Bradski et al. (US 9238304, hereinafter Bradski) in view of Malackowski et al. (US 20220117682, hereinafter Malackowski).
Regarding claim 18, modified Halvorsen teaches the limitations of claim 17. Halvorsen further teaches wherein the position and/or the orientation is determined by recording and evaluating first data (68) of the construction robot (10) (see at least [0140]: “The robot server 803 is able to calculate the precise position of the robotic drilling apparatus 100 and its drilling device 122, and can keep track of all the robotic drilling apparatus' movements. It can feedback statuses of its current position to the operator via the computing device 801.”; [0141]: “The current position may be calculated based on inputs from a laser total station 810 and a vision system 811 (preferably located on the robotic drilling apparatus 100). The total laser station 810 may be provided in the vicinity of the robotic drilling apparatus 100 on the construction site floor (or wall), close to where the drilling is to occur.”; [0186]: “The operator could line up a visible laser line parallel to the drill line. In this way the vision system 811 of the robot can pick up on this line and the robot server will automatically calculate the relative rotation of the drilling apparatus 100 to the drill line.”).
Halvorsen fails to explicitly teach recording and evaluating first image data (68) of a location marking (36) disposed on the manipulator (18) and second image data (70) of an external location marking (36) disposed separately from the robot (10).
However, Malackowski teaches a system and method for robotic obstacle avoidance that records and evaluates first image data (68) of a location marking (36) disposed on a manipulator (18) and second image data (70) of an external location marking (36) disposed separately from a robot (10) (see at least Fig. 1, [0062]-[0064], and [0073]: vision device 40 records and evaluates first image data of trackers 38 disposed on an end effector of a robot arm (manipulator) and second image data from tracker 36 located on a patient that is disposed separately from the robot arm.).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halvorsen to incorporate the teachings of Malackowski and provide a means record and evaluate first image data of a location marking disposed on a manipulator and second image data of an external location marking disposed separately from a robot, with a reasonable expectation of success, in order provide a triangulated position.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Groll et al. (US 20110066393) teaches an apparatus and method for calibration of a manipulator that utilizes markers and an optic sensor.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIEN MINH LE whose telephone number is (571)272-3903. The examiner can normally be reached Monday to Friday (8:30am-5:30pm eastern time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi Tran can be reached on (571)272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.M.L./Examiner, Art Unit 3656
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656