DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/27/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 14 have been considered but are moot because Applicant’s amendments have changed the scope of the claims. After an updated search, Examiner issues a new rejection over Fujita, as shown below.
Applicant’s arguments filed 4/27/2026, with respect to rejections under 35 U.S.C. 112 and 101 have been fully considered and are persuasive in light of Applicant’s amendments. The rejections under 112 and 101 have been withdrawn.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-2 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Fujita (US-11285609-B2) in view of Suzuki9005 (US 20140229005 A1).
Claim 1
Fujita teaches
a robot;
(Fujita - [col 3, ln 5-6] (13) Robot system 10 includes robot 20 and control device 70 configured to control robot 20. )
a calibration apparatus configured to calibrate a mechanism error parameter for adjusting control of the robot based on an operation program,
EXAMINER NOTE: The control device 70 and peripherals discussed throughout the document correspond to a calibration apparatus.
a processor
(Fujita - [col 3, ln 51-54] (16) Control device 70 is configured as a microprocessor in which CPU 71 is a main section, and includes ROM 72, HDD 73, RAM 74, drive circuit 75, and the like in addition to CPU 71.
Configured to: acquire a position of the robot in the three-dimensional reference coordinate system, and
calculate a mechanism error parameter based on the position of the robot in the three-dimensional reference coordinate system; and …
… the mechanism error parameter, relating to an error of a component in the robot, includes at least one selected from a group of … a Denavit Hartenberg (DH) parameter,
(Fujita - [col 5, ln 27-47] (24) The geometry adjustment is a step of optimizing the DH parameter used for the coordinate transformation. In preparation for the geometry adjustment, the operator attaches the measurement marker m to the distal link and installs three-dimensional measurement instrument 100 at each corner of workbench 11, similar to the matrix measurement/correction. When an instruction of the geometry adjustment is provided, CPU 71 sets the multiple measurement points in work area A of robot 20. The measurement point can be set, for example, by the operator manipulating input device 76 to designate the measurement point. Next, CPU 71 designates the spatial coordinate value of each measurement point to the target position, and controls arm 22 (first to fifth motors 51 to 55) such that the marker m moves to the designated target position. Then, CPU 71 inputs the actual position of the marker m measured by three-dimensional measurement instrument 100, and makes an inverse calculating operation of the DH parameter such that the difference (error) between the spatial coordinate value of the measurement point and the input actual position of the marker m comes to be minimized, thereby ending the geometry adjustment.)
EXAMINER NOTE: The DH parameter (mechanism error parameter) is calculated such that the error between the commanded location and the measured location is minimized.
a controller including a further processor configured to adjust the control of the robot based on the mechanism error parameter,
EXAMINER NOTE: Control device 70 corresponds to a controller
wherein
a first state is a state in which the robot having a first mechanism error parameter set therein is driven according to a command value of the operation program,
a second state is a state in which the robot is driven according to the same command value of the operation program after the first state,
(Fujita - [col 3, ln 64-66] (17) FIG. 4 is a flowchart illustrating an example of a robot control routine executed by control device 70. The routine is repeatedly executed at every predetermined time intervals.
[col 4, ln 10-11] (18) Next, CPU 71 corrects the acquired target position using a matrix correction parameter (S110).
[col 4, ln 42-49] (21) The following description describes a work position correction step at the time of controlling the operation of robot 20 using the target position. FIG. 6 is an explanatory diagram illustrating an example of the work position correction step. The work position correction step is performed by executing an assembly adjustment (S200), a geometry adjustment (S210), a calibration (S220), and a matrix measurement/correction (S230) in order.)
EXAMINER NOTE: This passage indicates that the geometry adjustment step cited above is repeated, meaning that for a given execution of the control routine, a first state would include the mechanism error parameter of the previous routine.
the processor is configured to calculate a second mechanism error parameter different from the first mechanism error parameter so that a position of the robot in the three-dimensional reference coordinate system in the second state matches a position of the robot in the three- dimensional reference coordinate system in the first state.
EXAMINER NOTE: As indicated above, the control routine execution is repeated meaning that for a given execution of the control routine, a first state would include the mechanism error parameter of the previous routine, and a second mechanism error parameter is calculated during the current routine. This parameter is calculated such that the difference (error) between the spatial coordinate value of the measurement point and the input actual position of the marker m comes to be minimized (see Fujita [col 5, ln 27-47], cited above).
Fujita, in effect, utilizes the work table as a reference coordinate system.
(Fujita - [col 5, ln 48-59] (25) The calibration is a step of grasping a relative positional relation between robot 20 and workbench 11 (work object), and reflecting the relative positional relation on the target position at the time of operating robot 20. FIG. 10 is an explanatory diagram illustrating a state of the calibration. In preparation for the calibration, the operator fixes object 110 to which mark M is attached in a predetermined position on workbench 11. When an instruction of the calibration is provided, CPU 71 first controls arm 22 (first to fifth motors 51 to 55) such that mark camera 24 moves above object 110. Subsequently, CPU 71 images mark M attached to object 110 by mark camera 24)
Fujita images the marks M (members) on the object 110. Fujita does not state whether the mark camera 24 is a three-dimensional measuring device. As shown above, Fujita does utilize a three-dimensional measuring instrument 100 to obtain the robot's location ([col 5, ln 27-47]), but Fujita is silent as to its use in locating reference members. However, Suzuki9004 teaches
the calibration apparatus comprising:
a plurality of members acting as reference points and installed at a region where the robot is installed;
(Suzuki9005 - [0068] In step S4, as illustrated in FIG. 3, the user installs a calibration plate (reference member) 10 in a common field which is commonly included in the measurement area of the camera 3 when a plurality of positions and orientations set in each position and orientation group is taken by the robot body 2. The calibration plate 10 is, for example, a planar plate having a plurality of (at least three) circular patterns 10a and 10b of a predetermined size displayed thereon. The camera 3 can measure positions and orientations of the calibration plate 10 having six degrees of freedom.
[0069] Only one of the plurality of circular patterns of the calibration plate 10 is a hollow circular pattern 10a, which is placed at a corner, and others are solid circular patterns 10b. The existence of the hollow circular pattern 10a provides a totally asymmetrical arrangement about a point, and enables unique identification of the position and orientation of the calibration plate 10 having six degrees of freedom. A calibration plate coordinate system P is set to the calibration plate 10 with reference to the calibration plate 10.)
EXAMINER NOTE: The camera detects the features (members) of the calibration plate and sets the coordinate system P accordingly.
a three-dimensional measuring device configured to measure positions of the reference points; and
(Suzuki9005 - [0114] The robot systems 1 according to the first exemplary embodiment and the second exemplary embodiment is described to use the monocular camera 3 as a visual sensor. However, the visual sensor is not limited thereto and may be, for example, a stereo camera or a laser scanner)
A processor configured to:
set a three-dimensional reference coordinate system in a three-dimensional space, based on the positions of the reference points measured by the three-dimensional measuring device,
(Suzuki9005 - [0069] Only one of the plurality of circular patterns of the calibration plate 10 is a hollow circular pattern 10a, which is placed at a corner, and others are solid circular patterns 10b. The existence of the hollow circular pattern 10a provides a totally asymmetrical arrangement about a point, and enables unique identification of the position and orientation of the calibration plate 10 having six degrees of freedom. A calibration plate coordinate system P is set to the calibration plate 10 with reference to the calibration plate 10.)
EXAMINER NOTE: The camera detects the features (members) of the calibration plate and sets the coordinate system P accordingly.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Fujita's calibration aspects with Suzuki9005's coordinate system establishment. Suzuki9005 utilizes the three-dimensional measuring device to capture different angles of the reference points by moving the robot. Fujita already utilizes a three-dimensional measuring instrument, which captures multiple angles of the workbench. Utilizing Fujita's three-dimensional measuring instrument to locate the reference marks and establish a reference coordinate system (as taught by Suzuki9005) would reduce complexity in the system by eliminating the need for mark camera 24. Further, Suzuki9005's transformation matrix serves the same purpose as Fujita's relative positional relationship between the robot and workbench. Substitution of an arbitrary relationship with a transformation matrix would have been within the capabilities of one of ordinary skill, and would achieve the same result of locating the robot relative to the workbench.
Claim 2
The combination of Fujita and Suzuki9005 teaches the limitations of claim 1 as outlined above. Fujita further teaches
wherein the three-dimensional reference coordinate system is an immovable coordinate system defined so as not to depend on an operation of the robot and an installation state of the robot.
EXAMINER NOTE: As discussed above with respect to claim 1, Fujita's workbench serves as a reference coordinate system, which is independent of robot operation and installation state because the markers are placed at a predetermined location.
Claim 9
The combination of Fujita and Suzuki9005 teaches the limitations of claim 1 as outlined above. Fujita further teaches
wherein the first state is a reference state of the robot before calibration of the mechanism error parameter is performed.
(Fujita - [col 3, ln 64-66] (17) FIG. 4 is a flowchart illustrating an example of a robot control routine executed by control device 70. The routine is repeatedly executed at every predetermined time intervals.
[col 4, ln 10-11] (18) Next, CPU 71 corrects the acquired target position using a matrix correction parameter (S110).
[col 4, ln 42-49] (21) The following description describes a work position correction step at the time of controlling the operation of robot 20 using the target position. FIG. 6 is an explanatory diagram illustrating an example of the work position correction step. The work position correction step is performed by executing an assembly adjustment (S200), a geometry adjustment (S210), a calibration (S220), and a matrix measurement/correction (S230) in order.)
EXAMINER NOTE: Refer again to the geometry correction section of Fujita cited above with respect to claim 1 ([col 5, ln 27-47]), as well as . This passage indicates that the geometry adjustment step cited above is repeated, meaning that for a given execution of the control routine, a first state would include the mechanism error parameter of the previous routine, which is a state before the DH parameter of the instant routine is calculated.
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fujita and Suzuki9005 as applied to claim 1 above, and further in view of Sivaneth (US 12030184 B2).
Claim 3
The combination of Fujita and Suzuki9005 teaches the limitations of claim 1 as outlined above. As shown above, Fujita teaches
and the processor is configured to
acquire, in the first state, a plurality of positions of the robot in the three- dimensional reference coordinate system when the robot is driven based on a plurality of command values including the command value,
acquire, in the second state, the plurality of positions of the robot in the three-dimensional reference coordinate system when the robot is driven based on the plurality of command values,
calculate a theoretical position of the robot in the base coordinate system based on the position of the robot in the three-dimensional reference coordinate system … when the robot is driven by each of the plurality of command values in the second state, and
(Fujita - [col 5, ln 27-47] (24) The geometry adjustment is a step of optimizing the DH parameter used for the coordinate transformation. In preparation for the geometry adjustment, the operator attaches the measurement marker m to the distal link and installs three-dimensional measurement instrument 100 at each corner of workbench 11, similar to the matrix measurement/correction. When an instruction of the geometry adjustment is provided, CPU 71 sets the multiple measurement points in work area A of robot 20. The measurement point can be set, for example, by the operator manipulating input device 76 to designate the measurement point. Next, CPU 71 designates the spatial coordinate value of each measurement point to the target position, and controls arm 22 (first to fifth motors 51 to 55) such that the marker m moves to the designated target position. Then, CPU 71 inputs the actual position of the marker m measured by three-dimensional measurement instrument 100, and makes an inverse calculating operation of the DH parameter such that the difference (error) between the spatial coordinate value of the measurement point and the input actual position of the marker m comes to be minimized, thereby ending the geometry adjustment.)
calculate the mechanism error parameter so that the command value matches the theoretical position of the robot in the base coordinate system.
EXAMINER NOTE: In the geometry correction section cited above the DH parameter is calculated such that the difference (error) between the spatial coordinate value of the measurement point and the input actual position of the marker m comes to be minimized (commanded/theoretical positions). Recall as well that this process is repeated during each execution of the control routine, which necessitates the distinction between the first and second states (see discussion above with respect to claim 1).
Fujita calculates relative positional relationships between the workbench and the robot, but is silent as to the specific use of a transformation matrix. However, Sivaneth demonstrates that this and the following aspects are old and notoriously well-known in the art. Sivaneth thus teaches
wherein the processor is configured to calculate a transformation matrix by which one coordinate value among a coordinate value of a base coordinate system set for the robot and a coordinate value of the three-dimensional reference coordinate system is converted into the other coordinate value,
(Sivaneth - [col 2, ln 28-32] In a variation on this embodiment, the system can further include a coordinate-transformation module configured to transform a pose determined by the machine-vision module from a camera-centered coordinate system to a robot-centered coordinate system.)
the command value being specified by the coordinate value of the base coordinate system,
(Sivaneth - [col 7, ln 64-67] (28) The controller of the robotic arm can generate a number of predetermined poses in the robot-base space (operation 308) and sequentially move the end-effector to those poses (operation 310).)
and the processor is configured to
acquire, in the first state, a plurality of positions of the robot in the three- dimensional reference coordinate system when the robot is driven based on a plurality of command values including the command value,
(Sivaneth - [col 8, ln 1-3] At each pose, the 3D machine-vision system can capture images of the calibration target and determine the pose of the calibration target in the camera space (operation 312).)
EXAMINER NOTE: The robot pose in the camera space (reference coordinates) is captured for each predetermined pose. See also passages cited with reference to claim 1.
calculate the transformation matrix, in the first state, based on the plurality of command values and the plurality of positions of the robot in the three-dimensional reference coordinate system,
(Sivaneth - [col 8, ln 3-15] The transformation matrix can then be derived based on poses generated in the robot-base space and the machine-vision-determined poses in the camera space (operation 314). Various techniques can be used to determine the transformation matrix. For example, equation (4) can be solved based on the predetermined poses in the robot-base space and the camera space using various techniques, including but not limited to: linear least square or SVD techniques, Lie-theory-based techniques, techniques based on quaternion and non-linear minimization or dual quaternion, techniques based on Kronecker product and vectorization, etc.)
acquire, in the second state, the plurality of positions of the robot in the three-dimensional reference coordinate system when the robot is driven based on the plurality of command values,
calculate a theoretical position of the robot in the base coordinate system based on the position of the robot in the three-dimensional reference coordinate system and the transformation matrix when the robot is driven by each of the plurality of command values in the second state, and
(Sivaneth - [col 5, ln 61 thru col 6, ln 3] (21) To reduce positioning/pose errors of a working robot in its entire working space in real time, the concept of an error matrix can be introduced. The error matrix can indicate the difference between a controller-desired pose (i.e., the pose programmed by the robotic controller) of the robot end-effector and the actual pose of the end-effector (which can be captured by the cameras and converted from the camera space to the robot-base space using the transformation matrix) and can vary as the position of the end-effector changes in the working space.
…
[col 6, ln 15-29] In one example, the robotic controller may send a command to move the end-effector to desired TCP pose Htd. However, due to errors (e.g., errors in the actuation of the joints and end-effector) in the robotic system, when the controller instructs the robotic arm to achieve this pose, the resulting pose is often different from H.sub.td. The actual pose of the end-effector measured by the 3D machine-vision module and transformed from the camera space to the robot-base space can be instructed pose Hti. Hence, given an instructed pose (i.e., a pose known to the camera), if error matrix E({right arrow over (r)}) is known, one can compute the desired pose that can be used by the controller to instruct the robotic arm to move the end-effector to the instructed pose, thus achieving the eye (camera)-to-hand (robotic controller) coordination.)
EXAMINER NOTE: The robot is commanded to move to a desired pose by the controller (command value in the base coordinate system) and the camera records the pose in the camera (reference) coordinate system. Using the transformation matrix, these poses are compared in a common coordinate system and the error matrix is derived and used such that the desired pose matches the transformed camera pose (theoretical pose). Note that according to Figure 4 the error matrix is constructed by carrying out this process multiple times, indicating a plurality of poses.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination cited in claim 1 with with Sivaneth’s error matrix in order to more precisely reduce positioning/pose errors of the robot in real time.
(Sivaneth – [col 5, ln 61 thru col 6, ln 5] To reduce positioning/pose errors of a working robot in its entire working space in real time, the concept of an error matrix can be introduced. The error matrix can indicate the difference between a controller-desired pose (i.e., the pose programmed by the robotic controller) of the robot end-effector and the actual pose of the end-effector (which can be captured by the cameras and converted from the camera space to the robot-base space using the transformation matrix) and can vary as the position of the end-effector changes in the working space. In some embodiments, the error matrix can be expressed as the transformation from the instructed pose to the desired pose in the robot-base space:)
Claim 4
The combination of Fujita, Suzuki9005, and Sivaneth teaches the limitations of claim 3 as outlined above. As shown above, the cited combination also teaches
calculate the transformation matrix so as to minimize a distance between the command value in the base coordinate system and the theoretical position of the robot in the base coordinate system,
which is calculated by the transformation matrix from the position of the robot in the three-dimensional reference coordinate system, in the first state.
(Sivaneth - [col 8, ln 3-15] The transformation matrix can then be derived based on poses generated in the robot-base space and the machine-vision-determined poses in the camera space (operation 314). Various techniques can be used to determine the transformation matrix. For example, equation (4) can be solved based on the predetermined poses in the robot-base space and the camera space using various techniques, including but not limited to: linear least square or SVD techniques, Lie-theory-based techniques, techniques based on quaternion and non-linear minimization or dual quaternion, techniques based on Kronecker product and vectorization, etc.
EXAMINER NOTE: The above techniques such as linear least squares are techniques which minimize error. When used to calculate transformation matrices such as the one described above, these errors correspond to deviations between commanded values and theoretical positions. )
Claim(s) 5 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fujita and Suzuki9005, and further in view of Islam (US-20200306977-A1)
Claim 5
The combination of Fujita and Suzuki9005 teaches the limitations of claim 3 as outlined above. The cited combination may not explicitly teach the following limitations in combination, but Islam teaches
wherein the processor is configured to evaluate accuracy of the position of the robot with respect to the command value of the operation program, and
in response to the robot being driven with the command value, determine a need for a calibrating the mechanism error parameter, based on the position of the robot in the three- dimensional reference coordinate system in the first state and the position of the robot in the three-dimensional reference coordinate system in the second state.
(Islam - [0004] b) outputting a first movement command to the communication interface, wherein the communication interface is configured to communicate the first movement command to the robot to cause the robot arm to move the verification symbol, during or after the first calibration operation, to a location within the camera field of view, the location being a reference location of one or more reference locations for verification of the first calibration operation … d) determining a reference image coordinate for the verification symbol …)
EXAMINER NOTE: First movement command is communicated to robot. According ot this command, the robot moves to a reference location, and the camera determines coordinates of verification symbol. As established above with respect to claim 1, movement commands include parameter values (command values).
(Islam - [0005] In an embodiment, the control circuit is configured to perform the calibration verification further by: f… g) outputting a third movement command to the communication interface, wherein the communication interface is configured to communicate the third movement command to the robot to cause the robot arm to move the verification symbol to at least the reference location during the idle period, h) receiving via the communication interface an additional image of the verification symbol from the camera, which is configured to capture the additional image of the verification symbol at least at the reference location during the idle period, the additional image being a verification image for the verification, i) determining a verification image coordinate used for the verification, the verification image coordinate being a coordinate at which the verification symbol appears in the verification image, j) determining a deviation parameter value based on an amount of deviation between the reference image coordinate and the verification image coordinate, the reference image coordinate and the verification image coordinate both associated with the reference location, wherein the deviation parameter value is indicative of a change since the first calibration operation…)
EXAMINER NOTE: Third movement command is communicated to the robot. According to this command, the robot moves to the reference location and the camera again determines the coordinates of the verification symbol. The difference between these coordinates is determined via the deviation parameter value
(Islam - [0005] … k) determining whether the deviation parameter value exceeds a defined threshold, and l) performing, in response to a determination that the deviation parameter value exceeds the defined threshold, a second calibration operation (e.g., a second camera calibration operation) to determine updated calibration information (e.g., updated camera calibration information).)
EXAMINER NOTE: If the deviation parameter exceeds a threshold, the calibration procedure is carried out.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the combination cited in claim 1 with Islam's suggestion to check for positioning errors in order to determine when the system requires recalibration, thus preventing undesirable robot operation.
(Islam - [0031] … When a value of the deviation parameter exceeds the defined deviation threshold, this condition may indicate that use of previously generated calibration information may lead to an undesirable amount of error in the robot operation. Thus, in some cases, the robot operation may be paused or stopped while the additional calibration operation is performed (the pause may be considered another idle period).
[0104] … Such an arrangement provides the advantage of allowing a robot controller or other computing system to be able to more frequently and/or more quickly perform calibration verification and to evaluate whether updated camera calibration needs to be determined.)
Claim 14
Fujita teaches
a robot;
(Fujita - [col 3, ln 5-6] (13) Robot system 10 includes robot 20 and control device 70 configured to control robot 20. )
a determination apparatus …
EXAMINER NOTE: The control device 70 and peripherals discussed throughout the document correspond to a determination apparatus.
the determination apparatus comprising:
a processor
(Fujita - [col 3, ln 51-54] (16) Control device 70 is configured as a microprocessor in which CPU 71 is a main section, and includes ROM 72, HDD 73, RAM 74, drive circuit 75, and the like in addition to CPU 71.
acquire a position of the robot in the three-dimensional reference coordinate system, and
evaluate accuracy of the position of the robot with respect to a command value of the operation program; and
(Fujita - [col 5, ln 27-47] (24) The geometry adjustment is a step of optimizing the DH parameter used for the coordinate transformation. In preparation for the geometry adjustment, the operator attaches the measurement marker m to the distal link and installs three-dimensional measurement instrument 100 at each corner of workbench 11, similar to the matrix measurement/correction. When an instruction of the geometry adjustment is provided, CPU 71 sets the multiple measurement points in work area A of robot 20. The measurement point can be set, for example, by the operator manipulating input device 76 to designate the measurement point. Next, CPU 71 designates the spatial coordinate value of each measurement point to the target position, and controls arm 22 (first to fifth motors 51 to 55) such that the marker m moves to the designated target position. Then, CPU 71 inputs the actual position of the marker m measured by three-dimensional measurement instrument 100, and makes an inverse calculating operation of the DH parameter such that the difference (error) between the spatial coordinate value of the measurement point and the input actual position of the marker m comes to be minimized, thereby ending the geometry adjustment.)
EXAMINER NOTE: The DH parameter (mechanism error parameter) is calculated such that the error between the commanded location and the measured location is minimized.
a controller including a further processor configured to adjust the control of the robot based on the mechanism error parameter,
(Fujita - [col 3, ln 51-54] (16) Control device 70 is configured as a microprocessor in which CPU 71 is a main section, and includes ROM 72, HDD 73, RAM 74, drive circuit 75, and the like in addition to CPU 71.
wherein a first state is a state in which the robot having a first mechanism error parameter set therein is driven according to a command value of the operation program,
a second state is a state in which the robot is driven according to the same command values of the operation program after the first state,
(Fujita - [col 3, ln 64-66] (17) FIG. 4 is a flowchart illustrating an example of a robot control routine executed by control device 70. The routine is repeatedly executed at every predetermined time intervals.
[col 4, ln 10-11] (18) Next, CPU 71 corrects the acquired target position using a matrix correction parameter (S110).
[col 4, ln 42-49] (21) The following description describes a work position correction step at the time of controlling the operation of robot 20 using the target position. FIG. 6 is an explanatory diagram illustrating an example of the work position correction step. The work position correction step is performed by executing an assembly adjustment (S200), a geometry adjustment (S210), a calibration (S220), and a matrix measurement/correction (S230) in order.)
EXAMINER NOTE: This passage indicates that the geometry adjustment step cited above is repeated, meaning that for a given execution of the control routine, a first state would include the mechanism error parameter of the previous routine.
the mechanism error parameter, relating to an error of a component in the robot, includes at least one selected from a group of … a Denavit Hartenberg (DH) parameter
EXAMINER NOTE: See geometry correction section cited above. The calibration includes calculation of a DH parameter such that the difference (error) between the spatial coordinate value of the measurement point and the input actual position of the marker m comes to be minimized,
Fujita, in effect, utilizes the work table as a reference coordinate system.
(Fujita - [col 5, ln 48-59] (25) The calibration is a step of grasping a relative positional relation between robot 20 and workbench 11 (work object), and reflecting the relative positional relation on the target position at the time of operating robot 20. FIG. 10 is an explanatory diagram illustrating a state of the calibration. In preparation for the calibration, the operator fixes object 110 to which mark M is attached in a predetermined position on workbench 11. When an instruction of the calibration is provided, CPU 71 first controls arm 22 (first to fifth motors 51 to 55) such that mark camera 24 moves above object 110. Subsequently, CPU 71 images mark M attached to object 110 by mark camera 24)
Fujita images the marks M (members) on the object 110. Fujita does not state whether the mark camera 24 is a three-dimensional measuring device. As shown above, Fujita does utilize a three-dimensional measuring instrument 100 to obtain the robot's location ([col 5, ln 27-47]), but Fujita is silent as to its use in locating reference members. However, Suzuki9004 teaches
the determination apparatus comprising:
a plurality of members acting as reference points and installed at a region where the robot is installed;
(Suzuki9005 - [0068] In step S4, as illustrated in FIG. 3, the user installs a calibration plate (reference member) 10 in a common field which is commonly included in the measurement area of the camera 3 when a plurality of positions and orientations set in each position and orientation group is taken by the robot body 2. The calibration plate 10 is, for example, a planar plate having a plurality of (at least three) circular patterns 10a and 10b of a predetermined size displayed thereon. The camera 3 can measure positions and orientations of the calibration plate 10 having six degrees of freedom.
[0069] Only one of the plurality of circular patterns of the calibration plate 10 is a hollow circular pattern 10a, which is placed at a corner, and others are solid circular patterns 10b. The existence of the hollow circular pattern 10a provides a totally asymmetrical arrangement about a point, and enables unique identification of the position and orientation of the calibration plate 10 having six degrees of freedom. A calibration plate coordinate system P is set to the calibration plate 10 with reference to the calibration plate 10.)
EXAMINER NOTE: The camera detects the features (members) of the calibration plate and sets the coordinate system P accordingly.
a three-dimensional measuring device configured to measure positions of the reference points; and
(Suzuki9005 - [0114] The robot systems 1 according to the first exemplary embodiment and the second exemplary embodiment is described to use the monocular camera 3 as a visual sensor. However, the visual sensor is not limited thereto and may be, for example, a stereo camera or a laser scanner)
a processor configured to:
set a three-dimensional reference coordinate system in a three-dimensional space, based on the positions of the reference points measured by the three-dimensional measuring device,
(Suzuki9005 - [0069] Only one of the plurality of circular patterns of the calibration plate 10 is a hollow circular pattern 10a, which is placed at a corner, and others are solid circular patterns 10b. The existence of the hollow circular pattern 10a provides a totally asymmetrical arrangement about a point, and enables unique identification of the position and orientation of the calibration plate 10 having six degrees of freedom. A calibration plate coordinate system P is set to the calibration plate 10 with reference to the calibration plate 10.)
EXAMINER NOTE: The camera detects the features (members) of the calibration plate and sets the coordinate system P accordingly.
Fujita may not explicitly teach the following limitations in combination. However, Islam teaches
a determination apparatus configured to determine a need for calibrating a … error parameter for adjusting control of the robot based on an operation program,
the determination apparatus comprising: a processor …
the processor is configured to determine a need for calibrating the mechanism error parameter, based on the position of the robot in the three-dimensional reference coordinate system in the first state and the position of the robot in the three-dimensional reference coordinate system in the second state.
(Islam - [0004] b) outputting a first movement command to the communication interface, wherein the communication interface is configured to communicate the first movement command to the robot to cause the robot arm to move the verification symbol, during or after the first calibration operation, to a location within the camera field of view, the location being a reference location of one or more reference locations for verification of the first calibration operation … d) determining a reference image coordinate for the verification symbol …)
EXAMINER NOTE: First movement command is communicated to robot. According ot this command, the robot moves to a reference location, and the camera determines coordinates of verification symbol. As established above with respect to claim 1, movement commands include parameter values (command values).
(Islam - [0005] In an embodiment, the control circuit is configured to perform the calibration verification further by: f… g) outputting a third movement command to the communication interface, wherein the communication interface is configured to communicate the third movement command to the robot to cause the robot arm to move the verification symbol to at least the reference location during the idle period, h) receiving via the communication interface an additional image of the verification symbol from the camera, which is configured to capture the additional image of the verification symbol at least at the reference location during the idle period, the additional image being a verification image for the verification, i) determining a verification image coordinate used for the verification, the verification image coordinate being a coordinate at which the verification symbol appears in the verification image, j) determining a deviation parameter value based on an amount of deviation between the reference image coordinate and the verification image coordinate, the reference image coordinate and the verification image coordinate both associated with the reference location, wherein the deviation parameter value is indicative of a change since the first calibration operation…)
EXAMINER NOTE: Third movement command is communicated to the robot. According to this command, the robot moves to the reference location and the camera again determines the coordinates of the verification symbol. The difference between these coordinates is determined via the deviation parameter value
(Islam - [0005] … k) determining whether the deviation parameter value exceeds a defined threshold, and l) performing, in response to a determination that the deviation parameter value exceeds the defined threshold, a second calibration operation (e.g., a second camera calibration operation) to determine updated calibration information (e.g., updated camera calibration information).)
EXAMINER NOTE: If the deviation parameter exceeds a threshold, the calibration procedure is carried out.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Fujita's calibration aspects with Suzuki9005's coordinate system establishment. Suzuki9005 utilizes the three-dimensional measuring device to capture different angles of the reference points by moving the robot. Fujita already utilizes a three-dimensional measuring instrument, which captures multiple angles of the workbench. Utilizing Fujita's three-dimensional measuring instrument to locate the reference marks and establish a reference coordinate system (as taught by Suzuki9005) would reduce complexity in the system by eliminating the need for mark camera 24.
As a further improvement, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Fujita with Islam's suggestion to check for positioning errors in order to determine when the system requires recalibration, thus preventing undesirable robot operation.
(Islam - [0031] … When a value of the deviation parameter exceeds the defined deviation threshold, this condition may indicate that use of previously generated calibration information may lead to an undesirable amount of error in the robot operation. Thus, in some cases, the robot operation may be paused or stopped while the additional calibration operation is performed (the pause may be considered another idle period).
[0104] … Such an arrangement provides the advantage of allowing a robot controller or other computing system to be able to more frequently and/or more quickly perform calibration verification and to evaluate whether updated camera calibration needs to be determined.)
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fujita, Suzuki9005, and Islam as applied to claim 5 above, and further in view of Bergantz (US-20210291375-A1)
Claim 6
The combination of Fujita, Suzuki9004, and Islam teaches the limitations of claim 5. The cited combination may not explicitly teach the following limitations in combination. However, Bergantz teaches
wherein the processor is configured to determine whether or not the accuracy of the position of the robot deviates from a predetermined determination range per predetermined period, and
in response to the accuracy of the position of the robot deviating from the predetermined determination range, determine that calibration of the mechanism error parameter is needed.
(Bergantz - [0119] FIG. 11 is flow chart for a method 1100 of determining whether a transfer sequence is out of calibration, according to embodiments of the present disclosure. A transfer sequence may be calibrated, and the actual position and/or orientation that is achieved for the transfer sequence may slowly drift over time after such calibration. This may be due to wear on one or more robots, for example. … To detect such drift and/or sudden changes, calibration operations may be performed periodically.)
EXAMINER NOTE: Because the calibration is performed periodically, the deviation is checked per period.
(Bergantz - [0121] At block 1115, the system controller compares the characteristic error values between the two or more times that the calibration procedure was performed. … For example, the system controller may determine whether there is a difference between the originally computed characteristic error value(s) and newly computed characteristic error value(s). ... The system controller may determine, based on such comparisons, any drift in the computed characteristic error values or any sudden change in the characteristic error values. If a difference is determined and that difference exceeds a difference threshold, then the method proceeds to block 1120 and system controller determines that the system has changed …)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to further modify the combination cited in claim 5 with Bergantz’s suggestion to determine deviation over time in order to detect drift due to component wear.
Claim(s) 7-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fujita, Suzuki9005, and Islam as applied to claim 5 above, and further in view of Shen (US-20200198146-A1).
Claim 7
The combination of Fujita, Suzuki9005, and Islam teaches the limitations of claim 5 as outlined above. The cited combination may not explicitly teach the following limitations in combination. However, Shen teaches
wherein the processor is configured to, in response to replacement of components constituting the robot being detected, determine that calibration of the mechanism error parameter is needed.
(Shen - [0047] As shown in FIG. 3, determination is made manually by user of the robot arm 2 or automatically by the robot controller 20 in the robot arm 2 to know whether the tool 22 of the robot arm 2 needs calibration (step S20). In one embodiment, user/the robot controller 20 determines that the tool 22 needs calibration when the tool 22 is replaced, the using time of the tool 22 exceeds a first threshold, or the preciseness of the tool 22 is smaller than a second threshold.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to further modify Fujita's calibration with Shen’s suggestion to recalibrate after replacement of the tool or robot in order to precisely know the position of the tool and enhance precision of the robot arm.
(Shen - [0045] If the tool 22 needs calibration, the calibration system 1 uses the robot controller 20 to control the robot arm 2 to move/rotate such an absolute position of the TWP 221 of the tool 22 in the robot arm coordinate system {B} can be obtained through above calibration method. (step S16) After obtaining the absolute position of the TWP 221 of the tool 22 in the robot arm coordinate system {B}, the robot controller 20 can precisely know the position of the tool 22 and the operation preciseness of the robot arm 2 can be enhanced.)
Claim 8
The combination of Fujita, Suzuki9005, and Islam teaches the limitations of claim 5 as outlined above. The cited combination may not explicitly teach the following limitations in combination. However, Shen teaches
wherein the processor is configured to, in response to replacement of the robot being detected, determine that calibration of the mechanism error parameter is needed.
(Shen - [0043] Refer now to FIG. 2, which shows the flowchart of the calibration method according to a first embodiment of the present invention. At first, user confirms whether the robot arm 2/imaging device 3 in the calibration system 1 needs re-installation or replacement (step S10). If the calibration system 1 is first-time set up, or any one of the robot arm 2 and the imaging device 3 is replaced, the transformation relationship between the robot arm coordinate system {B} and the imaging device coordinate system {I} needs to establish at first (step S12), namely, establish the transformation matrix.
[0053] Refer to FIG. 5, this figure shows the establishing flowchart according to the first embodiment of the present invention. As mentioned above, to accurately obtain the absolute position of the TWP 221 on the robot arm coordinate system {B}, the imaging device coordinate system {I} and the transformation matrix need to be established. Therefore, the user of the robot arm 2 first assembles or replaces the robot arm 2 and/or imaging device 3 in the calibration system 1 (step S40). Only when the robot arm 2 and/or imaging device 3 is first time assembled or replaced, the steps in FIG. 5 need to be executed to re-establish the imaging device coordinate system {I} and the transformation matrix. At this time, the tool 22 is not assembled to the robot arm 2 yet.
[0047] As shown in FIG. 3, determination is made manually by user of the robot arm 2 or automatically by the robot controller 20 in the robot arm 2 to know whether the tool 22 of the robot arm 2 needs calibration (step S20). In one embodiment, user/the robot controller 20 determines that the tool 22 needs calibration when the tool 22 is replaced, the using time of the tool 22 exceeds a first threshold, or the preciseness of the tool 22 is smaller than a second threshold.)
EXAMINER NOTE: Because the tool is not yet installed on the robot when the entire robot is set up/replaced, the calibration would necessarily need to be carried out once the tool is installed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to further modify Fujita's calibration with Shen’s suggestion to recalibrate after replacement of the tool or robot in order to precisely know the position of the tool and enhance precision of the robot arm.
(Shen - [0045] If the tool 22 needs calibration, the calibration system 1 uses the robot controller 20 to control the robot arm 2 to move/rotate such an absolute position of the TWP 221 of the tool 22 in the robot arm coordinate system {B} can be obtained through above calibration method. (step S16) After obtaining the absolute position of the TWP 221 of the tool 22 in the robot arm coordinate system {B}, the robot controller 20 can precisely know the position of the tool 22 and the operation preciseness of the robot arm 2 can be enhanced.)
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fujita and Suzuki9005, and further in view of Shen and Bergantz.
Claim 10
Fujita and and Suzuki9005 teaches the limitations of claim 9 as outlined above. The cited combination is silent as to the aspect of wherein the first state is a state immediately after the robot is installed. However, it is notoriously old and well-known to establish an initial calibration when a robot is installed, as evidenced by Shen
wherein the first state is a state immediately after the robot is installed,
(Shen - [0043] Refer now to FIG. 2, which shows the flowchart of the calibration method according to a first embodiment of the present invention. At first, user confirms whether the robot arm 2/imaging device 3 in the calibration system 1 needs re-installation or replacement (step S10). If the calibration system 1 is first-time set up, or any one of the robot arm 2 and the imaging device 3 is replaced, the transformation relationship between the robot arm coordinate system {B} and the imaging device coordinate system {I} needs to establish at first (step S12), namely, establish the transformation matrix.
[0053] Refer to FIG. 5, this figure shows the establishing flowchart according to the first embodiment of the present invention. As mentioned above, to accurately obtain the absolute position of the TWP 221 on the robot arm coordinate system {B}, the imaging device coordinate system {I} and the transformation matrix need to be established. Therefore, the user of the robot arm 2 first assembles or replaces the robot arm 2 and/or imaging device 3 in the calibration system 1 (step S40). Only when the robot arm 2 and/or imaging device 3 is first time assembled or replaced, the steps in FIG. 5 need to be executed to re-establish the imaging device coordinate system {I} and the transformation matrix. At this time, the tool 22 is not assembled to the robot arm 2 yet.
[0047] As shown in FIG. 3, determination is made manually by user of the robot arm 2 or automatically by the robot controller 20 in the robot arm 2 to know whether the tool 22 of the robot arm 2 needs calibration (step S20). In one embodiment, user/the robot controller 20 determines that the tool 22 needs calibration when the tool 22 is replaced, the using time of the tool 22 exceeds a first threshold, or the preciseness of the tool 22 is smaller than a second threshold.)
The above combination may not explicitly teach the following limitations in combination. However, Bergantz teaches
the second state is a state when at least some components of the robot deteriorate by using the robot.
(Bergantz - [0119] FIG. 11 is flow chart for a method 1100 of determining whether a transfer sequence is out of calibration, according to embodiments of the present disclosure. A transfer sequence may be calibrated, and the actual position and/or orientation that is achieved for the transfer sequence may slowly drift over time after such calibration. This may be due to wear on one or more robots, for example. … To detect such drift and/or sudden changes, calibration operations may be performed periodically.
[0121] At block 1115, the system controller compares the characteristic error values between the two or more times that the calibration procedure was performed. … For example, the system controller may determine whether there is a difference between the originally computed characteristic error value(s) and newly computed characteristic error value(s). ... The system controller may determine, based on such comparisons, any drift in the computed characteristic error values or any sudden change in the characteristic error values. If a difference is determined and that difference exceeds a difference threshold, then the method proceeds to block 1120 and system controller determines that the system has changed …)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to further modify the combination cited in claim 9 with Shen’s suggestion to recalibrate after replacement of the tool or robot in order to precisely know the position of the tool and enhance precision of the robot arm, and Bergantz’s suggestion to perform calibration periodically in order to detect drift due to wear.
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fujita and Suzuki9005 as applied to claim 9 above, and further in view of Shen and Suzuki (Us-20160059419-a1).
Claim 11
Fujita and Suzuki9005 teaches the limitations of claim 9 as outlined above. The cited combination may not explicitly teach the following limitations in combination. However, Suzuki teaches
wherein the first state is a state before the robot is replaced
(Suzuki - [0077] The N calibration positions and orientations .sup.RH.sub.T[i] generated as a result of the teaching performed in step S13 are stored in the memory (the ROM 42 or the RAM 43) as calibration position and position data. The teaching need not be performed again, and if the robot arm 1 or the fixed camera 3 is replaced and calibration is performed again, for example, results of a past teaching operation may be used.)
And Shen teaches
and the second state is a state after replacement of the robot, in which a new robot is installed.
(Shen - [0043] Refer now to FIG. 2, which shows the flowchart of the calibration method according to a first embodiment of the present invention. At first, user confirms whether the robot arm 2/imaging device 3 in the calibration system 1 needs re-installation or replacement (step S10). If the calibration system 1 is first-time set up, or any one of the robot arm 2 and the imaging device 3 is replaced, the transformation relationship between the robot arm coordinate system {B} and the imaging device coordinate system {I} needs to establish at first (step S12), namely, establish the transformation matrix.
[0053] Refer to FIG. 5, this figure shows the establishing flowchart according to the first embodiment of the present invention. As mentioned above, to accurately obtain the absolute position of the TWP 221 on the robot arm coordinate system {B}, the imaging device coordinate system {I} and the transformation matrix need to be established. Therefore, the user of the robot arm 2 first assembles or replaces the robot arm 2 and/or imaging device 3 in the calibration system 1 (step S40). Only when the robot arm 2 and/or imaging device 3 is first time assembled or replaced, the steps in FIG. 5 need to be executed to re-establish the imaging device coordinate system {I} and the transformation matrix. At this time, the tool 22 is not assembled to the robot arm 2 yet.
[0047] As shown in FIG. 3, determination is made manually by user of the robot arm 2 or automatically by the robot controller 20 in the robot arm 2 to know whether the tool 22 of the robot arm 2 needs calibration (step S20). In one embodiment, user/the robot controller 20 determines that the tool 22 needs calibration when the tool 22 is replaced, the using time of the tool 22 exceeds a first threshold, or the preciseness of the tool 22 is smaller than a second threshold.)
EXAMINER NOTE: Because the tool is not yet installed on the robot when the entire robot is set up/replaced, the calibration would necessarily need to be carried out once the tool is installed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Islam’s calibration with Shen’s suggestion to recalibrate after replacement of the tool or robot in order to precisely know the position of the tool and enhance precision of the robot arm, and Suzuki’s suggestion to store reference locations in order to eliminate the need to re-teach reference locations when installing a new robot.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fujita and Suzuki9005 as applied to claim 9 above, and further in view of Shen and Kim (US 20040010345 A1).
Claim 12
Fujita and Suzuki9005 teaches the limitations of claim 9 as outlined above. The cited combination may not explicitly teach the following limitations in combination. However, Kim teaches
wherein the first state is a state before some components of the robot are replaced, and
the second state is a state after some components of the robot are replaced.
(Kim - [0025] FIG. 4 is a control flowchart of a method of calibrating the robot, according to the present invention. As shown in FIG. 4, the control unit 100 performs an initial calibration using a self-calibration program of the robot arm 13, which is stored in the storage unit 140 at operation S100.
[0026] The control unit 100 stores correction data obtained by the performance of the initial calibration in the storage unit 140 at operation S101. The control unit 100 operates the motor 120 through the motor driving unit 110 to allow the robot arm 13 to move to a contact position at operation S102. At this time, the control unit 100 inputs the moving displacement of the robot arm 13, which is fed back through the encoder 130.)
EXAMINER NOTE: Robot is moved to a contact position (reference location, first state)
(Kim - [0028] If it is determined that the robot arm 13 has reached the contact position at operation S103, the control unit 100 stops the movement of the robot arm 13 by stopping the motor 120 through the motor driving unit 110 at operation Si04. The control unit 100 stores the moving displacement of the robot arm 13 obtained through the encoder 130 in the storage unit 140 as a first moving displacement at operation S105.
EXAMINER NOTE: The displacement obtained in moving to the contact position is recorded (reference coordinates).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to further modify Islam’s system with Kim’s suggestion to recalibrate after part changes to compensate for changes in offsets resulting from changing parts,
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Nilsson (US-20200298403) and Bennet (Kinematic Calibration by Direct estimation of the Jacobian Matrix) each disclose calibration systems involving deriving mechanical error parameters per applicant’s definition, and are considered especially pertinent to the independent claims.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES MILLER WATTS whose telephone number is (703)756-1249. The examiner can normally be reached 7:30-5:30 M-TH.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached at 571-270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES MILLER WATTS III/Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights