Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
The Office Action is in response to the amendment filed 01/18/2026. Claims 1-14 are presently pending and are presented for examination.
Response to Arguments
Applicant's arguments, see pages 10-14, filed 01/18/2026, regarding the rejection of claims 1-4 and 6-14 under 35 U.S.C. 102 as being anticipated by Yukihiro et al. US 20180004188 A1 (“Yukihiro”), have been fully considered but they are not persuasive. Applicant argues that Yukihiro does not teach the new language of the amended independent claims 1, 8, 10, and 11, particularly regarding “a positional relationship between a sensor coordinate system set to a vision sensor and a hand-tip coordinate system set to a hand tip of a robot configured to relatively move the vision sensor and an index mark”. However, as disclosed in further detail below, Yukihiro teaches these elements in FIGS. 37, 41-43, and associated paragraphs 429-449. For these reasons, the rejection of claims 1-4 and 6-14 under 35 U.S.C. 102 is maintained. Likewise, claim 5, which applicant argues should be allowable in light of the amendments to claim 1, is also rejected under 35 U.S.C. 103 because claim 1 is still rejected under the prior art.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4 and 6-14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yukihiro et al. US 20180004188 A1 (“Yukihiro”).
Regarding Claim 1. Yukihiro teaches a device configured to acquire a positional relationship between a sensor coordinate system set to a vision sensor and a hand-tip coordinate system set to a hand tip of a robot configured to relatively move the vision sensor and an index mark (Paragraphs 4, 53. FIG. 1, the marker at MK is the first marker [paragraph 155], shown to be located within the line of sight of the imaging unit 21 in FIG. 9. At the same time, FIG. 37 shows a schematic view of the robot with a coordinate system at the point of the manipulator, with another coordinate system from the image sensor shown as “xb” and “yb”. This is shown again in FIG. 42, with the process for updating the relationship and offset between the image coordinate system and the hand-tip coordinate system shown in FIG. 43), the device comprising a processor configured to:
acquire a direction of a line of sight of the vision sensor in the hand-tip coordinate system in advance (Line “AX1” in FIG. 9, shown to be in the coordinate system in the bottom left corner of FIG. 9 and referred to as the optical axis of the imaging unit 21 [paragraph 215], which is the same imaging unit 21 in FIG. 1);
operate the robot so as to change an orientation of the vision sensor or the index mark in a direction about an axis parallel to the direction of the line of sight, by a first orientation change amount (FIG. 6, paragraph 200, wherein angle θ represents the angle that the frame is rotated. Note that this rotation is made around an axis parallel to the line of sight at AX1 in FIG. 9);
acquire first image data of the index mark imaged by the vision sensor before and after the orientation is changed by the first orientation change amount (The first position reads on a trial measurement position [paragraph 19], wherein the image of the marker is captured by the imaging unit [paragraph 163-165]);
operate the robot so as to change the orientation of the vision sensor or the index mark in a direction about an axis orthogonal to the direction of the line of sight, by a second orientation change amount (The second position in paragraph 19 reads on the second orientation. Note FIG. 4, wherein the robot rotates the arm when determining the new position [paragraph 184]. In step S270 of FIG. 4, paragraph 20, the third predetermined angle is where the robot ends up after a second orientation change);
acquire the second image data of the index mark imaged by the vision sensor after the orientation is changed by the second orientation change amount;
acquire, as a trial measurement position (paragraphs 149-151), first coordinates of an origin of the sensor coordinate system in the hand-tip coordinate system, based on the first and second image data, and third image data of the index mark imaged by the vision sensor before the orientation is changed (The imaging unit at 21 of FIG. 1 is a camera, and a position of the captured imaging unit is expressed by X and Y coordinates in an imaging coordinate system CC [paragraph 149], and the position of the imaging unit is determined at least in part by the location of the first marker and the positions determined in FIG. 4 [paragraphs 212-214, FIGS. 4 and 9]);
operate the robot so as to change the orientation by a third orientation change amount larger than the first and second orientation change amount in an orientation change direction which is determined based on the trial measurement position;
acquire, as a real measurement position, second coordinates of the origin sensor coordinate system in the hand-tip coordinate system, based on image data of the index mark imaged by the vision sensor before and after the orientation is changed by the second orientation change amount (The position and attitude calculation part 45 detects the first marker MK contained in the zeroth image XP3 based on the zeroth image XP3. The position and attitude calculation part 45 calculates a third detection position as a position of the first marker MK detected from the zeroth image XP3 in the imaging unit coordinate system CC. The position and attitude calculation part 45 calculates a displacement from a coincidence point in the zeroth image XP3 to the calculated third detection position in the imaging unit coordinate system CC [paragraph 202]. The position and attitude calculation part 45 transforms the calculated displacement into a displacement in the robot coordinate system RC based on the above described coordinate transform matrix. Then, the position and attitude calculation part 45 calculates third position and attitude obtained by translation of the current position and attitude (i.e. first position and attitude) of the control point T by the displacement based on the transformed displacement and the current position and attitude of the control point T. The third position and attitude are a position and an attitude in the robot coordinate system RC. That is, when the position and the attitude of the control point T coincide with the third position and attitude, the position of the first marker MK contained in the captured image captured by the imaging unit 21 coincides with the coincidence point in the captured image. This is largely a repeat of the steps for finding the second position, and is shown in more detail in FIGS. 7 and 8); and
update coordinates of the sensor coordinate system in the hand-tip coordinate system stored in a memory from the first coordinates acquired as the trial measurement position to the second coordinates acquired as the real measurement position (FIG. 43 also shows the process by which the control apparatus obtains the offset component of the imaging reference point with respect to the axis coordinates, and then as shown in FIG. 43, the control apparatus 5 updates data from the designed offset components to the obtained real offset components Δx, Δy, Δz, Δu, Δv, Δw (step S228) [paragraphs 468-477]).
Regarding Claim 2. Yukihiro teaches the device of claim 1.
Yukihiro also teaches:
wherein the processor is further configured to:
determine the direction about the axis arranged at the trial measurement position, as the orientation change direction (Implied; it isn’t explicit, but the robot has to determine the direction about the axis to perform the orientation change. In FIG. 6, this is shown as angle θ);
operate the robot so as to rotate the vision sensor or the index mark in the orientation change direction in order to change the orientation by the second orientation change amount (FIG. 4, S270, paragraph 20); and
acquire a position of the line of sight in the hand-tip coordinate system as the trial measurement position and the real measurement position (FIG. 9 shows the line of sight at AX1 passing through the marker MK, with angle θ1 between the table TB and the line of sight. This marker is the same circle CT in FIG. 5 in the robot coordinate system RC [paragraph 197]).
Regarding Claim 3. Yukihiro teaches the device of claim 1.
Yukihiro also teaches:
wherein the processor is further configured to:
operate the robot so as to rotate the vision sensor or the index mark in a direction about the axis orthogonal to the direction of the line of sight in order to change the orientation by the first orientation change amount (FIG. 6, paragraph 200, wherein angle θ represents the angle that the frame is rotated around LN);
determine the direction about the axis arranged at an orientation reference position which is determined based on the trial measurement position, as the orientation change direction (Implied; it isn’t explicit, but the robot has to determine the direction about the axis to perform the orientation change. In FIG. 6, this is shown as angle θ);
operate the robot so as to rotate the vision sensor or the index mark in the orientation change direction in order to change the orientation by the second orientation change amount (FIG. 4, S270, paragraph 20); and
acquire the position of the vision sensor in the direction of the line of sight in the hand-tip coordinate system, as the trial measurement position and the real measurement position (FIG. 9 shows the line of sight at AX1 passing through the marker MK, with angle θ1 between the table TB and the line of sight. This marker is the same circle CT in FIG. 5 in the robot coordinate system RC [paragraph 197]).
Regarding Claim 4. Yukihiro teaches the device of claim 3.
Yukihiro also teaches:
wherein the processor is configured to:
acquire, based on the image data imaged by the vision sensor before changing the orientation by the second orientation change amount, a relative position of the index mark with respect to the vision sensor when the vision sensor images the image data (Paragraph 164); and
determine the orientation reference position with reference to the trial measurement position such that the acquired relative position and a relative position of the orientation reference position with respect to the trial measurement position are identical (In another aspect of the invention, the robot may be configured such that distances between the imaging unit and the first marker are the same between the first position and attitude and the second position and attitude [paragraph 12]. FIG. 9 shows this is more detail, with the same angle θ1 on either side, and in FIG. 10, the distances between each position from the marker labeled L1 and L2 can be equivalent).
Regarding Claim 6. Yukihiro teaches the device of claim 1.
Yukihiro also teaches:
wherein the device is a teaching device or a control device of the robot (paragraphs 2-3).
Regarding Claim 7. Yukihiro teaches a robot system comprising:
a vision sensor (FIG. 1, number 21);
a robot configured to relatively move the vision sensor and an index mark (FIG. 1, the robot at 20, and the marker at MK); and
the device of claim 1 (see Regarding Claim 1).
Regarding Claim 8. Yukihiro teaches a method of acquiring a positional relationship between a sensor coordinate system set to a vision sensor and a hand-tip coordinate system set to a hand-tip of a robot configured to relatively move the vision sensor and an index mark (Paragraphs 4, 53. FIG. 1, the marker at MK is the first marker [paragraph 155], shown to be located within the line of sight of the imaging unit 21 in FIG. 9. At the same time, FIG. 37 shows a schematic view of the robot with a coordinate system at the point of the manipulator, with another coordinate system from the image sensor shown as “xb” and “yb”. This is shown again in FIG. 42, with the process for updating the relationship and offset between the image coordinate system and the hand-tip coordinate system shown in FIG. 43), the method comprising, by a processor:
acquiring a direction of a line of sight of the vision sensor in the hand-tip control coordinate system in advance (Line “AX1” in FIG. 9, shown to be in the coordinate system in the bottom left corner of FIG. 9 and referred to as the optical axis of the imaging unit 21 [paragraph 215], which is the same imaging unit 21 in FIG. 1);
operating the robot so as to change an orientation of the vision sensor or the index mark in a direction about an axis parallel to the direction of the line of sight, by a first orientation change amount (FIG. 6, paragraph 200, wherein angle θ represents the angle that the frame is rotated. Note that this rotation is made around an axis parallel to the line of sight at AX1 in FIG. 9);
acquiring, first image data of the index mark imaged by the vision sensor after the orientation is changed by the first orientation change amount (The first position reads on a trial measurement position [paragraph 19], wherein the image of the marker is captured by the imaging unit [paragraph 163-165]);
operating the robot so as to change the orientation of the vision sensor or the index mark in a direction about an axis orthogonal to the direction of the line of sight, by a second orientation change amount (The second position in paragraph 19 reads on the second orientation. Note FIG. 4, wherein the robot rotates the arm when determining the new position [paragraph 184]. In step S270 of FIG. 4, paragraph 20, the third predetermined angle is where the robot ends up after a second orientation change);
acquiring the second image data of the index mark imaged by the vision sensor after the orientation is changed by the second orientation change amount;
acquiring, as a trial measurement position, first coordinates of an origin of the sensor coordinate system in the hand-tip coordinate system, based on the first and second image data, and third image data of the index mark imaged by the vision sensor before the orientation is changed (The imaging unit at 21 of FIG. 1 is a camera, and a position of the captured imaging unit is expressed by X and Y coordinates in an imaging coordinate system CC [paragraph 149], and the position of the imaging unit is determined at least in part by the location of the first marker and the positions determined in FIG. 4 [paragraphs 212-214, FIGS. 4 and 9]);
operating the robot so as to change the orientation by a third orientation change amount larger than the first orientation change amount in an orientation change direction which is determined based on the trial measurement position;
acquiring, as a real measurement position, second coordinates of the origin of the sensor coordinate system in the hand-tip coordinate system, based on image data of the index mark imaged by the vision sensor before and after the orientation is changed by the second orientation change amount (The position and attitude calculation part 45 detects the first marker MK contained in the zeroth image XP3 based on the zeroth image XP3. The position and attitude calculation part 45 calculates a third detection position as a position of the first marker MK detected from the zeroth image XP3 in the imaging unit coordinate system CC. The position and attitude calculation part 45 calculates a displacement from a coincidence point in the zeroth image XP3 to the calculated third detection position in the imaging unit coordinate system CC [paragraph 202]. The position and attitude calculation part 45 transforms the calculated displacement into a displacement in the robot coordinate system RC based on the above described coordinate transform matrix. Then, the position and attitude calculation part 45 calculates third position and attitude obtained by translation of the current position and attitude (i.e. first position and attitude) of the control point T by the displacement based on the transformed displacement and the current position and attitude of the control point T. The third position and attitude are a position and an attitude in the robot coordinate system RC. That is, when the position and the attitude of the control point T coincide with the third position and attitude, the position of the first marker MK contained in the captured image captured by the imaging unit 21 coincides with the coincidence point in the captured image. This is largely a repeat of the steps for finding the second position, and is shown in more detail in FIGS. 7 and 8); and
updating coordinates of the sensor coordinate system in the hand-tip coordinate system stored in a memory from the first coordinates acquired as the trial measurement position to the second coordinates acquired as the real measurement position (FIG. 43 also shows the process by which the control apparatus obtains the offset component of the imaging reference point with respect to the axis coordinates, and then as shown in FIG. 43, the control apparatus 5 updates data from the designed offset components to the obtained real offset components Δx, Δy, Δz, Δu, Δv, Δw (step S228) [paragraphs 468-477]).
Regarding Claim 9. Yukihiro teaches a computer-readable storage medium configured to storage a computer program configured to cause a processor to execute the method of claim 8 (Paragraph 504).
Regarding Claim 10. Yukihiro teaches a device configured to acquire a positional relationship between a sensor coordinate system set to a vision sensor and a hand-tip coordinate system set to a hand-tip of a robot configured to relatively move the vision sensor and an index mark (Paragraphs 4, 53. FIG. 1, the marker at MK is the first marker [paragraph 155], shown to be located within the line of sight of the imaging unit 21 in FIG. 9. At the same time, FIG. 37 shows a schematic view of the robot with a coordinate system at the point of the manipulator, with another coordinate system from the image sensor shown as “xb” and “yb”. This is shown again in FIG. 42, with the process for updating the relationship and offset between the image coordinate system and the hand-tip coordinate system shown in FIG. 43), the device comprising:
a processor configured to:
acquire a direction of a line of sight of the vision sensor in the hand-tip coordinate system in advance (Line “AX1” in FIG. 9, shown to be in the coordinate system in the bottom left corner of FIG. 9 and referred to as the optical axis of the imaging unit 21 [paragraph 215], which is the same imaging unit 21 in FIG. 1);
set a reference coordinate system in the hand-tip coordinate system (paragraphs 366 and 454, FIG. 47), the reference coordinate system including a first axis parallel to the direction of the line of sight and a second axis orthogonal to the direction of the line of sight (FIG. 9 shows the line of sight at AX1 passing through the marker MK, with angle θ1 between the table TB and the line of sight. This marker is the same circle CT in FIG. 5 in the robot coordinate system RC [paragraph 197]. FIG. 6 shows both the reference coordinate system at RC, and the rotation around a first axis at point VT which is perpendicular to the axis of the line of sight at the center of MK, shown again in FIG. 9);
operate the robot so as to change an orientation of the vision sensor or the index mark in a direction about the first axis or the second axis, by a first orientation change amount (FIG. 6, paragraph 200, wherein angle θ represents the angle that the frame is rotated. Note that this rotation is made around an axis parallel to the line of sight at AX1 in FIG. 9);
acquire, as a trial measurement position, first coordinates of an origin of the sensor coordinate system in the hand-tip coordinate system, based on image data of the index mark imaged by the vision sensor before and after the orientation is changed by the first orientation change amount (The first position reads on a trial measurement position [paragraph 19], wherein the image of the marker is captured by the imaging unit [paragraph 163-165]);
operate the robot so as to change the orientation by a second orientation change amount larger than the first orientation change amount in an orientation change direction which is determined based on the trial measurement position;
acquire, as a real measurement position, second coordinates of the origin of the sensor coordinate system in the hand-tip coordinate system, based on image data of the index mark imaged by the vision sensor before and after the orientation is changed by the second orientation change amount (The position and attitude calculation part 45 detects the first marker MK contained in the zeroth image XP3 based on the zeroth image XP3. The position and attitude calculation part 45 calculates a third detection position as a position of the first marker MK detected from the zeroth image XP3 in the imaging unit coordinate system CC. The position and attitude calculation part 45 calculates a displacement from a coincidence point in the zeroth image XP3 to the calculated third detection position in the imaging unit coordinate system CC [paragraph 202]. The position and attitude calculation part 45 transforms the calculated displacement into a displacement in the robot coordinate system RC based on the above described coordinate transform matrix. Then, the position and attitude calculation part 45 calculates third position and attitude obtained by translation of the current position and attitude (i.e. first position and attitude) of the control point T by the displacement based on the transformed displacement and the current position and attitude of the control point T. The third position and attitude are a position and an attitude in the robot coordinate system RC. That is, when the position and the attitude of the control point T coincide with the third position and attitude, the position of the first marker MK contained in the captured image captured by the imaging unit 21 coincides with the coincidence point in the captured image. This is largely a repeat of the steps for finding the second position, and is shown in more detail in FIGS. 7 and 8); and
update coordinates of the sensor coordinate system in the hand-tip coordinate system stored in a memory from the first coordinates acquired as the trial measurement position to the second coordinates acquired as the real measurement position (FIG. 43 also shows the process by which the control apparatus obtains the offset component of the imaging reference point with respect to the axis coordinates, and then as shown in FIG. 43, the control apparatus 5 updates data from the designed offset components to the obtained real offset components Δx, Δy, Δz, Δu, Δv, Δw (step S228) [paragraphs 468-477]).
Regarding Claim 11. Yukihiro teaches a device configured to acquire a positional relationship between a sensor coordinate system set to a vision sensor and a hand-tip coordinate system set to a hand tip of a robot configured to relatively move the vision sensor and an index mark (Paragraphs 4, 53. FIG. 1, the marker at MK is the first marker [paragraph 155], shown to be located within the line of sight of the imaging unit 21 in FIG. 9. At the same time, FIG. 37 shows a schematic view of the robot with a coordinate system at the point of the manipulator, with another coordinate system from the image sensor shown as “xb” and “yb”. This is shown again in FIG. 42, with the process for updating the relationship and offset between the image coordinate system and the hand-tip coordinate system shown in FIG. 43), the device comprising:
a processor configured to:
acquire a direction of a line of sight of the vision sensor in the hand-tip coordinate system in advance (Line “AX1” in FIG. 9, shown to be in the coordinate system in the bottom left corner of FIG. 9 and referred to as the optical axis of the imaging unit 21 [paragraph 215], which is the same imaging unit 21 in FIG. 1);
set a reference coordinate system in the hand-tip coordinate system (paragraphs 366 and 454, FIG. 47), the reference coordinate system including a first axis parallel to the direction of the line of sight and a second axis orthogonal to the direction of the line of sight (FIG. 6);
operate the robot so as to change an orientation of the vision sensor or the index mark in a direction about the first axis, by a first orientation change amount (FIG. 9 shows the line of sight at AX1 passing through the marker MK, with angle θ1 between the table TB and the line of sight. This marker is the same circle CT in FIG. 5 in the robot coordinate system RC [paragraph 197]. FIG. 6 shows both the reference coordinate system at RC, and the rotation around a first axis at point VT which is perpendicular to the axis of the line of sight at the center of MK, shown again in FIG. 9);
acquire first image data of the index mark imaged by the vision sensor after the orientation is changed by the first orientation change amount (The first position reads on a trial measurement position [paragraph 19], wherein the image of the marker is captured by the imaging unit [paragraph 163-165]);
operate the robot so as to change the orientation of the vision sensor or the index mark in a direction about the second axis, by a second orientation change amount;
acquire second image data of the index mark imaged by the vision sensor after the orientation is changed by the second orientation change amount;
acquire, as a trial measurement position, first coordinates of an origin of the sensor coordinate system in the hand-tip coordinate system, based on the first and second image data, and third image data of the index mark imaged by the vision sensor before the orientation is changed;
operate the robot so as to change the orientation by a third orientation change amount larger than the first and second orientation change amounts in an orientation change direction which is determined based on the trial measurement position; and
acquire, as a real measurement position, second coordinates of the origin of the sensor coordinate system in the hand-tip coordinate system, based on image data of the index mark imaged by the vision sensor before and after the orientation is changed by the third orientation change amount (The position and attitude calculation part 45 detects the first marker MK contained in the zeroth image XP3 based on the zeroth image XP3. The position and attitude calculation part 45 calculates a third detection position as a position of the first marker MK detected from the zeroth image XP3 in the imaging unit coordinate system CC. The position and attitude calculation part 45 calculates a displacement from a coincidence point in the zeroth image XP3 to the calculated third detection position in the imaging unit coordinate system CC [paragraph 202]. The position and attitude calculation part 45 transforms the calculated displacement into a displacement in the robot coordinate system RC based on the above described coordinate transform matrix. Then, the position and attitude calculation part 45 calculates third position and attitude obtained by translation of the current position and attitude (i.e. first position and attitude) of the control point T by the displacement based on the transformed displacement and the current position and attitude of the control point T. The third position and attitude are a position and an attitude in the robot coordinate system RC. That is, when the position and the attitude of the control point T coincide with the third position and attitude, the position of the first marker MK contained in the captured image captured by the imaging unit 21 coincides with the coincidence point in the captured image. This is largely a repeat of the steps for finding the second position, and is shown in more detail in FIGS. 7 and 8); and
update coordinates of the sensor coordinate system in the hand-tip coordinate system stored in a memory from the first coordinates acquired as the trial measurement position to the second coordinates acquired as the real measurement position (FIG. 43 also shows the process by which the control apparatus obtains the offset component of the imaging reference point with respect to the axis coordinates, and then as shown in FIG. 43, the control apparatus 5 updates data from the designed offset components to the obtained real offset components Δx, Δy, Δz, Δu, Δv, Δw (step S228) [paragraphs 468-477]).
Regarding Claim 12. Yukihiro teaches the device of claim 1.
Yukihiro also teaches:
wherein the processor is configured to acquire the direction of the line of sight of the vision sensor in the hand-tip coordinate system, based on the image data of the index mark imaged by the vision sensor before and after translation movement of the vision sensor of the index mark in a predetermined direction by the robot (FIG. 4 shows an example of the processing performed by the robot control apparatus. At step S260, the position and attitude calculation part 45 determines whether or not the calculated position and a coincidence point in the zeroth image XP4 coincide, and thereby, determines whether or not the current position and attitude of the control point T and the third position and attitude coincide (step S260). This is performed because the position of the first marker MK in the imaging unit coordinate system CC and the coincidence point in the zeroth image XP4 may not coincide due to an error of the above described coordinate transform matrix, an error caused by rigidity of the robot 20, or the like [paragraph 205], which reads on acquiring the direction of the line of sight of the vision sensor in the control system).
Regarding Claim 13. Yukihiro teaches the device of claim 1.
Yukihiro also teaches:
wherein the processor is configured to
acquire relative positions of the index mark with respect to the vision sensor when the vision sensor images the image data (paragraph 164);
acquire, as the trial measurement position (paragraphs 149-151), a position of the vision sensor in the hand-tip coordinate system, based on the first orientation change amount, a first relative position when the vision sensor images the image data before the orientation is changed (The imaging unit at 21 of FIG. 1 is a camera, and a position of the captured imaging unit is expressed by X and Y coordinates in an imaging coordinate system CC [paragraph 149], and the position of the imaging unit is determined at least in part by the location of the first marker and the positions determined in FIG. 4 [paragraphs 212-214, FIGS. 4 and 9]), and a second relative position when the vision sensor images the image data after the orientation is changed (The second position in paragraph 19 reads on the second orientation. Note FIG. 4, wherein the robot rotates the arm when determining the new position [paragraph 184]. In step S270 of FIG. 4, paragraph 20, the third predetermined angle is where the robot ends up after a second orientation change); and
set a reference coordinate system at an orientation reference position separated from the trial measurement position by the first relative position, the reference coordinate system including a first axis parallel to the direction of the line of sight and a second axis orthogonal to the direction of the line of sight (The position and attitude calculation part 45 detects the first marker MK contained in the zeroth image XP3 based on the zeroth image XP3. The position and attitude calculation part 45 calculates a third detection position as a position of the first marker MK detected from the zeroth image XP3 in the imaging unit coordinate system CC. The position and attitude calculation part 45 calculates a displacement from a coincidence point in the zeroth image XP3 to the calculated third detection position in the imaging unit coordinate system CC [paragraph 202]. The position and attitude calculation part 45 transforms the calculated displacement into a displacement in the robot coordinate system RC based on the above described coordinate transform matrix. Then, the position and attitude calculation part 45 calculates third position and attitude obtained by translation of the current position and attitude (i.e. first position and attitude) of the control point T by the displacement based on the transformed displacement and the current position and attitude of the control point T. The third position and attitude are a position and an attitude in the robot coordinate system RC. That is, when the position and the attitude of the control point T coincide with the third position and attitude, the position of the first marker MK contained in the captured image captured by the imaging unit 21 coincides with the coincidence point in the captured image. This is largely a repeat of the steps for finding the second position, and is shown in more detail in FIGS. 7 and 8).
Regarding Claim 14. Yukihiro teaches the device of claim 10.
Yukihiro also teaches:
wherein the processor is configured to acquire the direction of the line of sight of the vision sensor in the hand-tip coordinate system, based on the image data of the index mark imaged by the vision sensor before and after translation movement of the vision sensor of the index mark in an axis- direction of the hand-tip coordinate system by the robot (FIG. 4 shows an example of the processing performed by the robot control apparatus. At step S260, the position and attitude calculation part 45 determines whether or not the calculated position and a coincidence point in the zeroth image XP4 coincide, and thereby, determines whether or not the current position and attitude of the control point T and the third position and attitude coincide (step S260). This is performed because the position of the first marker MK in the imaging unit coordinate system CC and the coincidence point in the zeroth image XP4 may not coincide due to an error of the above described coordinate transform matrix, an error caused by rigidity of the robot 20, or the like [paragraph 205], which reads on acquiring the direction of the line of sight of the vision sensor in the control system).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Yukihiro et al. US 20180004188 A1 (“Yukihiro”).
Regarding Claim 5. Yukihiro teaches the device of claim 1.
Yukihiro also teaches:
wherein the vision sensor includes an image sensor configured to receive a subject image, and an optical lens configured to focus the subject image onto the image sensor (FIG. 37, which shows that the fixed camera has an imaging device at 23 including a CCD image sensor, and a lens at 22 [paragraph 375]), and
wherein the processor is configured to:
acquire a relative position of the index mark with respect to the vision sensor when the vision sensor images the image data, based on a position of the index mark in the image data, a size of the index mark shown in the image data, a size of the index mark in a real space, a focal distance of the optical lens, and a size of the image sensor (The markers will nearly have the same size as one another [paragraph 403]. In at least one embodiment, the shape and size of the second marker at 72 of FIG. 40 in the image captured by the mobile camera and the shape and size of the first marker in the first image in the memory unit are compared [paragraph 436]. The coordinates of the camera are also obtained [paragraph 427], meaning that the height and distance from the marker would be easy to determine. While it is not explicit that the distance is used to acquire a relative position of the index mark with respect to the vision sensor, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Yukihiro to include the distance in determining the relative position of the index mark because the distance from the vision sensor from the marker is important in determining the size of the marker);
acquire the trial measurement position based on the first orientation change amount (The first position reads on a trial measurement position [paragraph 19], wherein the image of the marker is captured by the imaging unit [paragraph 163-165]), the relative position when the image data is imaged before changing the orientation by the first orientation change amount (Paragraph 164), and the relative position when the image data is imaged after changing the orientation by the first orientation change amount (Paragraph 165); and
acquire the real measurement position based on the second orientation change amount, the relative position when the image data is imaged before changing the orientation by the second orientation change amount, and the relative position when the image data is imaged after changing the orientation by the third orientation change amount (The imaging unit at 21 of FIG. 1 is a camera, and a position of the captured imaging unit is expressed by X and Y coordinates in an imaging coordinate system CC [paragraph 149], and the position of the imaging unit is determined at least in part by the location of the first marker and the positions determined in FIG. 4, as well as the data collected before and after changing the image sensor’s position and orientation [paragraphs 164-165 and 212-214, FIGS. 4 and 9]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON G CAIN whose telephone number is (571)272-7009. The examiner can normally be reached Monday: 7:30am - 4:30pm EST to Friday 7:30pm - 4:30am.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached at (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON G CAIN/Examiner, Art Unit 3656
Prosecution Insights