Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office action is in response to the amendment filed on 08/25/2026. Claims 1-2, 4, and 6-9 are currently pending with claims 1, 8, and 9 being amended, and claims 3 and 5 being cancelled.
Response to Amendment
The amendments to the claims submitted on 08/25/2026 overcome the claim objections set forth in the previous Office action except for those set forth in the claim objection section.
Response to Arguments
Examiner notes wherein Applicant argues the newly amended limitations, which have not been addressed by the prior art of record. As such, Examiner has augmented the below rejection(s) in view of the prior art of record to address the newly amended limitations.
Applicant's arguments filed 08/25/2026 with respect to the rejections of claims 1-9 under 35 U.S.C. 103 have been fully considered but they are not persuasive. The Applicant has argued that the control device is limited to movement only in the xy plane which is perpendicular to the z axis. Movement along the z axis would involve either move closer to the xy plane which parallels the pattern or further away. This limitation to the functionality of the robot does not appear to be supported by the written disclosure. The specification and figures present a six axis robot which would be able to move in the direction of the z axis. The specification does not disclose limiting the movement to be only within the xy plane. Further, the cited prior art discloses in at least paragraph 0034 of Hwang that “The driving member 112 is driven to actuate the mechanical arm 110 so as to adjust a position of the movable end 114 along a direction X1 and a direction Y1 in parallel with a plane where the positioning pattern 400 is located. The center point 822A of the positioning image 820A is thus allowed to be moved to the image center 802A of the comparison image 800A.”. This demonstrates movement parallel to the is independent of the movement perpendicular to the plane. This is further demonstrated in at least Figure 3 of Hwang where the steps S603 describe the movement in the xy plane and S605 the movement along the z axis respectively. This suggests the functionality which is claimed of moving the center of gravity of the pattern towards the center of the image by moving in a plane parallel to the pattern plane. Further, Huang discloses performing calibration calculations in order to identify coordinates to move the robot closer to a desired position iteratively. This, in combination with the other cited references, would suggest performing calculations on the portion of the image and then moving parallel to the image in order to iteratively calibrate the robotic system.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 and 9 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. The written disclosure does not support the limiting of motion of the distal end of the robot to a direction along the x axis and/or the y axis. The written disclosure and drawings disclose a six-axis robot which moves away from the calibration pattern as well as moving in a xy plane parallel to the pattern.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 4 and 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang et al. (US 20180111271 A1), hereinafter Hwang in view of Tonogai et al. (US 20190047152 A1), hereinafter Tonogai and Huang et al. (US 9193073 B1), hereinafter Huang.
Regarding claim 1, Hwang teaches:
1. (Currently Amended) A robot system comprising;
a robot;
a control device configured to control the robot; (Paragraph 0037, "Since the mechanical arm positioning method 600 first adjusts the movable end 114 to position the center point of the positioning image at the image center of the comparison image, for example, the center point 822B of the positioning image 820B is overlapped with the image center 802B of the comparison image 800B so that the movable end 114 is collinear with the fixed point A along the direction Z1 perpendicular to the plane of the positioning pattern 400. After that, the movable end 114 is adjusted along the direction Z1 such that the area of the positioning image to be substantially equal to the predetermined area, for example, such that the area A.sub.2 of the positioning image 920A′ to be substantially equal to the predetermined area A.sub.0. As a result, the movable end 114 can be adjusted to the fixed point A from the other moving points P1, P2, P3 in the space with the assistance of the image-capturing module 200. Even more, the computing device 300 can further perform the mechanical arm positioning method 600 automatically to achieve full automation of the positioning of the mechanical arm system 100 through judging the comparison image captured by the image-capturing module 200 to actuate the mechanical arm 110 correspondingly.")
… and
a camera attached to one of the positioning target object and a distal end portion of the robot, wherein a processor of the control device is configured to cause the camera to capture an image (Paragraph 0012, "Another aspect of the present invention is related to a mechanical arm system that utilizes the image-capturing module disposed at the movable end of the mechanical arm to capture the positioning pattern so as to generate the comparison image with the image of the positioning pattern. In addition, distance relationships between movable end and the fixed point along various axes in the space are determined through comparing the relative position and relative area between the image of the positioning pattern and the comparison image so as to drive the driving member to adjust the movable end to the fixed point. As a result, the movable end of the mechanical arm can be more accurately positioned at the fixed point, and the amount of computation and computation time required for adjusting the mechanical arm are reduced to reduce the burden of the computing device and the length of the computation time. At the same time, the time required for repositioning is reduced.") of a pattern which includes (Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns.") and wherein, in a case in which only a portion of the pattern is included in the image, and, in a case in which the robot system has a coordinate system including a Z axis penetrating the camera and the pattern, an X axis, and a Y axis perpendicular to the Z axis, the processor is configured to cause the robot to move only in a direction along the X axis and/or the Y axis (Paragraph 0034, "The driving member 112 is driven to actuate the mechanical arm 110 so as to adjust a position of the movable end 114 along a direction X1 and a direction Y1 in parallel with a plane where the positioning pattern 400 is located.") so that a center of gravity position … moves toward a center of the image. (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.")
Hwang does not specifically teach a positioning target object, determining an origin of the robot or target object, or the calculation of desired movement for calibration based on only a portion of the pattern. However, Tonogai, in the same field of endeavor of robotics, teaches:
… a positioning target object; (Paragraph 0019, "According to an aspect, the image data is changed so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm, and thus, an image pattern with a size appropriate for calibration can be displayed in accordance with a relative positional relationship between the display device and the image capture device. A calibration of a coordinate system of an image capture device and a coordinate system of a robot arm is performed to improve the accuracy of predetermined processing for an object using a robot arm (e.g. gripping, suction, fitting, winding etc. of the object). Accordingly, the calibration accuracy can be improved by changing the image data so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm that acts on an object, and performing a calibration using a plurality of types of captured images that are based on different image patterns.") … from which an origin coordinate (Paragraph 0080, "Next, the coordinates of the display 22 (calibration object) are obtained. Specifically, the coordinates of the display 22 are obtained based on the known shape data (length data) of the display 22, with the leading end coordinates of the robot arm R serving as a reference. If the position and orientation of the display 22 (calibration object) relative to the leading end coordinate system of the robot arm R are changeable, the coordinates of the display 22 are obtained based not only on the shape data of the display 22, but also on a changed position and orientation. Next, the coordinates of the display 22 are obtained with the origin coordinates of the robot arm R serving as a reference, based on the coordinates of the display 22 relative to the leading end coordinate system of the robot arm R.") and which is attached to the other one of the positioning target object and the distal end portion of the robot, (Please see Figures 2 and 6) …
However, Huang, in the same field of endeavor of robotics, teaches:
… of the plurality of figures or the plurality of characteristic shapes of the portion of the pattern … (Column 3, Lines 9-19, “A method of calibrating a robot arm includes positioning a visual system of the robot arm to have at least a part of an encoded calibrating plate within a photographic range of the visual system, focusing the visual system to capture an image of the encoded calibrating plate, determining position of the visual system according to coordinates indicated in a coordinates encoding in the image of the encoded calibration plate and a focus of the eye in hand camera, comparing the coordinates of the eye in hand camera before and after positioning to calculate a moving error, and calibrating the position of the eye in hand camera.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Hwang with the target surface and ability to determine an origin of the robot and other objects relative to the pattern as taught by Tonogai. While Hwang is silent on the plane which contains the calibration pattern being a surface which has the object being worked upon, Tonogai specifically discusses the processing of the object on the surface which contains the calibration pattern. Combining the calibration methods which Hwang uses with the ability to determine relative positioning of the robot and other structures within the environment as taught by Tonogai and further with the functionality taught by Huang to perform calibration movement calculations despite there only being a portion of the pattern would ensure that the robot may accurately perform calibration even with limited space (Huang col. 1, paragraph 5) and to perform operations on the object being processed with a higher degree of accuracy.
Regarding claim 2, where all the limitations of claim 1 are discussed above, Hwang further teaches:
2. (Previously Presented) The robot system according to claim 1, wherein the control device is further configured to determine whether or not a whole of the first feature is included in the image acquired by the camera, (Paragraph 0014, "In the foregoing, the computing device is further configured to determine a magnitude relationship between the area of the positioning image and the predetermined area. The driving member is driven such that the movable end to move away from the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area. The driving member is driven to adjust the mechanical arm such that the movable end to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning is smaller than the predetermined area.") … based on the first feature in the image (Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns.") when it is determined that the whole of the first feature is included in the image. (Paragraph 0014, "In the foregoing, the computing device is further configured to determine a magnitude relationship between the area of the positioning image and the predetermined area. The driving member is driven such that the movable end to move away from the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area. The driving member is driven to adjust the mechanical arm such that the movable end to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning is smaller than the predetermined area.")
Hwang does not specifically teach determining the origin relative to the display. However, Tonogai, in the same field of endeavor of robotics, teaches:
… and acquire the origin coordinate of the other one of the positioning target object and the robot (Paragraph 0080, "Next, the coordinates of the display 22 (calibration object) are obtained. Specifically, the coordinates of the display 22 are obtained based on the known shape data (length data) of the display 22, with the leading end coordinates of the robot arm R serving as a reference. If the position and orientation of the display 22 (calibration object) relative to the leading end coordinate system of the robot arm R are changeable, the coordinates of the display 22 are obtained based not only on the shape data of the display 22, but also on a changed position and orientation. Next, the coordinates of the display 22 are obtained with the origin coordinates of the robot arm R serving as a reference, based on the coordinates of the display 22 relative to the leading end coordinate system of the robot arm R.") …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Hwang with the ability to determine an origin of the robot and other objects relative to the pattern as taught by Tonogai. Combining the calibration methods which Hwang uses with the ability to determine relative positioning of the robot and other structures within the environment as taught by Tonogai would ensure that the robot may accurately perform operations on the object being processed.
Regarding claim 4, where all the limitations of claim 1 are discussed above, Hwang further teaches:
4. (Previously Presented) The robot system according to claim 1, wherein the pattern includes a second feature that is composed of at least a portion of the plurality of figures or the plurality of characteristic shapes and that is distributed in a predetermined range around the first feature. (Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns." Examiner note: The first "feature" may be considered to be the center of the code pattern and the surrounding pattern may be considered to be a second feature.)
Regarding claim 9, Hwang further teaches:
9. (Currently Amended) A calibration method comprising:
capturing an image, with a camera (Paragraph 0012, "Another aspect of the present invention is related to a mechanical arm system that utilizes the image-capturing module disposed at the movable end of the mechanical arm to capture the positioning pattern so as to generate the comparison image with the image of the positioning pattern. In addition, distance relationships between movable end and the fixed point along various axes in the space are determined through comparing the relative position and relative area between the image of the positioning pattern and the comparison image so as to drive the driving member to adjust the movable end to the fixed point. As a result, the movable end of the mechanical arm can be more accurately positioned at the fixed point, and the amount of computation and computation time required for adjusting the mechanical arm are reduced to reduce the burden of the computing device and the length of the computation time. At the same time, the time required for repositioning is reduced.") attached to one of a distal end portion of a robot (Paragraph 0037, "Since the mechanical arm positioning method 600 first adjusts the movable end 114 to position the center point of the positioning image at the image center of the comparison image, for example, the center point 822B of the positioning image 820B is overlapped with the image center 802B of the comparison image 800B so that the movable end 114 is collinear with the fixed point A along the direction Z1 perpendicular to the plane of the positioning pattern 400. After that, the movable end 114 is adjusted along the direction Z1 such that the area of the positioning image to be substantially equal to the predetermined area, for example, such that the area A.sub.2 of the positioning image 920A′ to be substantially equal to the predetermined area A.sub.0. As a result, the movable end 114 can be adjusted to the fixed point A from the other moving points P1, P2, P3 in the space with the assistance of the image-capturing module 200. Even more, the computing device 300 can further perform the mechanical arm positioning method 600 automatically to achieve full automation of the positioning of the mechanical arm system 100 through judging the comparison image captured by the image-capturing module 200 to actuate the mechanical arm 110 correspondingly.") and … a pattern which includes(Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns.")
determining whether or not a whole of the first feature is included in the image captured by the camera, … when it is determined that the whole of the first feature is included in the image; (Paragraph 0014, "In the foregoing, the computing device is further configured to determine a magnitude relationship between the area of the positioning image and the predetermined area. The driving member is driven such that the movable end to move away from the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area. The driving member is driven to adjust the mechanical arm such that the movable end to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning is smaller than the predetermined area.")
controlling the robot, in a case in which only a portion of the pattern is included in the image, and, in a case in which the robot system has a coordinate system including a Z axis penetrating the camera and the pattern, an X axis, and a Y axis perpendicular to the Z axis, to cause the distal end of the robot to move only in a direction along the X axis and/or the Y axis (Paragraph 0034, "The driving member 112 is driven to actuate the mechanical arm 110 so as to adjust a position of the movable end 114 along a direction X1 and a direction Y1 in parallel with a plane where the positioning pattern 400 is located.") so that moves toward a center of the image. (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.")
Hwang does not specifically teach a positioning target object, determining an origin of the robot or target object, or the calculation of desired movement for calibration based on only a portion of the pattern. However, Tonogai, in the same field of endeavor of robotics, teaches:
… a positioning target object (Paragraph 0019, "According to an aspect, the image data is changed so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm, and thus, an image pattern with a size appropriate for calibration can be displayed in accordance with a relative positional relationship between the display device and the image capture device. A calibration of a coordinate system of an image capture device and a coordinate system of a robot arm is performed to improve the accuracy of predetermined processing for an object using a robot arm (e.g. gripping, suction, fitting, winding etc. of the object). Accordingly, the calibration accuracy can be improved by changing the image data so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm that acts on an object, and performing a calibration using a plurality of types of captured images that are based on different image patterns.") … from which an origin coordinate and which is attached to the other one of the positioning target object and the distal end portion of the robot, (Please see Figures 2 and 6) … acquiring the origin coordinate based on the first feature in the image (Paragraph 0080, "Next, the coordinates of the display 22 (calibration object) are obtained. Specifically, the coordinates of the display 22 are obtained based on the known shape data (length data) of the display 22, with the leading end coordinates of the robot arm R serving as a reference. If the position and orientation of the display 22 (calibration object) relative to the leading end coordinate system of the robot arm R are changeable, the coordinates of the display 22 are obtained based not only on the shape data of the display 22, but also on a changed position and orientation. Next, the coordinates of the display 22 are obtained with the origin coordinates of the robot arm R serving as a reference, based on the coordinates of the display 22 relative to the leading end coordinate system of the robot arm R.") …
However, Huang, in the same field of endeavor of robotics, teaches:
… of the plurality of figures or the plurality of characteristic shapes of the portion of the pattern … (Column 3, Lines 9-19, “A method of calibrating a robot arm includes positioning a visual system of the robot arm to have at least a part of an encoded calibrating plate within a photographic range of the visual system, focusing the visual system to capture an image of the encoded calibrating plate, determining position of the visual system according to coordinates indicated in a coordinates encoding in the image of the encoded calibration plate and a focus of the eye in hand camera, comparing the coordinates of the eye in hand camera before and after positioning to calculate a moving error, and calibrating the position of the eye in hand camera.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Hwang with the target surface and ability to determine an origin of the robot and other objects relative to the pattern as taught by Tonogai. While Hwang is silent on the plane which contains the calibration pattern being a surface which has the object being worked upon, Tonogai specifically discusses the processing of the object on the surface which contains the calibration pattern. Combining the calibration methods which Hwang uses with the ability to determine relative positioning of the robot and other structures within the environment as taught by Tonogai and further with the functionality taught by Huang to perform calibration movement calculations despite there only being a portion of the pattern would ensure that the robot may accurately perform calibration even with limited space (Huang col. 1, paragraph 5) and to perform operations on the object being processed with a higher degree of accuracy.
Claim(s) 6-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang in view of Tonogai and Huang and in further view of and Tate et al. (US 20090190826 A1), hereinafter Tate.
Regarding claim 6, where all the limitations of claim 4 are discussed above, Hwang further teaches:
6. (Previously Presented) The robot system according to claim 4, wherein the control device is further configured to … using the plurality of figures or the plurality of characteristic shapes of the whole of the pattern. (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.")
Hwang does not specifically discuss calculating a center of a calibration pattern. However, Tate, in the same field of endeavor of robotics, teaches:
… calculate the center of gravity position …(Paragraph 0012, “Also, according to another aspect of the present invention, there is provided a calibration method of a working apparatus equipped with a working unit which executes work on a work subject, and a calibration jig on which is arranged a plurality of markers in a radial pattern from a center point of markers, the plurality of marker being distributed in three dimensions, the calibration jig being attached to the working unit such that a set calibration reference point of the working unit matches with the center point of markers, and the method comprising: an image capture step of capturing the working unit; a calculation step of calculating image coordinates of a center point of markers based on a marker image existing in an image captured by the image capture step; and a calibration step of calibrating a transform process to transform image coordinates and apparatus coordinates based on coordinates of a center point of markers calculated by the calculation step and apparatus coordinates of the reference point of the working unit.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic calibration methods and system as taught by Hwang with the ability to identify a reference center point as taught by Tate. This would allow the system to be calibrated using a variety of calibration patterns and without predefined knowledge of the center/reference point. Further, these methods make it “possible to execute calibration of a working unit position even when a portion containing a marker center part is obscured during image measurement” (See Tate, Paragraph 0099).
Regarding claim 7, where all the limitations of claim 4 are discussed above, Hwang does not specifically discuss calculating a center of a calibration pattern or applying weights to different features within the pattern. However, Tate, in the same field of endeavor of robotics, teaches:
7. (Previously Presented) The robot system according to claims 4, wherein the control device is further configured to calculate the center of gravity position (Paragraph 0012, “Also, according to another aspect of the present invention, there is provided a calibration method of a working apparatus equipped with a working unit which executes work on a work subject, and a calibration jig on which is arranged a plurality of markers in a radial pattern from a center point of markers, the plurality of marker being distributed in three dimensions, the calibration jig being attached to the working unit such that a set calibration reference point of the working unit matches with the center point of markers, and the method comprising: an image capture step of capturing the working unit; a calculation step of calculating image coordinates of a center point of markers based on a marker image existing in an image captured by the image capture step; and a calibration step of calibrating a transform process to transform image coordinates and apparatus coordinates based on coordinates of a center point of markers calculated by the calculation step and apparatus coordinates of the reference point of the working unit.”) …
However, Huang, in the same field of endeavor of robotics, teaches:
… by giving a larger weight to the first feature than to the second feature included in the image. (Col 3, Line 58-Col 4, Line 23, "FIG. 2 illustrates a diagram of an encoded calibration plate 20 of a robot calibration apparatus according to an embodiment of the present invention. The encoded calibration plate 20 may have a chessboard pattern. The chessboard pattern may comprise of interchanging black squares 21 and white squares 22. The black squares 21 may have orientation encodings for indicating an orientation of the encoded calibration plate 20 and the white squares 22 may have coordinates encodings for indicating positions on the encoded calibration plate 20. An orientation encoding of a black square 21 may include an icon 23 positioned near a corner of the black square 21 to indicate that an origin O of the encoded calibration plate 20 may be close to a corresponding corner of the encoded calibration plate 20. A coordinates encoding in a white square 22 may comprise encoding icons arranged in a matrix to indicate a position of the white square 22 relative to the origin O of the encoded calibration plate 20. In an embodiment, the encoding icons in a white square 22 may comprise of only solid icons 25, or a combination of both hollow icons 24 and solid icons 25 arranged in two columns. As shown in FIG. 2, a first column on one side of the white square 22 may be used to represent an X-coordinate of the encoded calibration plate 20 and a second column on another side of the white square 22 may be used to represent a Y-coordinate of the encoded calibration plate 20. Each of the X-coordinate and Y-coordinate may be represented by a combination of an encoding icon in row A, an encoding icon in row B, and an encoding icon in row C. Each encoding icon may represent a binary bit. Encoding icons from row A may have a binary weight of 2.sup.0, encoding icons from row B may have a binary weight of 2.sup.1, and encoding icons from row C may have a binary weight of 2.sup.2. The hollow icons 24 may represent a bit 0 value and the solid icons 25 may represent a bit 1 value.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic calibration methods and system as taught by Hwang with the ability to identify a reference center point as taught by Tate and further with the ability to provide different weights to different portions of the calibration pattern as taught by Huang. This would allow the system to be calibrated using a variety of calibration patterns and without predefined knowledge of the center/reference point. Further, these methods make it “possible to execute calibration of a working unit position even when a portion containing a marker center part is obscured during image measurement” (See Tate, Paragraph 0099). Incorporating weights into the calibration code/pattern as taught by Huang would allow the system to determine the positioning of the visual more efficiently even when the image does not include the entire pattern and allow the system to determine the appropriate movements to make in order to reach a calibrated state.
Regarding claim 8, where all the limitations of claim 3 are discussed above, Hwang does not specifically discuss calculating a center of a calibration pattern or the use of a single feature in the pattern. However, Tate, in the same field of endeavor of robotics, teaches:
8. (Currently Amended) The robot system according to claim 1, wherein the control device is further configured to calculate the center of gravity position (Paragraph 0012, “Also, according to another aspect of the present invention, there is provided a calibration method of a working apparatus equipped with a working unit which executes work on a work subject, and a calibration jig on which is arranged a plurality of markers in a radial pattern from a center point of markers, the plurality of marker being distributed in three dimensions, the calibration jig being attached to the working unit such that a set calibration reference point of the working unit matches with the center point of markers, and the method comprising: an image capture step of capturing the working unit; a calculation step of calculating image coordinates of a center point of markers based on a marker image existing in an image captured by the image capture step; and a calibration step of calibrating a transform process to transform image coordinates and apparatus coordinates based on coordinates of a center point of markers calculated by the calculation step and apparatus coordinates of the reference point of the working unit.”) …
However, Tonogai, in the same field of endeavor of robotics, teaches:
… using only the first feature included in the image. (Paragraph 0075, "FIG. 3 shows an example of an initial setting procedure for the display device D. Initially, the type of calibration pattern to be displayed on the display 22 is determined (S31). FIGS. 4A to 4D show examples of calibration patterns. FIG. 4A shows a dot image pattern constituted by a square frame line and a total of 49 (7 rows by 7 columns) black dots that are arranged at regular intervals in this frame line. Meanwhile, one of the corners is painted in a triangular shape to specify the direction of the square. That is to say, the calibration pattern shown in FIG. 4A is a pattern that includes black dots arranged at predetermined intervals in the frame line, and a polygonal mark marker for specifying the direction of the pattern. FIG. 4B shows image patterns of an AR marker (left) and a two-dimensional barcode (right). FIG. 4C shows an example of a checkerboard pattern (an image pattern in which black squares and white squares are arranged alternately), and FIG. 4D shows an example of an image pattern constituted by triangles. Various image patterns can be used in accordance with the positional relationship between the robot R and the image sensor S, the purpose of calibration, required accuracy, or the like. For example, the number and size of black dots in the aforementioned dot pattern can be set arbitrarily. In an embodiment, the dot image pattern is selected. The display control unit 20 is configured to output image data for displaying the selected image pattern to the display 22.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic calibration methods and system as taught by Hwang with the ability to identify a reference center point as taught by Tate as well as with the ability to use different types of patterns, including those with a single feature as taught by Tonogai to perform the calibration of the system. This would allow the system to be calibrated using a variety of calibration patterns and without predefined knowledge of the center/reference point while maintaining efficient and accurate operation. Further, these methods make it “possible to execute calibration of a working unit position even when a portion containing a marker center part is obscured during image measurement” (See Tate, Paragraph 0099).
Conclusion
The Examiner has cited particular paragraphs or columns and line numbers in the referencesapplied to the claims above for the convenience of the Applicant. Although the specified citations arerepresentative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the Applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP 2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed Invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATHER KENIRY whose telephone number is (571)270-5468. The examiner can normally be reached M-F 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached at (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.J.K./Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights