Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office action is in response to the amendment filed on 04/27/2026. Claims 1-9 are currently pending with claims 1, 3-6, and 9 being amended.
Response to Amendment
The amendments to the claims submitted on 04/27/2026 overcome the claim objections set forth in the previous Office action except for those set forth in the claim objection section.
Response to Arguments
Examiner notes wherein Applicant argues the newly amended limitations, which have not been addressed by the prior art of record. As such, Examiner has augmented the below rejection(s) in view of the prior art of record to address the newly amended limitations.
Applicant’s arguments, see amendment, filed 04/27/2026, with respect to rejections of claims 1, 5, and 9 under 35 U.S.C. 112(b) have been fully considered and are persuasive. The rejections of claims 1, 5, and 9 under 35 U.S.C. 112(b) has been withdrawn.
Applicant's arguments filed 04/27/2026 with respect to the rejections of claims 1-9 under 35 U.S.C. 103 have been fully considered but they are not persuasive. The Applicant has asserted that the prior art of record does not teach or suggest the claimed invention because the cited art of Hwang acquires an image and moves such that a known center position is aligned with the center of the image. The claims do not preclude a known center position. They only require that “a center of gravity position of the plurality of figures or the plurality of characteristic shapes of the portion of the pattern moves toward a center of the image.” (emphasis added). The broadest reasonable interpretation of the limitation does not require the calculation of this position or that it is unknown prior to the image being take. Further, there is no requirement that the “center of gravity position” coincides with a “center of the image” or a definition of what the “center of the image” entails. The pattern is also not required to be a non-uniform pattern, only that it includes “a plurality of figures or a plurality of characteristic shapes which, if the pattern is uniform, the “center of gravity” of the pattern” would coincide with the center of the image. Accordingly, the broadest reasonable interpretation of the claim would encompass a uniform or non-uniform pattern being imaged by the system and movement which causes the center of the pattern to move toward a center of the image which may be a center point, a center line (horizontal or vertical), a center region etc. for at least a portion of the movement. The claim does not require that the position reaches any destination or that the movement is always towards the center. The movement must only, at some point, bring the center of gravity position towards the center of the image.
The applicant has also asserted that the claimed invention is not suggested because the cited prior art may move in the Z direction in order to capture an entirety of the pattern in the image. In Hwang Fig. 7, the step of S1006 identifies whether or not the pattern is entirely shown and responds by moving in the Z direction. This action is not precluded by the currently provided claim language. There is no requirement that no movement other than the claimed movement is performed, and further, moving in the Z direction would move the center of the pattern towards the center of the image frame along with the image frame capturing more of the pattern. This appears to be the case in claim 5 when the claimed invention is attempting to capture the “whole of the first feature” in the image and the system moves the robot away from the pattern.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., calculation of a centroid etc.) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4 and 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang et al. (US 20180111271 A1), hereinafter Hwang in view of Tonogai et al. (US 20190047152 A1), hereinafter Tonogai.
Regarding claim 1, Hwang teaches:
1. (Currently Amended) A robot system comprising;
a robot;
a control device configured to control the robot; (Paragraph 0037, "Since the mechanical arm positioning method 600 first adjusts the movable end 114 to position the center point of the positioning image at the image center of the comparison image, for example, the center point 822B of the positioning image 820B is overlapped with the image center 802B of the comparison image 800B so that the movable end 114 is collinear with the fixed point A along the direction Z1 perpendicular to the plane of the positioning pattern 400. After that, the movable end 114 is adjusted along the direction Z1 such that the area of the positioning image to be substantially equal to the predetermined area, for example, such that the area A.sub.2 of the positioning image 920A′ to be substantially equal to the predetermined area A.sub.0. As a result, the movable end 114 can be adjusted to the fixed point A from the other moving points P1, P2, P3 in the space with the assistance of the image-capturing module 200. Even more, the computing device 300 can further perform the mechanical arm positioning method 600 automatically to achieve full automation of the positioning of the mechanical arm system 100 through judging the comparison image captured by the image-capturing module 200 to actuate the mechanical arm 110 correspondingly.")
… and
a camera attached to one of the positioning target object and the robot, wherein a processor of the control device is configured to cause the camera to capture an image (Paragraph 0012, "Another aspect of the present invention is related to a mechanical arm system that utilizes the image-capturing module disposed at the movable end of the mechanical arm to capture the positioning pattern so as to generate the comparison image with the image of the positioning pattern. In addition, distance relationships between movable end and the fixed point along various axes in the space are determined through comparing the relative position and relative area between the image of the positioning pattern and the comparison image so as to drive the driving member to adjust the movable end to the fixed point. As a result, the movable end of the mechanical arm can be more accurately positioned at the fixed point, and the amount of computation and computation time required for adjusting the mechanical arm are reduced to reduce the burden of the computing device and the length of the computation time. At the same time, the time required for repositioning is reduced.") of a patternincluding a first feature … the pattern including a plurality of figures or a plurality of characteristic shapes, (Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns.") and wherein, in a case in which only a portion of the pattern is included in the image, the processor is configured to cause the robot to move so that a center of gravity position of the plurality of figures or the plurality of characteristic shapes of the portion of the pattern moves toward (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.")
Hwang does not specifically teach a positioning target object or determining an origin of the robot or target object. However, Tonogai, in the same field of endeavor of robotics, teaches:
… a positioning target object; (Paragraph 0019, "According to an aspect, the image data is changed so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm, and thus, an image pattern with a size appropriate for calibration can be displayed in accordance with a relative positional relationship between the display device and the image capture device. A calibration of a coordinate system of an image capture device and a coordinate system of a robot arm is performed to improve the accuracy of predetermined processing for an object using a robot arm (e.g. gripping, suction, fitting, winding etc. of the object). Accordingly, the calibration accuracy can be improved by changing the image data so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm that acts on an object, and performing a calibration using a plurality of types of captured images that are based on different image patterns.") … from which an origin coordinate of the other one of the positioning target object and the robot can be acquired, (Paragraph 0080, "Next, the coordinates of the display 22 (calibration object) are obtained. Specifically, the coordinates of the display 22 are obtained based on the known shape data (length data) of the display 22, with the leading end coordinates of the robot arm R serving as a reference. If the position and orientation of the display 22 (calibration object) relative to the leading end coordinate system of the robot arm R are changeable, the coordinates of the display 22 are obtained based not only on the shape data of the display 22, but also on a changed position and orientation. Next, the coordinates of the display 22 are obtained with the origin coordinates of the robot arm R serving as a reference, based on the coordinates of the display 22 relative to the leading end coordinate system of the robot arm R.") …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Hwang with the target surface and ability to determine an origin of the robot and other objects relative to the pattern as taught by Tonogai. While Hwang is silent on the plane which contains the calibration pattern being a surface which has the object being worked upon, Tonogai, specifically discusses the processing of the object on the surface which contains the calibration pattern. Combining the calibration methods which Hwang uses with the ability to determine relative positioning of the robot and other structures within the environment as taught by Tonogai would ensure that the robot may accurately perform operations on the object being processed.
Regarding claim 2, where all the limitations of claim 1 are discussed above, Hwang further teaches:
2. (Previously Presented) The robot system according to claim 1, wherein the control device is further configured to determine whether or not a whole of the first feature is included in the image acquired by the camera, (Paragraph 0014, "In the foregoing, the computing device is further configured to determine a magnitude relationship between the area of the positioning image and the predetermined area. The driving member is driven such that the movable end to move away from the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area. The driving member is driven to adjust the mechanical arm such that the movable end to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning is smaller than the predetermined area.") … based on the first feature in the image (Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns.") when it is determined that the whole of the first feature is included in the image. (Paragraph 0014, "In the foregoing, the computing device is further configured to determine a magnitude relationship between the area of the positioning image and the predetermined area. The driving member is driven such that the movable end to move away from the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area. The driving member is driven to adjust the mechanical arm such that the movable end to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning is smaller than the predetermined area.")
Hwang does not specifically teach determining the origin relative to the display. However, Tonogai, in the same field of endeavor of robotics, teaches:
… and acquire the origin coordinate of the other one of the positioning target object and the robot (Paragraph 0080, "Next, the coordinates of the display 22 (calibration object) are obtained. Specifically, the coordinates of the display 22 are obtained based on the known shape data (length data) of the display 22, with the leading end coordinates of the robot arm R serving as a reference. If the position and orientation of the display 22 (calibration object) relative to the leading end coordinate system of the robot arm R are changeable, the coordinates of the display 22 are obtained based not only on the shape data of the display 22, but also on a changed position and orientation. Next, the coordinates of the display 22 are obtained with the origin coordinates of the robot arm R serving as a reference, based on the coordinates of the display 22 relative to the leading end coordinate system of the robot arm R.") …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Hwang with the ability to determine an origin of the robot and other objects relative to the pattern as taught by Tonogai. Combining the calibration methods which Hwang uses with the ability to determine relative positioning of the robot and other structures within the environment as taught by Tonogai would ensure that the robot may accurately perform operations on the object being processed.
Regarding claim 3, where all the limitations of claim 2 are discussed above, Hwang further teaches:
3. (Currently Amended) The robot system according to claim 2, wherein the control device is further configured to operate the robot so that the center of gravity position of the plurality of figures or the plurality of characteristic shapes moves toward (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.") when it is determined that the whole of the first feature is not included in the image acquired by the camera. (Paragraph 0014, "In the foregoing, the computing device is further configured to determine a magnitude relationship between the area of the positioning image and the predetermined area. The driving member is driven such that the movable end to move away from the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area. The driving member is driven to adjust the mechanical arm such that the movable end to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning is smaller than the predetermined area.")
Regarding claim 4, where all the limitations of claim 1 are discussed above, Hwang further teaches:
4. (Currently Amended) The robot system according to claim 1, wherein the pattern that is composed of at least a portion of the plurality of figures or the plurality of characteristic shapes and that is distributed in a predetermined range around the first feature. (Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns." Examiner note: The first "feature" may be considered to be the center of the code pattern and the surrounding pattern may be considered to be a second feature.)
Regarding claim 9, Hwang further teaches:
9. (Currently Amended) A calibration method comprising:
capturing an image, with a camera (Paragraph 0012, "Another aspect of the present invention is related to a mechanical arm system that utilizes the image-capturing module disposed at the movable end of the mechanical arm to capture the positioning pattern so as to generate the comparison image with the image of the positioning pattern. In addition, distance relationships between movable end and the fixed point along various axes in the space are determined through comparing the relative position and relative area between the image of the positioning pattern and the comparison image so as to drive the driving member to adjust the movable end to the fixed point. As a result, the movable end of the mechanical arm can be more accurately positioned at the fixed point, and the amount of computation and computation time required for adjusting the mechanical arm are reduced to reduce the burden of the computing device and the length of the computation time. At the same time, the time required for repositioning is reduced.") attached to one of a robot (Paragraph 0037, "Since the mechanical arm positioning method 600 first adjusts the movable end 114 to position the center point of the positioning image at the image center of the comparison image, for example, the center point 822B of the positioning image 820B is overlapped with the image center 802B of the comparison image 800B so that the movable end 114 is collinear with the fixed point A along the direction Z1 perpendicular to the plane of the positioning pattern 400. After that, the movable end 114 is adjusted along the direction Z1 such that the area of the positioning image to be substantially equal to the predetermined area, for example, such that the area A.sub.2 of the positioning image 920A′ to be substantially equal to the predetermined area A.sub.0. As a result, the movable end 114 can be adjusted to the fixed point A from the other moving points P1, P2, P3 in the space with the assistance of the image-capturing module 200. Even more, the computing device 300 can further perform the mechanical arm positioning method 600 automatically to achieve full automation of the positioning of the mechanical arm system 100 through judging the comparison image captured by the image-capturing module 200 to actuate the mechanical arm 110 correspondingly.") and … a patternthe pattern including a plurality of figures or a plurality of characteristic shapes; (Paragraph 0031, "A description is provided with reference to FIG. 1 and FIG. 2. The image-capturing module 200 is fixed to the movable end 114, and can freely move in a space with the movable end 114. In other embodiments, the image-capturing module 200 may be further fixed to a position beside the gripping unit 116. The image-capturing module 200 may be configured to capture a positioning pattern 400 in a field of view 220 at different moving points, such as the fixed point A, moving points P1, P2, P3, etc., and generate a comparison image with a positioning image, for example, comparison images 800A-900B and positioning images 820A-920B depicted in FIG. 5A to FIG. 6B. However, the present invention is not limited in this regard, and a detailed description is provided as follows. The positioning image corresponds to the positioning pattern 400. In one embodiment, the positioning pattern 400 may be a two-dimensional QR code or some other suitable two-dimensional patterns.")
determining whether or not a whole of the first feature is included in the image captured by the camera, … when it is determined that the whole of the first feature is included in the image; (Paragraph 0014, "In the foregoing, the computing device is further configured to determine a magnitude relationship between the area of the positioning image and the predetermined area. The driving member is driven such that the movable end to move away from the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area. The driving member is driven to adjust the mechanical arm such that the movable end to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning is smaller than the predetermined area.")
controlling the robot, in a case in which only a portion of the pattern is included in the image, to make a center of gravity position of the plurality of figures or the plurality of characteristic shapes of the portion of the pattern move toward a center of the image (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.")
Hwang does not specifically teach a positioning target object or determining an origin of the robot or target object. However, Tonogai, in the same field of endeavor of robotics, teaches:
… a positioning target object (Paragraph 0019, "According to an aspect, the image data is changed so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm, and thus, an image pattern with a size appropriate for calibration can be displayed in accordance with a relative positional relationship between the display device and the image capture device. A calibration of a coordinate system of an image capture device and a coordinate system of a robot arm is performed to improve the accuracy of predetermined processing for an object using a robot arm (e.g. gripping, suction, fitting, winding etc. of the object). Accordingly, the calibration accuracy can be improved by changing the image data so as to differentiate the size of the image pattern in accordance with the coordinates of the leading end of the robot arm that acts on an object, and performing a calibration using a plurality of types of captured images that are based on different image patterns.") … from which an origin coordinate of the other one of the robot and the positioning target object can be acquired, … acquiring the origin coordinate based on the first feature in the image (Paragraph 0080, "Next, the coordinates of the display 22 (calibration object) are obtained. Specifically, the coordinates of the display 22 are obtained based on the known shape data (length data) of the display 22, with the leading end coordinates of the robot arm R serving as a reference. If the position and orientation of the display 22 (calibration object) relative to the leading end coordinate system of the robot arm R are changeable, the coordinates of the display 22 are obtained based not only on the shape data of the display 22, but also on a changed position and orientation. Next, the coordinates of the display 22 are obtained with the origin coordinates of the robot arm R serving as a reference, based on the coordinates of the display 22 relative to the leading end coordinate system of the robot arm R.") …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Hwang with the target surface and ability to determine an origin of the robot and other objects relative to the pattern as taught by Tonogai. While Hwang is silent on the plane which contains the calibration pattern being a surface which has the object being worked upon, Tonogai, specifically discusses the processing of the object on the surface which contains the calibration pattern. Combining the calibration methods which Hwang uses with the ability to determine relative positioning of the robot and other structures within the environment as taught by Tonogai would ensure that the robot may accurately perform operations on the object being processed.
Claim(s) 5-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang in view of Tonogai and in further view of Huang et al. (US 9193073 B1), hereinafter Huang and Tate et al. (US 20090190826 A1), hereinafter Tate.
Regarding claim 5, where all the limitations of claim 1 are discussed above, Hwang further teaches:
5. (Currently Amended) The robot system according to claim 4, wherein the control device is further configured to cause the camera to capture an image of the pattern (Paragraph 0012, "Another aspect of the present invention is related to a mechanical arm system that utilizes the image-capturing module disposed at the movable end of the mechanical arm to capture the positioning pattern so as to generate the comparison image with the image of the positioning pattern. In addition, distance relationships between movable end and the fixed point along various axes in the space are determined through comparing the relative position and relative area between the image of the positioning pattern and the comparison image so as to drive the driving member to adjust the movable end to the fixed point. As a result, the movable end of the mechanical arm can be more accurately positioned at the fixed point, and the amount of computation and computation time required for adjusting the mechanical arm are reduced to reduce the burden of the computing device and the length of the computation time. At the same time, the time required for repositioning is reduced.") after moving the robot in a direction away from the pattern (Paragraph 0008, "In the foregoing, the step of determining whether the area of the positioning image is substantially equal to a predetermined area comprises: determining a magnitude relationship between the area of the positioning image and the predetermined area; adjusting the mechanical arm such that the mechanical arm to move away from the positioning pattern along a direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is larger than the predetermined area; and adjusting the mechanical arm such that the mechanical arm to move closer to the positioning pattern along the direction perpendicular to the plane where the positioning pattern is located if the area of the positioning image is smaller than the predetermined area.") and the (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.")
Hwang does not specifically discuss calculating a center of a calibration pattern. However, Tate, in the same field of endeavor of robotics, teaches:
… , which is calculated, …(Paragraph 0012, “Also, according to another aspect of the present invention, there is provided a calibration method of a working apparatus equipped with a working unit which executes work on a work subject, and a calibration jig on which is arranged a plurality of markers in a radial pattern from a center point of markers, the plurality of marker being distributed in three dimensions, the calibration jig being attached to the working unit such that a set calibration reference point of the working unit matches with the center point of markers, and the method comprising: an image capture step of capturing the working unit; a calculation step of calculating image coordinates of a center point of markers based on a marker image existing in an image captured by the image capture step; and a calibration step of calibrating a transform process to transform image coordinates and apparatus coordinates based on coordinates of a center point of markers calculated by the calculation step and apparatus coordinates of the reference point of the working unit.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic calibration methods and system as taught by Hwang with the ability to identify a reference center point as taught by Tate. This would allow the system to be calibrated using a variety of calibration patterns and without predefined knowledge of the center/reference point. Further, these methods make it “possible to execute calibration of a working unit position even when a portion containing a marker center part is obscured during image measurement” (See Tate, Paragraph 0099).
Regarding claim 6, where all the limitations of claim 4 are discussed above, Hwang further teaches:
6. (Currently Amended) The robot system according to claim 4, wherein the control device is further configured to … using the plurality of figures or the plurality of characteristic shapes of the whole of the pattern (Paragraph 0013, "The invention provides a mechanical arm system. The mechanical arm system comprises a mechanical arm, an image-capturing module, and a computing device. The mechanical arm comprises a movable end and at least one driving member. The driving member is configured to move the movable end to a fixed point. The image-capturing module is fixed to the movable end. The image-capturing module is configured to capture a positioning pattern at a moving point so as to generate a comparison image with a positioning image. The positioning image corresponds to the positioning pattern. The computing device is configured to determine whether a center of the positioning image is located at a center of the comparison image. If not, the driving member is driven to adjust a position of the movable end in parallel with a plane where the positioning pattern is located such that the center of the positioning image to be located at the center of the comparison image. The computing device is further configured to determine whether an area of the positioning image is substantially equal to a predetermined area. If not, the driving member is driven to adjust a position of the movable end along a direction perpendicular to the plane where the positioning pattern is located to change a distance between the image-capturing module and the positioning pattern so as such that the area of the positioning image to be substantially equal to the predetermined area.")
Hwang does not specifically discuss calculating a center of a calibration pattern. However, Tate, in the same field of endeavor of robotics, teaches:
… calculate the center of gravity position …(Paragraph 0012, “Also, according to another aspect of the present invention, there is provided a calibration method of a working apparatus equipped with a working unit which executes work on a work subject, and a calibration jig on which is arranged a plurality of markers in a radial pattern from a center point of markers, the plurality of marker being distributed in three dimensions, the calibration jig being attached to the working unit such that a set calibration reference point of the working unit matches with the center point of markers, and the method comprising: an image capture step of capturing the working unit; a calculation step of calculating image coordinates of a center point of markers based on a marker image existing in an image captured by the image capture step; and a calibration step of calibrating a transform process to transform image coordinates and apparatus coordinates based on coordinates of a center point of markers calculated by the calculation step and apparatus coordinates of the reference point of the working unit.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic calibration methods and system as taught by Hwang with the ability to identify a reference center point as taught by Tate. This would allow the system to be calibrated using a variety of calibration patterns and without predefined knowledge of the center/reference point. Further, these methods make it “possible to execute calibration of a working unit position even when a portion containing a marker center part is obscured during image measurement” (See Tate, Paragraph 0099).
Regarding claim 7, where all the limitations of claim 4 are discussed above, Hwang does not specifically discuss calculating a center of a calibration pattern or applying weights to different features within the pattern. However, Tate, in the same field of endeavor of robotics, teaches:
7. (Previously Presented) The robot system according to claims 4, wherein the control device is further configured to calculate the center of gravity position (Paragraph 0012, “Also, according to another aspect of the present invention, there is provided a calibration method of a working apparatus equipped with a working unit which executes work on a work subject, and a calibration jig on which is arranged a plurality of markers in a radial pattern from a center point of markers, the plurality of marker being distributed in three dimensions, the calibration jig being attached to the working unit such that a set calibration reference point of the working unit matches with the center point of markers, and the method comprising: an image capture step of capturing the working unit; a calculation step of calculating image coordinates of a center point of markers based on a marker image existing in an image captured by the image capture step; and a calibration step of calibrating a transform process to transform image coordinates and apparatus coordinates based on coordinates of a center point of markers calculated by the calculation step and apparatus coordinates of the reference point of the working unit.”) …
However, Huang, in the same field of endeavor of robotics, teaches:
… by giving a larger weight to the first feature than to the second feature included in the image. (Col 3, Line 58-Col 4, Line 23, "FIG. 2 illustrates a diagram of an encoded calibration plate 20 of a robot calibration apparatus according to an embodiment of the present invention. The encoded calibration plate 20 may have a chessboard pattern. The chessboard pattern may comprise of interchanging black squares 21 and white squares 22. The black squares 21 may have orientation encodings for indicating an orientation of the encoded calibration plate 20 and the white squares 22 may have coordinates encodings for indicating positions on the encoded calibration plate 20. An orientation encoding of a black square 21 may include an icon 23 positioned near a corner of the black square 21 to indicate that an origin O of the encoded calibration plate 20 may be close to a corresponding corner of the encoded calibration plate 20. A coordinates encoding in a white square 22 may comprise encoding icons arranged in a matrix to indicate a position of the white square 22 relative to the origin O of the encoded calibration plate 20. In an embodiment, the encoding icons in a white square 22 may comprise of only solid icons 25, or a combination of both hollow icons 24 and solid icons 25 arranged in two columns. As shown in FIG. 2, a first column on one side of the white square 22 may be used to represent an X-coordinate of the encoded calibration plate 20 and a second column on another side of the white square 22 may be used to represent a Y-coordinate of the encoded calibration plate 20. Each of the X-coordinate and Y-coordinate may be represented by a combination of an encoding icon in row A, an encoding icon in row B, and an encoding icon in row C. Each encoding icon may represent a binary bit. Encoding icons from row A may have a binary weight of 2.sup.0, encoding icons from row B may have a binary weight of 2.sup.1, and encoding icons from row C may have a binary weight of 2.sup.2. The hollow icons 24 may represent a bit 0 value and the solid icons 25 may represent a bit 1 value.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic calibration methods and system as taught by Hwang with the ability to identify a reference center point as taught by Tate and further with the ability to provide different weights to different portions of the calibration pattern as taught by Huang. This would allow the system to be calibrated using a variety of calibration patterns and without predefined knowledge of the center/reference point. Further, these methods make it “possible to execute calibration of a working unit position even when a portion containing a marker center part is obscured during image measurement” (See Tate, Paragraph 0099). Incorporating weights into the calibration code/pattern as taught by Huang would allow the system to determine the positioning of the visual more efficiently even when the image does not include the entire pattern and allow the system to determine the appropriate movements to make in order to reach a calibrated state.
Regarding claim 8, where all the limitations of claim 3 are discussed above, Hwang does not specifically discuss calculating a center of a calibration pattern or the use of a single feature in the pattern. However, Tate, in the same field of endeavor of robotics, teaches:
8. (Previously Presented) The robot system according to claim 3, wherein the control device is further configured to calculate the center of gravity position (Paragraph 0012, “Also, according to another aspect of the present invention, there is provided a calibration method of a working apparatus equipped with a working unit which executes work on a work subject, and a calibration jig on which is arranged a plurality of markers in a radial pattern from a center point of markers, the plurality of marker being distributed in three dimensions, the calibration jig being attached to the working unit such that a set calibration reference point of the working unit matches with the center point of markers, and the method comprising: an image capture step of capturing the working unit; a calculation step of calculating image coordinates of a center point of markers based on a marker image existing in an image captured by the image capture step; and a calibration step of calibrating a transform process to transform image coordinates and apparatus coordinates based on coordinates of a center point of markers calculated by the calculation step and apparatus coordinates of the reference point of the working unit.”) …
However, Tonogai, in the same field of endeavor of robotics, teaches:
… using only the first feature included in the image. (Paragraph 0075, "FIG. 3 shows an example of an initial setting procedure for the display device D. Initially, the type of calibration pattern to be displayed on the display 22 is determined (S31). FIGS. 4A to 4D show examples of calibration patterns. FIG. 4A shows a dot image pattern constituted by a square frame line and a total of 49 (7 rows by 7 columns) black dots that are arranged at regular intervals in this frame line. Meanwhile, one of the corners is painted in a triangular shape to specify the direction of the square. That is to say, the calibration pattern shown in FIG. 4A is a pattern that includes black dots arranged at predetermined intervals in the frame line, and a polygonal mark marker for specifying the direction of the pattern. FIG. 4B shows image patterns of an AR marker (left) and a two-dimensional barcode (right). FIG. 4C shows an example of a checkerboard pattern (an image pattern in which black squares and white squares are arranged alternately), and FIG. 4D shows an example of an image pattern constituted by triangles. Various image patterns can be used in accordance with the positional relationship between the robot R and the image sensor S, the purpose of calibration, required accuracy, or the like. For example, the number and size of black dots in the aforementioned dot pattern can be set arbitrarily. In an embodiment, the dot image pattern is selected. The display control unit 20 is configured to output image data for displaying the selected image pattern to the display 22.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic calibration methods and system as taught by Hwang with the ability to identify a reference center point as taught by Tate as well as with the ability to use different types of patterns, including those with a single feature as taught by Tonogai to perform the calibration of the system. This would allow the system to be calibrated using a variety of calibration patterns and without predefined knowledge of the center/reference point while maintaining efficient and accurate operation. Further, these methods make it “possible to execute calibration of a working unit position even when a portion containing a marker center part is obscured during image measurement” (See Tate, Paragraph 0099).
Conclusion
The Examiner has cited particular paragraphs or columns and line numbers in the referencesapplied to the claims above for the convenience of the Applicant. Although the specified citations arerepresentative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the Applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP 2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed Invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
/H.J.K./Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights