Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s Amendments filed on September 3, 2025, has been entered and made of record.
Independent Claim(s) 1 and 11-12
Amended Claim(s) 1 and 11-12
Canceled Claim(s) 2
Currently Pending Claim(s) 1-12
Response to Arguments
This office action is responsive to Applicant’s Arguments/Remarks Made in an Amendment
received on September 3, 2025.
In view of amendments filed on September 3, 2025, the Applicant has amended claims 1 and 11-12 to incorporate partial subject matter from the dependent claim 5. Specifically, claims 1 and 11-12 now indicate that the first calibration tool comprises a detection apparatus which specifies a relative position of the second calibration tool with respect to the first calibration tool.
This subject matter was addressed in the Non-Final Rejection mailed on August 8, 2025. The Examiner indicated that the reference of Tamir (US 2019/0052865 A1), when applied in combination with the reference of Zhang (US 8,964,027 B2), rendered the use of a calibration tool comprising a detection apparatus to detect the second calibration tool obvious. This is because Zhang teaches the well-known calibration method which utilizes the constraint of a known relative distance between calibration tools, and Tamir teaches calibration between drones, which are calibration tools which each comprise a detection apparatus for detecting one another for calibration. Thus, the rejections have been updated in the body of rejection below to address the amendments.
In view of Applicant Arguments/Remarks filed on September 3, 2025, with respect to the pending claims, the Applicant argues on Page 6 of the Remarks that neither Zhang nor Kyung (KR 101931955 B1) teaches the content of the newly amended claims 1 and 11-12, because they fail to teach a calibration method where the first calibration tool comprises a detection apparatus for detecting the second calibration tool so that the calibration tools can be independently mobile. The Examiner respectfully disagrees. First, the Examiner stated in the previous office action that Zhang and Kyung, either taken alone or in combination, fail to teach independently mobile calibration tools. However, Tamir, paragraph 0022, teaches the independently mobile calibration tools each comprising detection apparatuses for detecting one another. And, in the rejection of claim 5 in the Non-Final Rejection, the Examiner showed that Tamir, when applied to Zhang (in combination with Kyung), renders this limitation obvious. Thus, the Examiner maintains the rejection to claims 1 and 11-12; see the updated rejections including the art of Tamir in the body of this office action below.
Regarding claim 4, the Applicant argues on page 6 of the Remarks that claim 4 should become allowable due to its dependence on claim 1. However, the Examiner maintains the rejections to the independent claims 1 and 11-12; thus, the rejection to claim 4 is maintained. Riberio ("Photogrammetric Multi-Camera Calibration Using An Industrial Programmable Robotic Arm," 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), Dubrovnik, Croatia, 2019, pp. 288-294, doi: 10.1109/ISPA.2019.8868928.) teaches using a robotic arm for mounting calibration tools in paragraph 2 on page 4, and the Examiner stated in the previous office action that utilizing a robot arm for mounting calibration tools is rendered obvious by Riberio.
Regarding claims 5-10, the Applicant argues on page 6 of the Remarks that claims 5-10 should be allowable due to their dependence on claim 1. Additionally, the Applicant argued that Tamir fails to make up for the deficiencies with regard to claim 1. However, the Examiner believes that the combination of Zhang and Tamir render the amended limitation in claim 1 obvious as discussed above and has updated the rejection to include the reference of Tamir. Thus, the rejections to the dependent claims 5-10 is maintained. Tamir teaches the use of drones, each comprising a detection apparatus to detect one another, for mounting calibration tools in paragraphs 0013, 0021, and 0022. The Examiner stated in the previous office action that utilizing drones for the purpose of moving calibration tools and detecting the relative position of adjacent drones/tools is rendered obvious by Tamir.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, and 5-12 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 8964027 B2), hereafter Zhang, further in view of Kyung et al. (KR 101931955 B1), hereafter Kyung, and Tamir et al. (US 20190052865 A1), hereafter Tamir.
Regarding claim 1, Zhang teaches a calibration system (Fig. 1) comprising:
a first calibration tool; a second calibration tool (Figure 3; Column 2, Line 24 "a rigid bar fasten with two targets respectively corresponding to the vision sensors." Targets are calibration tools.),
a first sensor configured to be able to detect the first calibration tool, wherein the first sensor comprises a camera, a shape acquisition sensor, a position detection sensor, or a light-receiving element; a second sensor configured to be able to detect the second calibration tool, wherein the second sensor comprises a camera, a shape acquisition sensor, a position detection sensor, or a light-receiving element (Fig. 3 "vision sensor 1" and "vision sensor 2"; Column 2, Line 26 "corresponding to the vision sensors; capturing images of the respective targets by their corresponding vision sensors; extracting coordinates of the feature points of the respective targets on their corresponding images; and computing 3D coordinates of each feature points of the respective targets under their corresponding vision sensor coordinate frames;"); and
a processor configured to calculate the relative position of the second sensor with respect to the first sensor based on the result of detecting the first calibration tool by the first sensor and the result of detecting the second calibration tool by the second sensor (Fig. 2 "the rotation matrix" and "the translation vector" defining the relative position between the two sensors; Fig. 3; Column 2, line 50 “In the Solutions above, computing the transformation matrix between the two vision sensors may include: computing the rotation matrix and translation vectors; establishing an objective function on the basis of minimizing the re-projection error, Solving an non-linear optimal Solution for the rotation matrix and translation vectors to acquire the transformation matrix between the two vision sensors, by using a non-linear optimization approach.”);
wherein the first sensor is at least one of an RGB camera, a motion capture camera, a thermal camera, a shape acquisition sensor, a position detection sensor, and a light-receiving element, and the second sensor is one of an RGB camera, a motion capture camera, a thermal camera, a shape acquisition sensor, a position detection sensor, and a light-receiving element (Fig. 3 "vision sensor 1" and "vision sensor 2"; Column 9, Line 46 “In this embodiment here, the multi-sensor vision system to be calibrated consists of two vision sensors—vision sensor 1 and 2. The two sensors are both Aigo digital cameras with 23 mm Schneider lens,”).
Zhang fails to teach wherein the second sensor is different from the first sensor and the type of the second calibration tool being different from that of the first calibration tool. However, Kyung teaches a second sensor that is different from the first sensor (Figs. 1-2; Paragraph 0009 “…it is an object of the present invention to provide a calibration system including a calibration jig and a calibration jig capable of simultaneously performing calibration of a radar and a camera.”), and Kyung teaches the type of the second calibration tool being different from that of the first calibration tool (Figs 1-2; Paragraph 0018 “The calibration jig 100 according to an embodiment of the present invention includes a radar reflector portion 110 and a camera pattern board portion 120.”).
Zhang and Kyung are analogous to the claimed invention, because both teach a method of calibrating a system of two sensors. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang’s invention by using a second sensor-tool pair that is different from the first sensor-tool pair For example, the first sensor-tool pair can be the combination of a camera and a calibration board, and the second sensor-tool pair can be the combination of a radar and a reflector. This modification allows for Zhang’s well known calibration method to be applicable for calibrating systems, such as surveillance systems, that use different sensor types simultaneously (Kyung Paragraphs 0006-0009 “In recent years, radar and camera have been used in peripheral surveillance systems to complement the above shortcomings of radar and camera. In such a peripheral surveillance system, it is necessary to acquire the coordinate relationship between the radar and the camera, and calibrate the radar and the camera to increase the measurement accuracy. Conventionally, since the radar calibration and the camera calibration are performed separately, calibration of the radar and the camera must be performed for a long time.”), and calibrating sensors of different types would require the calibration tool to be of different types (Paragraph 0014 “…a radar reflector for calibrating a radar is formed in a central area, and a calibrating jig having a camera pattern board for calibrating the camera is provided in other areas, Since the calibration can be performed in parallel, it is possible to shorten the time for calibration of the radar and the camera, and since the radar and the camera generate the distance and angle information and the image information based on the center point of the calibration jig, respectively, The relationship can be accurately obtained.”).
Additionally, Zhang fails to teach a first calibration tool comprising a detection apparatus and wherein the detection apparatus specifies a relative position of the second calibration tool with respect to the first calibration tool. However, Tamir teaches a first calibration tool comprising a detection apparatus and wherein the detection apparatus specifies a relative position of the second calibration tool with respect to the first calibration tool (Paragraph 0022 “Intrinsic information for each camera (e.g., field of view, disparity, focal length, lens information, measurement of the focal length in both x and y directions, etc.) may be known and extrinsic information such as the distance 114 between drones 104 and 106, an angle between the cameras 110 and 112, which may be measured using the image taken by camera 108, which may be determined using a sensor, such as an accelerometer, gyroscope, magnetoscope, GPS device, etc.” All three drones in the system can have calibration targets mounted and use sensors and/or cameras to detect one another.).
Zhang and Tamir are analogous to the claimed invention because both teach calibrating optical sensors by observing calibration targets. Therefore, it would have been obvious to one of ordinary skill in the art to apply Zhang’s calibration method to systems where the calibration tools are independently mobile, such as a systems utilizing drones for calibration. This modification allows for the calibration of drone systems for compensating for different instabilities, such as altitude, between the drones (Tamir Paragraph 0023 “In an example, a relative distance between the lead drone 102 and one or both of the drones 104 or 106 may be determined for use in compensating for flight instabilities (e.g., altitude differences, camera angle differences, or the like).”).
Regarding claim 3, Zhang, Kyung, and Tamir teach the calibration system according to Claim 1. Zhang further teaches the system comprises a coupling member configured to couple the first calibration tool to the second calibration tool (Figure 1; Column 2, Line 24 "a rigid bar fasten with two targets respectively corresponding to the vision sensors.").
Regarding claim 5, Zhang, Kyung and Tamir teach the calibration system according to Claim 1. Tamir further teaches wherein the first calibration tool is provided with a sensor configured to be able to specify a relative position of the second calibration tool with respect to the first calibration tool by detecting the second calibration tool, wherein the sensor is the same type of sensor as the second sensor (Paragraph 0022 “Intrinsic information for each camera (e.g., field of view, disparity, focal length, lens information, measurement of the focal length in both x and y directions, etc.) may be known and extrinsic information such as the distance 114 between drones 104 and 106, an angle between the cameras 110 and 112, which may be measured using the image taken by camera 108, which may be determined using a sensor, such as an accelerometer, gyroscope, magnetoscope, GPS device, etc.” All three drones in the system can have calibration targets mounted and use sensors and/or cameras to detect one another.).
Therefore, it would have been obvious to one of ordinary skill in the art to modify the teachings of Zhang and Kyung with the teaching of Tamir by mounting the calibration tools onto drones. This modification allows for the distance between the drones to be known, which is used for compensating for different instabilities, such as altitude, between the drones (Tamir Paragraph 0023 “In an example, a relative distance between the lead drone 102 and one or both of the drones 104 or 106 may be determined for use in compensating for flight instabilities (e.g., altitude differences, camera angle differences, or the like).”).
Regarding claim 6, Zhang, Kyung, and Tamir teach the calibration system according to Claim 5. Tamir further teaches a first robot on which the first calibration tool is mounted; and a second robot on which the second calibration tool is mounted (Fig. 1A-B; Paragraph 0021 “…the lead drone 102, or one or both of the drones 104 or 106, may include a calibration target. The calibration target may be used to individually calibrate each set of images (e.g., frames from two or more of the cameras 108, 110, and 112).).
Therefore, it would have been obvious to one of ordinary skill in the art to modify the teachings of Zhang with the teachings of Tamir by mounting the calibration tools onto drones. This modification allows for the calibration methods taught by Zhang and Kyung to be applied in moving systems which require dynamic calibration, such as systems performing depth-mapping (Tamir Paragraph 0013 “A drone of the system may carry an active or a passive optical marker, such as a calibration chart or an LED, to facilitate dynamic calibration… Calculations to generate depth information may be done in real-time, such as to dynamically correct the stereo depth calibration constant and may be used to calculate depth accurately.”).
Regarding claim 7, Zhang, Kyung, and Tamir teach the calibration system according to Claim 6. Tamir further teaches wherein at least one of the first and second robots is configured to be able to move (Paragraph 0013 “In an example, the drones of the system may fly in a triangle configuration (in the three drone case, for example). A drone of the system may carry an active or a passive optical marker, such as a calibration chart or an LED, to facilitate dynamic calibration.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang and Kyung with the teaching of Tamir by utilizing moving robots, such as drones, for mounting the calibration tools. This modification allows for the calibration methods taught by Zhang and Kyung to be applied in moving systems which require dynamic calibration, such as systems performing depth-mapping (Tamir Paragraph 0013 “A drone of the system may carry an active or a passive optical marker, such as a calibration chart or an LED, to facilitate dynamic calibration… Calculations to generate depth information may be done in real-time, such as to dynamically correct the stereo depth calibration constant and may be used to calculate depth accurately.”).
Regarding claim 8, Zhang, Kyung, and Tamir teach the calibration system of claim 1. Tamir further teaches a third calibration tool, wherein the first calibration tool is provided with a first sensor configured to be able to specify a relative position of the third calibration tool with respect to the first calibration tool by detecting the third calibration tool, and the third calibration tool is provided with a second sensor configured to be able to specify a relative position of the second calibration tool with respect to the third calibration tool by detecting the second calibration tool (Paragraph 0022 “Intrinsic information for each camera (e.g., field of view, disparity, focal length, lens information, measurement of the focal length in both x and y directions, etc.) may be known and extrinsic information such as the distance 114 between drones 104 and 106, an angle between the cameras 110 and 112, which may be measured using the image taken by camera 108, which may be determined using a sensor, such as an accelerometer, gyroscope, magnetoscope, GPS device, etc.” All three drones in the system can have calibration targets mounted and use sensors and/or cameras to detect one another.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang and Kyung by utilizing a third calibration tool with a sensor configured to be able to specify a relative position to the second calibration tool. This modification allows for the calibration methods taught by Zhang and Kyung to be applied in dynamic and expandable systems which require calibration, such as systems performing depth-mapping (Tamir Paragraph 0013 “A drone of the system may carry an active or a passive optical marker, such as a calibration chart or an LED, to facilitate dynamic calibration… Calculations to generate depth information may be done in real-time, such as to dynamically correct the stereo depth calibration constant and may be used to calculate depth accurately.” Paragraph 0014 “Again, more than three drones may be used. Additionally, other configurations than the ones shown in views 100A-100B may be used, such as a square or diamond configuration with four drones, different orientations of the cameras within the system, three or more drones in a line with a lead drone ahead, behind, above, below, etc., or the like.”).
Regarding claim 9, Zhang, Kyung, and Tamir teach the calibration system according to Claim 8. Tamir further teaches a first robot on which the first calibration tool is mounted; a second robot on which the second calibration tool is mounted; and a third robot on which the third calibration tool is mounted (Paragraph 0014 “Again, more than three drones may be used. Additionally, other configurations than the ones shown in views 100A-100B may be used, such as a square or diamond configuration with four drones, different orientations of the cameras within the system, three or more drones in a line with a lead drone ahead, behind, above, below, etc., or the like.” Paragraph 0021 “In this example, the lead drone 102, or one or both of the drones 104 or 106, may include a calibration target. The calibration target may be used to individually calibrate each set of images (e.g., frames from two or more of the cameras 108, 110, and 112).”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang and Kyung by utilizing a third calibration tool with a sensor configured to be able to specify a relative position to the second calibration tool. This modification allows for the calibration methods taught by Zhang and Kyung to be applied in dynamic and expandable systems which require calibration, such as systems performing depth-mapping (Tamir Paragraph 0013 “A drone of the system may carry an active or a passive optical marker, such as a calibration chart or an LED, to facilitate dynamic calibration… Calculations to generate depth information may be done in real-time, such as to dynamically correct the stereo depth calibration constant and may be used to calculate depth accurately.” Paragraph 0014 “Again, more than three drones may be used. Additionally, other configurations than the ones shown in views 100A-100B may be used, such as a square or diamond configuration with four drones, different orientations of the cameras within the system, three or more drones in a line with a lead drone ahead, behind, above, below, etc., or the like.”).
Regarding claim 10, Zhang, Kyung, and Tamir teach the calibration system according to Claim 9. Tamir further teaches wherein at least one of the first to third robots is configured to be able to move (Paragraph 0013 “In an example, the drones of the system may fly in a triangle configuration (in the three drone case, for example). A drone of the system may carry an active or a passive optical marker, such as a calibration chart or an LED, to facilitate dynamic calibration.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang and Kyung with the teaching of Tamir by utilizing moving robots, such as drones, for mounting the calibration tools. This modification allows for the calibration methods taught by Zhang and Kyung to be applied in moving systems which require dynamic calibration, such as systems performing depth-mapping (Tamir Paragraph 0013 “A drone of the system may carry an active or a passive optical marker, such as a calibration chart or an LED, to facilitate dynamic calibration… Calculations to generate depth information may be done in real-time, such as to dynamically correct the stereo depth calibration constant and may be used to calculate depth accurately.”).
Regarding claim 11, Zhang teaches a method for controlling a calibration system (Fig. 2), the system comprising:
a first calibration tool; a second calibration tool (Figure 3; Column 2, Line 24 "a rigid bar fasten with two targets respectively corresponding to the vision sensors." Targets are calibration tools.)
a first sensor configured to be able to detect the first calibration tool, wherein the first sensor comprises a camera, a shape acquisition sensor, a position detection sensor, or a light-receiving element; a second sensor configured to be able to detect the second calibration tool, wherein the second sensor comprises a camera, a shape acquisition sensor, a position detection sensor, or a light-receiving element (Fig. 3 "vision sensor 1" and "vision sensor 2"; Column 2, Line 26 "corresponding to the vision sensors; capturing images of the respective targets by their corresponding vision sensors; extracting coordinates of the feature points of the respective targets on their corresponding images; and computing 3D coordinates of each feature points of the respective targets under their corresponding vision sensor coordinate frames;"); and
a processor, the method comprising:
detecting the first calibration tool by the first sensor; detecting the second calibration tool by the second sensor (Fig. 3 "vision sensor 1" and "vision sensor 2"; Column 2, Line 26 "corresponding to the vision sensors; capturing images of the respective targets by their corresponding vision sensors; extracting coordinates of the feature points of the respective targets on their corresponding images; and computing 3D coordinates of each feature points of the respective targets under their corresponding vision sensor coordinate frames;"); and
calculating, using the processor, the relative position of the second sensor with respect to the first sensor based on the result of detecting the first calibration tool by the first sensor and the result of detecting the second calibration tool by the second sensor (Fig. 2 "the rotation matrix" and "the translation vector" defining the relative position between the two sensors; Fig. 3; Column 2, line 50 “In the Solutions above, computing the transformation matrix between the two vision sensors may include: computing the rotation matrix and translation vectors; establishing an objective function on the basis of minimizing the re-projection error, Solving an non-linear optimal Solution for the rotation matrix and translation vectors to acquire the transformation matrix between the two vision sensors, by using a non-linear optimization approach.”);
Zhang fails to teach the type of the second calibration tool being different from that of the first calibration tool (Fig. 5 shows that both calibration tools are calibration boards). However, Kyung teaches the type of the second calibration tool being different from that of the first calibration tool (Figs 1-2; Paragraph 0018 “The calibration jig 100 according to an embodiment of the present invention includes a radar reflector portion 110 and a camera pattern board portion 120.”). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang’s invention to use calibration tools that are different from one another. This modification is necessary when calibrating sensors of different types (Kyung Paragraph 0014 “…a radar reflector for calibrating a radar is formed in a central area, and a calibrating jig having a camera pattern board for calibrating the camera is provided in other areas, Since the calibration can be performed in parallel, it is possible to shorten the time for calibration of the radar and the camera, and since the radar and the camera generate the distance and angle information and the image information based on the center point of the calibration jig, respectively, The relationship can be accurately obtained.”).
Additionally, Zhang fails to teach a first calibration tool comprising a detection apparatus and wherein the detection apparatus specifies a relative position of the second calibration tool with respect to the first calibration tool. However, Tamir teaches a first calibration tool comprising a detection apparatus and wherein the detection apparatus specifies a relative position of the second calibration tool with respect to the first calibration tool (Paragraph 0022 “Intrinsic information for each camera (e.g., field of view, disparity, focal length, lens information, measurement of the focal length in both x and y directions, etc.) may be known and extrinsic information such as the distance 114 between drones 104 and 106, an angle between the cameras 110 and 112, which may be measured using the image taken by camera 108, which may be determined using a sensor, such as an accelerometer, gyroscope, magnetoscope, GPS device, etc.” All three drones in the system can have calibration targets mounted and use sensors and/or cameras to detect one another.).
Therefore, it would have been obvious to one of ordinary skill in the art to apply Zhang’s calibration method to systems where the calibration tools are independently mobile, such as a systems utilizing drones for calibration. This modification allows for the calibration of drone systems for compensating for different instabilities, such as altitude, between the drones (Tamir Paragraph 0023 “In an example, a relative distance between the lead drone 102 and one or both of the drones 104 or 106 may be determined for use in compensating for flight instabilities (e.g., altitude differences, camera angle differences, or the like).”).
Regarding claim 12, Zhang teaches a non-transitory computer readable medium storing a control program (Fig. 2) of a calibration system,
the calibration system comprising:
a first calibration tool; a second calibration tool (Figure 3; Column 2, Line 24 “a rigid bar fasten with two targets respectively corresponding to the vision sensors.” Targets are calibration tools.)
a first sensor configured to be able to detect the first calibration tool, wherein the first sensor comprises a camera, a shape acquisition sensor, a position detection sensor, or a light-receiving element; a second sensor configured to be able to detect the second calibration tool, wherein the second sensor comprises a camera, a shape acquisition sensor, a position detection sensor, or a light-receiving element (Column 2, Line 26 "corresponding to the vision sensors; capturing images of the respective targets by their corresponding vision sensors; extracting coordinates of the feature points of the respective targets on their corresponding images; and computing 3D coordinates of each feature points of the respective targets under their corresponding vision sensor coordinate frames;"); and
a processor, the control program causing a computer to execute the processing of:
detecting the first calibration tool by the first sensor; detecting the second calibration tool by the second sensor (Column 2, Line 26 “corresponding to the vision sensors; capturing images of the respective targets by their corresponding vision sensors; extracting coordinates of the feature points of the respective targets on their corresponding images; and computing 3D coordinates of each feature points of the respective targets under their corresponding vision sensor coordinate frames;”); and
calculating, using the processor, the relative position of the second sensor with respect to the first sensor based on the result of detecting the first calibration tool by the first sensor and the result of detecting the second calibration tool by the second sensor (Column 3, Line 61 “and computing the 3D coordinates of the feature points on the targets under the coordinate frame of their corresponding vision sensor, respectively. And then to compute the transformation matrix between the two vision sensors.”).
Zhang fails to teach the type of the second calibration tool being different from that of the first calibration tool (Fig. 5 shows that both calibration tools are calibration boards). However, Kyung teaches the type of the second calibration tool being different from that of the first calibration tool (Figs 1-2; Paragraph 0018 “The calibration jig 100 according to an embodiment of the present invention includes a radar reflector portion 110 and a camera pattern board portion 120.”). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang’s invention to use calibration tools that are different from one another. This modification is necessary when calibrating sensors of different types (Kyung Paragraph 0014 “…a radar reflector for calibrating a radar is formed in a central area, and a calibrating jig having a camera pattern board for calibrating the camera is provided in other areas, Since the calibration can be performed in parallel, it is possible to shorten the time for calibration of the radar and the camera, and since the radar and the camera generate the distance and angle information and the image information based on the center point of the calibration jig, respectively, The relationship can be accurately obtained.”).
Additionally, Zhang fails to teach a first calibration tool comprising a detection apparatus and wherein the detection apparatus specifies a relative position of the second calibration tool with respect to the first calibration tool. However, Tamir teaches a first calibration tool comprising a detection apparatus and wherein the detection apparatus specifies a relative position of the second calibration tool with respect to the first calibration tool (Paragraph 0022 “Intrinsic information for each camera (e.g., field of view, disparity, focal length, lens information, measurement of the focal length in both x and y directions, etc.) may be known and extrinsic information such as the distance 114 between drones 104 and 106, an angle between the cameras 110 and 112, which may be measured using the image taken by camera 108, which may be determined using a sensor, such as an accelerometer, gyroscope, magnetoscope, GPS device, etc.” All three drones in the system can have calibration targets mounted and use sensors and/or cameras to detect one another.).
Therefore, it would have been obvious to one of ordinary skill in the art to apply Zhang’s calibration method to systems where the calibration tools are independently mobile, such as a systems utilizing drones for calibration. This modification allows for the calibration of drone systems for compensating for different instabilities, such as altitude, between the drones (Tamir Paragraph 0023 “In an example, a relative distance between the lead drone 102 and one or both of the drones 104 or 106 may be determined for use in compensating for flight instabilities (e.g., altitude differences, camera angle differences, or the like).”).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Kyung and Tamir as applied to claim 1 above, and further in view of Ribeiro et al. ("Photogrammetric Multi-Camera Calibration Using An Industrial Programmable Robotic Arm," 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), Dubrovnik, Croatia, 2019, pp. 288-294, doi: 10.1109/ISPA.2019.8868928.), hereafter Ribeiro.
Regarding claim 4, Zhang, Kyung, and Tamir teach the calibration system according to Claim 1, but fails to further teach the system comprises a robot configured to couple the first calibration tool to the second calibration tool by a robot arm.
However, Ribeiro teaches a robot configured to couple the first calibration tool to the second calibration tool by a robot arm (Fig. 5; Page 4, Paragraph 2 "In order to obtain a fully repeatable calibration pipeline, we implement an automatic capturing procedure using a robotic arm... The pattern is rigidly attached to the robot end-effector." A robot arm moves a mounted calibration board into the view of one camera, then to another camera.).
Zhang and Ribeiro are both considered to be analogous to the claimed invention because they are both in the same fields of camera calibration systems. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang to incorporate the teaching of Ribeiro to include a robotic arm for mounting calibration tools, because such a modification is the result of a substitution of one known element for another. More specifically, both Zhang's rigid rod and Ribeiro's robotic arm can perform the same function of providing a mount for calibration tools.
Additionally, modifying the invention to use a robotic arm would allow for repeatable results (Ribeiro Page 1, Column 2, Paragraph 3 "With this method, we create an extensive dataset of calibration images with known relative target positions.") and for the elimination of manually moving multiple calibration tools for each capture step (Ribeiro Abstract "In order to obtain fully repeatable results, we propose the elimination of the manual capture step using a programmable robotic arm.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gao et al. (A Method of Spatial Calibration for Camera and Radar. Proceedings of the 8th World Congress on Intelligent Control and Automation July 6-9 2010. Jinan, China.) teaches a method for multi-sensor fusion between a camera and a radar. This method involves calibration between the camera and radar based on their relative positions to one another, and this distance allows for transformations between image coordinates of each sensor and the world coordinates.
Camilo et al. (DE 102018132808 A1) teaches a method for improved object detection for autonomous vehicles using a combination of a camera and a radar. Objects are detected by both sensors in their own coordinate frames, and the object location in the world coordinate system is determined using the relative position between the camera and radar.
Barreto et al. (US 9,438,897 B2) teaches a method for calibrating a camera based on imaging a checkerboard pattern from multiple angles. This method includes determining and correcting the radial distortion associated with the camera.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action.
In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC JAMES SHOEMAKER whose telephone number is (571)272-6605. The examiner can normally be reached Monday through Friday from 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw, can be reached at (571)272-4637. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Eric Shoemaker/
Patent Examiner
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676
Prosecution Insights