DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s amendments overcome the objection to the drawings. The objection has been withdrawn.
Applicant’s amendments overcome the 112b rejections of claim 1-14. The 112b rejections of claims 1-14 have been withdrawn.
Applicant’s arguments with respect to the 103 rejections claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Although the examiner agrees that Li (US 20220292719 A1, previously cited) does not explicitly teach the newly amended limitations of wherein obtaining the second image includes obtaining a sequence of images via the second imaging sensor while thermally exciting at least a portion of the calibration artifact, and determining whether a calibration artifact pattern is present in one of the sequence of images indicative of uniform heating of the calibration artifact and assigning the determined one of the sequence of images as the second image for determining the second pixel mapping, in response to detecting the calibration artifact pattern in the determined one of the sequence of images, Li does teach that the calibration frame must have a uniform heating region because if heating is uneven or only a specific region heats, a calibration frame image in an image acquired by the thermal imaging camera is completely different from a calibration frame image in an image acquired by the visible light camera. In this case, since variables cannot be controlled, the calibration frame cannot achieve a reference effect as a calibration frame ([0051]). Additionally, CN 112750169 A by Tan (newly cited; translation provided) is further relied upon to teach the new limitations in the rejection below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “imaging calibration module” in claims 1-3 and 6-9.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Regarding claim 1, the claim recites “imaging calibration module” which uses the generic placeholder “module” that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Accordingly, the limitation on “imaging calibration module” is interpreted under 35 U.S.C. 112(f) as corresponding to a circuit like a processor or controller according to applicant’s specification [0086].
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over US20220292719A1 by Li (previously cited) in view of US20230133662A1 by Moser et al. (hereinafter “Moser”; previously cited) and CN 112750169 A by Tan (newly cited; translation provided).
Regarding claim 1, Li teaches A calibration system for multiple image sensors ([0007]; Fig, 1), the calibration system comprising:
a first imaging sensor (visible light camera 300) configured to capture a first object image an object (target object [0049]);
a second imaging sensor (thermal imaging camera 200) configured to capture a second object image the object, wherein the second imaging sensor is positioned at a different location than the first imaging sensor (Fig. 1 shows cameras at two different locations; [0004] purpose of invention is to calibrate two cameras that have a relative physical displacement between different orientations or positions; [0098]) and is configured to capture at least one different wavelength than the first imaging sensor ([0049] captures thermal radiation image instead of visible light wavelength range image);
a calibration artifact located within a plane of the manufacturing apparatus ([0051] object selected in the calibration frame is the calibration artifact); and an imaging calibration module (processor 400; [0048]) configured to:
obtain a first image of the calibration artifact via the first imaging sensor ([0054] visible light camera 300 acquires a visible light image including a second calibration frame image; [0057]);
obtain a second image of the calibration artifact via the second imaging sensor ([0054] thermal imaging camera 200 acquires a thermal imaging image including a first calibration frame image, first calibration frame image and the second calibration frame image are images of a same calibration frame; [0056]);
determine a first pixel mapping between the first image and a common coordinate system, according to a location of the calibration artifact in the first image (Fig. 3; [0057] processor 400 acquires second position information of the second calibration frame image in the visible light image. The second position information includes a width, a height and coordinates of the second calibration frame image in the visible light image; [0060]) ;
determine a second pixel mapping between the second image and the common coordinate system according to a location of the calibration artifact in the second image (Fig. 3; [0056] processor 400 acquires first position information of the first calibration frame image in the thermal imaging image. The first position information includes a width, a height and coordinates of the first calibration frame image in the thermal imaging image; [0059]);
convert the first object image of the object captured by the first imaging sensor to the common coordinate system according to the first pixel mapping (Fig. 3; [0060]); and
convert the second object image of the object captured by the second imaging sensor to the common coordinate system according to the second pixel mapping (Fig. 3; [0059]).
Li is silent as to a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object, the first and second imaging sensors configured to capture an image of the manufacturing object and the imaging calibration module configured to convert an image of the manufacturing object captured by the first and second imaging sensors.
However, Moser does address this limitation. Moser and Li are considered to be analogous to the present invention as they are in the same field of sensor calibration.
Moser teaches a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object ([0163] "After the calibration, a method for process monitoring of a laser machining process may be carried out by the first optical sensor 200. The process monitoring method may be carried out before the laser machining process (pre-process), during the laser machining process (in-process) or after the laser machining process (post-process), for example to obtain a workpiece geometry, to define a machining position for the laser machining process, or to measure the machining result, such as a weld seam"; where the weld seam is on the manufacturing object), the imaging sensor configured to capture an image of the manufacturing object and the imaging calibration module configured to convert an image of the manufacturing object captured by the imaging sensor ([0163] first sensor 200, e.g. the camera, is used, for example, to determine a machining position on the workpiece surface before the actual laser machining process, in particular when the machining laser beam is switched off, a lateral offset between the position determined in the image and the actual position, e.g. the target position the subsequent laser machining process, can be reduced or even avoided).
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to use a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object and capture images of the manufacturing object and convert them to the common coordinate system based on the calibration. Therefore, it would have been obvious to modify Li to include a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object, the first and second imaging sensors configured to capture an image of the manufacturing object and the imaging calibration module configured to convert an image of the manufacturing object captured by the first and second imaging sensors as suggested by Moser in order to increase the accuracy of the manufacturing operation.
Further, Li teaches that the calibration frame must have a uniform heating region because if heating is uneven or only a specific region heats, a calibration frame image in an image acquired by the thermal imaging camera is completely different from a calibration frame image in an image acquired by the visible light camera. In this case, since variables cannot be controlled, the calibration frame cannot achieve a reference effect as a calibration frame ([0051]). Therefore, the calibration frame needs to have a definite shape and a uniform heating region overlapping a visible light region.
However, Li does not explicitly teach wherein obtaining the second image includes obtaining a sequence of images via the second imaging sensor while thermally exciting at least a portion of the calibration artifact, and determining whether a calibration artifact pattern is present in one of the sequence of images indicative of uniform heating of the calibration artifact and assigning the determined one of the sequence of images as the second image for determining the second pixel mapping, in response to detecting the calibration artifact pattern in the determined one of the sequence of images.
However, Tan does address this limitation. Tan and Li are considered to be analogous to the present invention as they are in the same field of camera calibration.
Tan teaches obtaining the second image includes obtaining a sequence of images via the second imaging sensor ([015] Controlling the infrared camera to collect images of the preset calibration board from multiple collection positions to obtain a corresponding temperature calibration map) while thermally exciting at least a portion of the calibration artifact ([014] Heating the preset calibration plate), and determining whether a calibration artifact pattern is present in one of the sequence of images indicative of uniform heating of the calibration artifact ([015] determine whether the infrared camera meets the preset calibration conditions according to the temperature calibration map; [082] If the infrared camera reaches the preset calibration conditions, the temperature calibration map collected by the infrared camera contains all the calibration patterns on the preset calibration board, and the temperature calibration map can be determined to be the first temperature map to determine the infrared camera's temperature based on the first temperature map; thus it appears that the camera may continue to collect data, or a sequence of images, until the preset calibration conditions are reached, thus selecting the first temperature calibration map after the calibration condition is met would be equivalent to selecting the image with a uniform heating of the calibration artifact) and
assigning the determined one of the sequence of images as the second image for determining the second pixel mapping ([017]-[019] Obtain the pixel coordinates of the calibration pattern), in response to detecting the calibration artifact pattern in the determined one of the sequence of images ([016] When the infrared camera reaches a preset calibration condition, it is determined that the temperature calibration map is the first temperature map; [086]).
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to take a sequence of images and select the appropriate image for further processing. Therefore, it would have been obvious to modify Li to include wherein obtaining the second image includes obtaining a sequence of images via the second imaging sensor while thermally exciting at least a portion of the calibration artifact, and determining whether a calibration artifact pattern is present in one of the sequence of images indicative of uniform heating of the calibration artifact and assigning the determined one of the sequence of images as the second image for determining the second pixel mapping, in response to detecting the calibration artifact pattern in the determined one of the sequence of images as suggested by Tan in order to improve calibration accuracy.
Regarding claim 2, Li modified by Moser and Tan teach the system of claim 1, and Li further teaches wherein the imaging calibration module is configured to:
perform image processing on the first object image of the manufacturing object (see claim 1 for explanation) captured by the first imaging sensor to identify a target region ([0062]-[0063]); and
apply the target region to the second object image of the manufacturing object (see claim 1 for explanation) captured by the second imaging sensor using the common coordinate system ([0062] an upper left vertex of the thermal imaging image synthesized in the visible light image; [0064] processor 400 may adjust, according to the calibration parameters, a position of the thermal imaging image synthesized in the visible light image).
Regarding claim 3, Li modified by Moser and Tan teach the system of claim 2, but Li is silent as to wherein the imaging calibration module is configured to control the manufacturing apparatus to perform the manufacturing operation on the manufacturing object according to the target region as applied to the second object image of the manufacturing object captured by the second imaging sensor.
However, Moser does address this limitation.
Moser teaches wherein the imaging calibration module is configured to control the manufacturing apparatus to perform the manufacturing operation on the manufacturing object according to the target region as applied to the image of the manufacturing object captured by the imaging sensor ([0163 "a lateral offset between the position determined in the image and the actual position, e.g. the target position the subsequent laser machining process, can be reduced or even avoided".)
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to use the results from a calibration in order to perform a manufacturing operation on the manufacturing object. Therefore, it would have been obvious to modify Li to include wherein the imaging calibration module is configured to control the manufacturing apparatus to perform the manufacturing operation on the manufacturing object according to the target region as applied to the second object image of the manufacturing object captured by the second imaging sensor as suggested by Moser in order to efficiently and accurately perform the manufacturing operation in the target region.
Regarding claim 4, Li modified by Moser and Tan teach the system of claim 3, but Li is silent as to wherein the manufacturing apparatus is a weld inspection machine; and the manufacturing operation includes a weld inspection operation performed on the manufacturing object.
However, Moser does address this limitation.
Moser teaches wherein the manufacturing apparatus is a weld inspection machine ([0163] since the apparatus can measure the weld seam, it can be considered a weld inspection apparatus); and
the manufacturing operation includes a weld inspection operation performed on the manufacturing object ([0163] measure the machining result, such as a weld seam; where the weld seam is on the manufacturing object).
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention that a weld inspection machine which includes a weld inspection operation performed on the manufacturing object would need to have calibrated sensors. Therefore, it would have been obvious to modify Li to include wherein: the manufacturing apparatus is a weld inspection machine; and the manufacturing operation includes a weld inspection operation performed on the manufacturing object as suggested by Moser in order to apply the calibration to a field of use, thus increasing the efficiently and accuracy of the weld inspection.
Regarding claim 5, Li modified by Moser and Tan teach the system of claim 1, and Li further teaches wherein:
the first imaging sensor is a visible light camera ([0049] visible light camera 300); and
the second imaging sensor is an infrared camera ([0049] thermal imaging camera 200).
Regarding claim 6, Li modified by Moser and Tan teach the system of claim 5, and Li further teaches wherein the imaging calibration module is configured to ([0083] executed by one or more processors):
illuminate at least one of a front surface of the calibration artifact or a back surface of the calibration artifact while capturing the first image of the calibration artifact using the visible light camera ([0051] visible light region); and
thermally excite at least a portion of the calibration artifact while capturing the second image of the calibration artifact using the infrared camera ([0051] object selected as the calibration frame needs to have a definite shape and have a uniform heating region overlapping a visible light region; calibration frame is a rectangular heating frame; [0053] uniform heating source; thus it is evident that at least of portion of the calibration artifact is thermally excited).
Although Li is silent as to wherein the imaging calibration module is configured to activate a light source, it would have been well known and obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to include this step in order to efficiently and reliably capture images using the visible light camera.
Further, even if Li does not explicitly teach wherein the imaging calibration module is configured to perform these specific steps, Li does teach that a person of ordinary skill in the art may understand that all or some of the processes in the methods of the foregoing embodiments may be implemented by a computer program instructing relevant hardware ([0083]). Also, the manner of operating the device does not differentiate the device from the prior art, see MPEP 2114 Sec. II “[A]pparatus claims cover what a device is, not what a device does.” Hewlett-Packard Co.v.Bausch & Lomb Inc., 909 F.2d 1464, 1469, 15 USPQ2d 1525, 1528 (Fed. Cir. 1990”).
Further, Moser does address this limitation.
Moser teaches activate a light source to illuminate at least one of a front surface of the calibration artifact or a back surface of the calibration artifact while capturing the first image of the calibration artifact using the visible light camera ([0076] The illumination may be aimed at the reference (calibration plate). The illumination may produce white light or light of a specific wavelength (color). The optical measurement signal may correspond to a (two-dimensional) camera image or a photograph; computing device 700; [0088])
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to activate a light source provide illumination. Therefore, it would have been obvious to modify Li to include activate a light source to illuminate at least one of a front surface of the calibration artifact or a back surface of the calibration artifact while capturing the first image of the calibration artifact using the visible light camera as suggested by Moser in order to efficiently and reliably capture images using the visible light camera.
Regarding claim 13, Li modified by Moser and Tan teach the system of claim 1, and Li further teaches wherein the first imaging sensor is at a different distance from the calibration artifact than the second imaging sensor (Fig. 1 shows cameras at two different locations; [0004] purpose of invention is to calibrate two cameras that have a relative physical displacement between different orientations or positions; [0082] relative physical position change; [0098] physical displacement).
Regarding claim 14, Li modified by Moser and Tan teach the system of claim 1, and Li further teaches wherein the first imaging sensor is oriented at a different angle with respect to the calibration artifact than the second imaging sensor ([0082] different angles of the plurality of sensors during use; [0004]-[0005]).
Regarding claim 15, Li teaches A method of calibrating multiple image sensors ([0007]; Fig, 1), the method comprising:
obtaining a first image of a calibration artifact via a first imaging sensor ([0054] visible light camera 300 acquires a visible light image including a second calibration frame image; [0057]);
obtaining a second image of the calibration artifact via a second imaging sensor, wherein the second imaging sensor is positioned at a different location than the first imaging sensor and is configured to capture at least one different wavelength than the first imaging sensor ([0054] thermal imaging camera 200 acquires a thermal imaging image including a first calibration frame image, first calibration frame image and the second calibration frame image are images of a same calibration frame; [0056]; Fig. 1 shows cameras at two different locations; [0004] purpose of invention is to calibrate two cameras that have a relative physical displacement between different orientations or positions; [0098]) and is configured to capture at least one different wavelength than the first imaging sensor ([0049] captures thermal radiation image instead of visible light wavelength range image);
determining a first pixel mapping between the first image and a common coordinate system, according to a location of the calibration artifact in the first image (Fig. 3; [0057] processor 400 acquires second position information of the second calibration frame image in the visible light image. The second position information includes a width, a height and coordinates of the second calibration frame image in the visible light image; [0060]);
determining a second pixel mapping between the second image and the common coordinate system according to a location of the calibration artifact in the second image (Fig. 3; [0056] processor 400 acquires first position information of the first calibration frame image in the thermal imaging image. The first position information includes a width, a height and coordinates of the first calibration frame image in the thermal imaging image; [0059]);
converting an image captured by the first imaging sensor to the common coordinate system according to the first pixel mapping (Fig. 3; [0060]); and
converting an image captured by the second imaging sensor to the common coordinate system according to the second pixel mapping (Fig. 3; [0059]).
Li is silent as to wherein the calibration artifact located within a plane of a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object and converting an image of the manufacturing object captured by the first imaging sensor and the second imaging sensor.
However, Moser does address this limitation. Moser and Li are considered to be analogous to the present invention as they are in the same field of sensor calibration.
Moser teaches wherein the calibration artifact located within a plane of a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object ([0163] "After the calibration, a method for process monitoring of a laser machining process may be carried out by the first optical sensor 200. The process monitoring method may be carried out before the laser machining process (pre-process), during the laser machining process (in-process) or after the laser machining process (post-process), for example to obtain a workpiece geometry, to define a machining position for the laser machining process, or to measure the machining result, such as a weld seam"; where the weld seam is on the manufacturing object.) and converting an image of the manufacturing object captured by the imaging sensor ([0163] first sensor 200, e.g. the camera, is used, for example, to determine a machining position on the workpiece surface before the actual laser machining process, in particular when the machining laser beam is switched off, a lateral offset between the position determined in the image and the actual position, e.g. the target position the subsequent laser machining process, can be reduced or even avoided).
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to use a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object and capture images of the manufacturing object and convert them to the common coordinate system based on the calibration. Therefore, it would have been obvious to modify Li to include wherein the calibration artifact located within a plane of a manufacturing apparatus configured to perform a manufacturing operation on a manufacturing object and converting an image of the manufacturing object captured by the first imaging sensor and the second imaging sensor as suggested by Moser in order to increase the accuracy of the manufacturing operation.
Further, Li teaches that the calibration frame must have a uniform heating region because if heating is uneven or only a specific region heats, a calibration frame image in an image acquired by the thermal imaging camera is completely different from a calibration frame image in an image acquired by the visible light camera. In this case, since variables cannot be controlled, the calibration frame cannot achieve a reference effect as a calibration frame ([0051]). Therefore, the calibration frame needs to have a definite shape and a heating region overlapping a visible light region
However, Li does not explicitly teach wherein obtaining the second image includes obtaining a sequence of images via the second imaging sensor while thermally exciting at least a portion of the calibration artifact, and determining whether a calibration artifact pattern is present in one of the sequence of images indicative of uniform heating of the calibration artifact and assigning the determined one of the sequence of images as the second image for determining the second pixel mapping, in response to detecting the calibration artifact pattern in the determined one of the sequence of images.
However, Tan does address this limitation. Tan and Li are considered to be analogous to the present invention as they are in the same field of camera calibration.
Tan teaches obtaining the second image includes obtaining a sequence of images via the second imaging sensor ([015] Controlling the infrared camera to collect images of the preset calibration board from multiple collection positions to obtain a corresponding temperature calibration map) while thermally exciting at least a portion of the calibration artifact ([014] Heating the preset calibration plate), and determining whether a calibration artifact pattern is present in one of the sequence of images indicative of uniform heating of the calibration artifact ([015] determine whether the infrared camera meets the preset calibration conditions according to the temperature calibration map; [082] If the infrared camera reaches the preset calibration conditions, the temperature calibration map collected by the infrared camera contains all the calibration patterns on the preset calibration board, and the temperature calibration map can be determined to be the first temperature map to determine the infrared camera's temperature based on the first temperature map; thus it appears that the camera may continue to collect data, or a sequence of images, until the preset calibration conditions are reached, thus selecting the first temperature calibration map after the calibration condition is met would be equivalent to selecting the image with a uniform heating of the calibration artifact) and
assigning the determined one of the sequence of images as the second image for determining the second pixel mapping ([017]-[019] Obtain the pixel coordinates of the calibration pattern), in response to detecting the calibration artifact pattern in the determined one of the sequence of images ([016] When the infrared camera reaches a preset calibration condition, it is determined that the temperature calibration map is the first temperature map; [086]).
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to take a sequence of images and select the appropriate image for further processing. Therefore, it would have been obvious to modify Li to include wherein obtaining the second image includes obtaining a sequence of images via the second imaging sensor while thermally exciting at least a portion of the calibration artifact, and determining whether a calibration artifact pattern is present in one of the sequence of images indicative of uniform heating of the calibration artifact and assigning the determined one of the sequence of images as the second image for determining the second pixel mapping, in response to detecting the calibration artifact pattern in the determined one of the sequence of images as suggested by Tan in order to improve calibration accuracy.
Regarding claim 16, Li modified by Moser and Tan teach the method of claim 15, and Li further teaches comprising performing image processing on the image of the manufacturing object (see claim 15 for explanation) captured by the first imaging sensor to identify a target region ([0062]-[0063]); and
applying the target region to the image of the manufacturing object (see claim 15 for explanation) captured by the second imaging sensor using the common coordinate system ([0062] an upper left vertex of the thermal imaging image synthesized in the visible light image; [0064] processor 400 may adjust, according to the calibration parameters, a position of the thermal imaging image synthesized in the visible light image).
Regarding claim 17, Li modified by Moser and Tan teach the method of claim 16, but Li is silent as to wherein further comprising controlling the manufacturing apparatus to perform the manufacturing operation on the manufacturing object according to the target region as applied to the image of the manufacturing object captured by the second imaging sensor.
However, Moser does address this limitation.
Moser teaches controlling the manufacturing apparatus to perform the manufacturing operation on the manufacturing object according to the target region as applied to the image of the manufacturing object captured by the imaging sensor ([0163 "a lateral offset between the position determined in the image and the actual position, e.g. the target position the subsequent laser machining process, can be reduced or even avoided".)
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to use the results from a calibration in order to perform a manufacturing operation on the manufacturing object. Therefore, it would have been obvious to modify Li to include controlling the manufacturing apparatus to perform the manufacturing operation on the manufacturing object according to the target region as applied to the image of the manufacturing object captured by the second imaging sensor as suggested by Moser in order to efficiently and accurately perform the manufacturing operation in the target region.
Regarding claim 18, Li modified by Moser and Tan teach the method of claim 17, but Li is silent as to wherein: the manufacturing apparatus is a weld inspection machine; and
the manufacturing operation includes a weld inspection operation performed on the manufacturing object.
However, Moser does address this limitation.
Moser teaches wherein the manufacturing apparatus is a weld inspection machine ([0163] since the apparatus can measure the weld seam, it can be considered a weld inspection apparatus); and
the manufacturing operation includes a weld inspection operation performed on the manufacturing object ([0163] measure the machining result, such as a weld seam; where the weld seam is on the manufacturing object).
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention that a weld inspection machine which includes a weld inspection operation performed on the manufacturing object would need to have calibrated sensors. Therefore, it would have been obvious to modify Li to include wherein: the manufacturing apparatus is a weld inspection machine; and the manufacturing operation includes a weld inspection operation performed on the manufacturing object as suggested by Moser in order to apply the calibration to a field of use, thus increasing the efficiently and accuracy of the weld inspection.
Regarding claim 19, Li modified by Moser and Tan teach the method of claim 15, and Li further teaches wherein:
the first imaging sensor is a visible light camera ([0049] visible light camera 300); and
the second imaging sensor is an infrared camera ([0049] thermal imaging camera 200).
Regarding claim 20, Li modified by Moser and Tan teach the method of claim 19, and Li further teaches comprising illuminating at least one of a front surface of the calibration artifact or a back surface of the calibration artifact while capturing the first image of the calibration artifact using the visible light camera ([0051] visible light region); and
thermally excite at least a portion of the calibration artifact while capturing the second image of the calibration artifact using the infrared camera ([0051] object selected as the calibration frame needs to have a definite shape and have a uniform heating region overlapping a visible light region; calibration frame is a rectangular heating frame; [0053] uniform heating source; thus it is evident that at least of portion of the calibration artifact is thermally excited).
Although Li is silent as to activating a light source, it would have been well known and obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to include this step in order to efficiently and reliably capture images using the visible light camera.
Further, Moser does address this limitation.
Moser teaches activating a light source to illuminate at least one of a front surface of the calibration artifact or a back surface of the calibration artifact while capturing the first image of the calibration artifact using the visible light camera ([0076] The illumination may be aimed at the reference (calibration plate). The illumination may produce white light or light of a specific wavelength (color). The optical measurement signal may correspond to a (two-dimensional) camera image or a photograph; computing device 700; [0088])
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to activate a light source provide illumination. Therefore, it would have been obvious to modify Li to include activating a light source to illuminate at least one of a front surface of the calibration artifact or a back surface of the calibration artifact while capturing the first image of the calibration artifact using the visible light camera as suggested by Moser in order to efficiently and reliably capture images using the visible light camera.
Claims 8 and 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Moser and Tan as applied to claim 1 and 6 above and in further view of CN107067442A by Deng (previously cited).
Regarding claim 8, Li modified by Moser and Tan teach the system of claim 6, and Li further teaches wherein the front surface of the calibration artifact includes a material configured to absorb and emit heat in the form of infrared radiation ([0051] calibration frame is a rectangular heating frame).
Further, even if Li does not explicitly teach a material configured to absorb and emit heat, Deng does address this limitation. Deng and Li are considered to be analogous to the present invention as they are in the same field of infrared and visible light double-camera calibration.
Deng teaches wherein the front surface of the calibration artifact includes a material configured to absorb and emit heat in the form of infrared radiation ([0034] better thermal conductivity alloy material, in order to achieve better heat transfer effect).
It would have been well known and obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to use a material configured to absorb and emit heat in the form of infrared radiation in order to accurately calibrate a thermal imaging sensor.
Regarding claim 10, Li modified by Moser, Tan and Deng teach the system of claim 8, but Li and Moser are silent as to wherein the material on the front surface of the calibration artifact includes a paint material configured to absorb and emit heat in the form of infrared radiation.
However, Deng does address this limitation.
Deng teaches wherein the material on the front surface of the calibration artifact includes a paint material configured to absorb and emit heat in the form of infrared radiation ([0032] color of the panel 1 is white, the appearance color of the panel 5 is black, the temperature of the panel 1 is higher than the temperature of the panel 5; [0035] the color of the exterior is realized by using different coating materials; paint would be considered a coating material.)
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to use a paint material configured to absorb and emit heat. Therefore, it would have been obvious to modify Li to include wherein the material on the front surface of the calibration artifact includes a paint material configured to absorb and emit heat in the form of infrared radiation as suggested by Deng in order to accurately calibrate a thermal imaging sensor using low cost materials.
Regarding claim 11, Li modified by Moser and Tan teach the system of claim 1, and Li further teaches wherein the calibration artifact includes a rectangular plate having a front side ([0087] calibration frame is a rectangular heating frame), and although Li does not explicitly teach wherein the calibration artifact includes a rectangular plate having a back side, it would be inherent that the calibration artifact has a back side.
Further, Li is silent as to wherein the calibration artifact includes a rectangular plate having at least one chamfered corner, and an array of openings defined from the front side to the back side.
However, Deng does address this limitation. Deng and Li are considered to be analogous to the present invention as they are in the same field of infrared and visible light double-camera calibration.
Deng teaches wherein the calibration artifact includes a rectangular plate having a front side, a back side (Fig. 2; [011] calibration plate includes panel 1 which has a front and back), at least one chamfered corner ([012 square corner cut; [0045]), and an array of openings defined from the front side to the back side ([012] hollow circular array; [0037]).
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to use a specialized calibration plate in order to calibrate an infrared and visible light double-camera. Therefore, it would have been obvious to modify Li to include wherein the calibration artifact includes a rectangular plate having at least one chamfered corner, and an array of openings defined from the front side to the back side as suggested by Deng in order to provide a highly efficient method of calibrating an infrared and visible light double-camera ([0027]-[009]).
Regarding claim 12, Li modified by Moser, Tan and Deng teach the system of claim 11, but Li and Moser are silent as to wherein the rectangular plate defines a thickness from the front side to the back side, and each of the openings and each edge of the rectangular plate is chamfered.
However, Deng does address this limitation.
Deng teaches wherein the rectangular plate defines a thickness from the front side to the back side (Fig. 1 shows panel 1 has thickness from the front side to backside), and although Deng does not explicitly teach each of the openings and each edge of the rectangular plate is chamfered, Deng does teach chamfered corners ([045]) and that the square borders are notched ([012]; [0036] gap on the square border).
Thus, it would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to chamfer corners or edges during manufacturing. Therefore, it would have been obvious to modify Li to include wherein the rectangular plate defines a thickness from the front side to the back side, and each of the openings and each edge of the rectangular plate is chamfered in order to decrease risk of injury and decrease the mass of the calibration artifact to increase cost efficiency.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Moser, Tan and Deng as applied to claim 8 above and in further view of US20210031309A1 by McGovern et al. (cited in the IDS as US11278990B2; hereinafter “McGovern”).
Regarding claim 9, Li modified by Moser, Tan and Deng teach the system of claim 8, and although, Li teaches a uniform heating source, such as a computer display screen, can be used as a calibration artifact ([0053]) which is thermally excited while capturing the second image of the calibration artifact using the infrared camera ([0049]), Li does not explicitly teach wherein the imaging calibration module is configured to apply infrared radiation to the front surface of the calibration artifact.
However, McGovern does address this limitation. McGovern and Li are considered to be analogous to the present invention as they are in the same field of thermal imaging.
McGovern teaches wherein the imaging calibration module is configured to apply infrared radiation to a front surface ([0039] heat source 34, 36 is controlled by the controller 32, and is adapted to sequentially emit respective heat pulses 38, 40 that facilitate controlled heating of the work product 24 and weld 22 to be inspected.)
It would have been well known to someone of ordinary skill in the art before the effective filing date of the claimed invention to configure the imaging calibration module to apply infrared radiation to thermally excite the calibration artifact. Therefore, it would have been obvious to modify Li to include wherein the imaging calibration module is configured to apply infrared radiation to the front surface of the calibration artifact as suggested by McGovern in order to ensure that the heat is desirably absorbed by the work product as oppose to being reflected off of the work product (McGovern [0040]), thus increasing accuracy.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN E KIDWELL whose telephone number is (703)756-1719. The examiner can normally be reached Monday - Friday 8 a.m. - 5 p.m. ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tarifur Chowdhury can be reached at 571-272-2287. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KAITLYN E KIDWELL/Examiner, Art Unit 2877
/TARIFUR R CHOWDHURY/Supervisory Patent Examiner, Art Unit 2877
Prosecution Insights