DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a correction unit that corrects a parallax deviation...” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. Examiner is interpreting the hardware in accordance with paragraph 24 of the publication of the application.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites an imaging device that is disposed at a known distance and corrects a parallax deviation by capturing a chart having a first pattern and a second pattern, the imaging device comprising: a first imaging unit; a second imaging unit disposed at a position separated by a constant distance from the first imaging unit; a first parallax calculator that calculates a first parallax based on an image of the first pattern captured by the first imaging unit and an image of the first pattern captured by the second imaging unit; a second parallax calculator that calculates a second parallax based on an image of the first pattern captured by the first imaging unit and an image of the second pattern captured by the second imaging unit; and a correction unit that corrects a parallax deviation between the first imaging unit and the second imaging unit based on the first parallax calculated by the first parallax calculator and the second parallax calculated by the second parallax calculator.
The limitation of having imaging units is interpreted as insignificant pre-solution activity. The parallax calculation steps are interpreted as math and fall under the mathematical concepts grouping of abstract ideas. The limitation of “corrects a parallax deviation between the first imaging unit and the second imaging unit based on the first parallax calculated”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting a device, nothing in the claim element precludes the step from practically being performed in the mind. For example, “corrects” in the context of this claim encompasses the user manually determining more or less correction is required based on the amount of parallax. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 2 is more pure math and falls in the mathematical concepts grouping of abstract ideas.
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 3 is more pure math and falls in the mathematical concepts grouping of abstract ideas.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 4 is more pure math and falls in the mathematical concepts grouping of abstract ideas.
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 5 describes more of what is used in the calculation but does not add a description of physical structure.
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 6 describes more of what is used in the calculation but does not add a description of physical structure.
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 7 describes more of what is used in the calculation but does not add a description of physical structure.
Claims 8-14 are rejected with the same rationale as claims 1-7.
Examiner recommends adding language in accordance with the specification to demonstrate the correction is used in a vehicle driver assistance program such as collision damage mitigation brake functions or similar language. There are several practical applications discussed in the specification that would make the claims patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Murata et al. (JPH10341458A [Machine Translation]) in view of Nishihara (US 20090116742 A1).
Regarding claims 1 and 8, Murata et al. disclose an imaging device that is disposed at a known distance and corrects a parallax deviation by capturing a chart having a first pattern and a second pattern, the imaging device comprising and parallax deviation correction method for correcting a parallax deviation of an imaging device using a chart that is disposed at a known distance and has a first pattern and a second pattern, the method comprising: a first imaging unit (camera 1, [0022], [0028]
PNG
media_image1.png
406
430
media_image1.png
Greyscale
); a second imaging unit disposed at a position separated by a constant distance from the first imaging unit (The two cameras, 1 and 2, are positioned apart from each other, and the distance between the centers of their lenses is b, [0022], In Figure 3, cameras 1 and 2 are mounted on the front of vehicle 3 in the standard stereo configuration shown in Figure 1, [0028]); a first parallax calculator that calculates a first parallax based on an image of the first pattern captured by the first imaging unit and an image of the first pattern captured by the second imaging unit (a traffic light (green light) is used as a feature object for calibration, [0028], In left camera 1, traffic light 4 is visible on the optical axis. On the other hand, camera 2 captures the traffic light 4 at a distance dA from the optical axis. Therefore, the parallax observed at point A is dA, [0030]); a second parallax calculator that calculates a second parallax based on an image of the first pattern captured by the first imaging unit and an image of the second pattern captured by the second imaging unit (recognizing the same stationary feature from images captured at a plurality of calibration shooting points on the road, [0018], distance to traffic light 4 is measured at point B, parallax observed at point B is in dB, [0032], Characteristic features include traffic lights, signs, pedestrian overpasses, buildings, and billboards, [0045]); and a correction unit that corrects a parallax deviation between the first imaging unit and the second imaging unit based on the first parallax calculated by the first parallax calculator and the second parallax calculated by the second parallax calculator (The angle of deviation Δθ in the camera's installation direction can be calculated using the measured values dA, dB, and Z as follows. The actual distance DA + ΔDA to the stationary object at point A is expressed by the following equation (2). Here, (dA - Δd) is the parallax value that should be observed when the installation directions of both cameras 1 and 2 are not misaligned, [0034], one of α or β can be identified as the parallax error Δd, [0039], “If a stereo camera is equipped with a drive mechanism for correcting the camera's orientation, the camera can be calibrated by rotating it by an angle Δθ to correct its orientation. Alternatively, the measurement results may be corrected using Δd as a camera calibration. In other words, by substituting the value obtained by subtracting Δd from the parallax observed during distance measurement into equation (1), the correct measurement value can be obtained. In this case, camera rotation is not necessary”, [0040]).
As multiple landmarks around the vehicle can be used, this could be interpreted as a first pattern and a second pattern, however another reference is added to make this more explicit. As the term “chart” is part of the preamble and not main body of the claim, this is not given patentable weight. As the terms first pattern and a second pattern are in the body of the claim, these terms are given some weight though as noted the terms can be interpreted broadly.
Nishihara teaches a first pattern and a second pattern (FIG. 3 illustrates an example of a calibration pattern 100 for the gesture recognition interface system 10 in accordance with an aspect of the invention. The calibration pattern 100 includes a non-continuous border 102 that can be located near the edges of the projection area of the projector 26, and also includes a first gap 104, a second gap 106, and a third gap 108, [0030], As an example, FIG. 4 illustrates another example of a calibration pattern 150 for a gesture recognition interface system in accordance with an aspect of the invention. The calibration pattern 150 includes a repeated pattern of features that contrast with the background surface 20, demonstrated in the example of FIG. 4 as a plurality of black dots 152, [0033], “At 256, at least one calibration pattern is projected onto the background surface in the second light spectrum. The second light spectrum can be visible light. At 258, features of the at least one calibration pattern are associated with known physical locations. The at least one calibration pattern can include a first calibration pattern with features that define the projection boundaries of a projector that projects the calibration pattern relative to the background surface. The at least one calibration pattern can also include a second calibration pattern with features that are associated with the known physical locations in two-dimensional space on the background surface. The association can occur based on a parallax separation of the features in images received at each of multiple stereo cameras”, [0045]).
PNG
media_image2.png
660
458
media_image2.png
Greyscale
).
Murata et al. and Nishihara are in the same art of stereo camera calibration (Murata et al., [0001]; Nishihara, [0045]). The combination of Nishihara with Murata et al. will enable using a first pattern and a second pattern. It would have been obvious at the time of filing to combine the patterns of Nishihara with the invention of Murata et al. as this was known at the time of filing, the combination would have predictable results, and as Nishihara indicate “As described herein, the gesture recognition system can be calibrated in an automated manner, such that lengthy and specific manual calibration can be avoided” ([0016]) and “The controller 24 can thus correlate the location of the features of the predefined calibration pattern 202 appearing in the images of the cameras 12 and 14 with the precise physical locations on the background surface 20. As a result, the controller 24 can extrapolate three-dimensional locations of the input object 22 based on the calibration information that is calculated based on the physical locations of the dots 204 on the background surface 20” ([0042]) thereby providing a speed and accuracy benefit to combining inventions.
Regarding claims 7 and 14, Murata et al. and Nishihara disclose the imaging device and method according to claims 1 and 8. Murata et al. and Nishihara further indicate when the first pattern and the second pattern have different patterns, the second parallax calculator: uses a template image of the first pattern preliminarily prepared as a standard image and the image of the first pattern captured by the first imaging unit as a reference image to detect a position of the first pattern on the image by template matching; uses a template image of the second pattern preliminarily prepared as a standard image and the image of the second pattern captured by the second imaging unit as a reference image to detect a position of the second pattern on the image by template matching; and calculates a difference between the detected position of the first pattern on the image and the detected position of the second pattern on the image as the second parallax (Murata et al., The angle of deviation Δθ in the camera's installation direction can be calculated using the measured values dA, dB, and Z as follows. The actual distance DA + ΔDA to the stationary object at point A is expressed by the following equation (2). Here, (dA - Δd) is the parallax value that should be observed when the installation directions of both cameras 1 and 2 are not misaligned, [0034], one of α or β can be identified as the parallax error Δd, [0039], “If a stereo camera is equipped with a drive mechanism for correcting the camera's orientation, the camera can be calibrated by rotating it by an angle Δθ to correct its orientation. Alternatively, the measurement results may be corrected using Δd as a camera calibration. In other words, by substituting the value obtained by subtracting Δd from the parallax observed during distance measurement into equation (1), the correct measurement value can be obtained. In this case, camera rotation is not necessary”, [0040]; Nishihara, FIG. 3 illustrates an example of a calibration pattern 100 for the gesture recognition interface system 10 in accordance with an aspect of the invention. The calibration pattern 100 includes a non-continuous border 102 that can be located near the edges of the projection area of the projector 26, and also includes a first gap 104, a second gap 106, and a third gap 108, [0030], As an example, FIG. 4 illustrates another example of a calibration pattern 150 for a gesture recognition interface system in accordance with an aspect of the invention. The calibration pattern 150 includes a repeated pattern of features that contrast with the background surface 20, demonstrated in the example of FIG. 4 as a plurality of black dots 152, [0033], “At 256, at least one calibration pattern is projected onto the background surface in the second light spectrum. The second light spectrum can be visible light. At 258, features of the at least one calibration pattern are associated with known physical locations. The at least one calibration pattern can include a first calibration pattern with features that define the projection boundaries of a projector that projects the calibration pattern relative to the background surface. The at least one calibration pattern can also include a second calibration pattern with features that are associated with the known physical locations in two-dimensional space on the background surface. The association can occur based on a parallax separation of the features in images received at each of multiple stereo cameras”, [0045]).
Claim(s) 2-4 and 9-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Murata et al. (JPH10341458A [Machine Translation]) and Nishihara (US 20090116742 A1) as applied to claims 1 and 8 above, further in view of Imagawa et al. (JP2021025922A [Machine Translation]).
Regarding claims 2 and 9, Murata et al. and Nishihara disclose the imaging device and method according to claims 1 and 8. Murata et al. further partly indicate the correction unit calculates an installation deviation amount of the chart based on the first parallax calculated by the first parallax calculator, the second parallax calculated by the second parallax calculator, and a parallax calculated based on the known distance, calculates an installation distance of the chart based on the calculated installation deviation amount of the chart and the known distance, and corrects a parallax deviation between the first imaging unit and the second imaging unit based on the calculated installation distance of the chart (Furthermore, the present invention relates to an in-vehicle stereo camera having a function for calibrating misalignment of the camera's installation direction, [0001], The distance measurement method described above assumes that the two cameras are installed in the correct positions shown in Figure 1. However, the stereo cameras mounted on the vehicle are used in harsh conditions, such as when they are subjected to vibrations while driving. If the two cameras become misaligned due to vibration or other factors, a significant error will occur in the distance measurement results. Therefore, the camera's orientation deviation is obtained using the following method, and the camera calibration is performed using the results. [0027], “The parallax dA is the normal parallax plus the offset Δd due to the orientation of the installation, [0030], The angle of deviation Δθ in the camera's installation direction can be calculated using the measured values dA, dB, and Z as follows. The actual distance DA + ΔDA to the stationary object at point A is expressed by the following equation (2). Here, (dA - Δd) is the parallax value that should be observed when the installation directions of both cameras 1 and 2 are not misaligned”, [0034], Therefore, one of α or β can be identified as the parallax error Δd. Furthermore, using equation (9) below, the angle of displacement Δθ in the camera installation direction can be determined from Δd, [0039]).
Murata et al. and Nishihara do not disclose a chart [As the term “chart” was part of the preamble and not main body of claim 1, this was not given patentable weight in the independent claim but is given weight in the dependent claim citing chart in the body of the claim].
Imagawa et al. teach the correction unit calculates an installation deviation amount of the chart based on the first parallax calculated by the first parallax calculator, the second parallax calculated by the second parallax calculator, and a parallax calculated based on the known distance, calculates an installation distance of the chart based on the calculated installation deviation amount of the chart and the known distance, and corrects a parallax deviation between the first imaging unit and the second imaging unit based on the calculated installation distance of the chart (This document describes a method of photographing multiple charts placed at different distances using a stereo camera, correcting for camera optical distortion, and calibrating the translational misalignment of one camera relative to the other, [0002], “This stereo camera adjustment device includes a correction unit and a calibration unit. Here, the correction unit processes a pair of image data output by capturing a chart having a predetermined pattern with a stereo camera, and calculates correction parameters for each image data to transform the image data so that the pattern projected onto the image plane defined by the image data approaches the predetermined pattern of the chart. Here, this correction parameter corrects at least the distortion of image data caused by the camera's optical distortion. Furthermore, the calibration unit calculates calibration parameters that correct the misalignment of a pair of image data caused by the positional misalignment of a pair of cameras, using the characteristics of the images related to the pair of image data transformed by the calculated correction parameters as constraint”, [0003], “Therefore, when mounted on the vehicle 101, variations in windshield thickness and other factors can cause image rotation and image plane shift, which are corrected to update the parameter memory 205”, [0059], “As described above, the correction device of this embodiment can provide charts of multiple distances in a space-saving and inexpensive manner, and can correct the stereo camera 200 with high precision while relaxing the installation accuracy requirements for the vehicle 101”, [0064], The infinite distance chart 401B is rewritten (606) by taking into account the correction amount obtained in steps 603 and 604 above, using the data in the correction parameter memory 205 shown in Figure 10, [0083], As described above, by arranging two optical devices 301 and 4011A that illuminate the infinitely distant charts 304 and 401B and performing imaging with the stereo camera 200, the relative translation and rotation of the left and right images can be corrected, [0084], (8) A method for correcting the stereo camera (200) described in (2) or (3) above, wherein a second optical device (401A) has a third chart (401B) having a predetermined pattern at a position different from the first optical device (301), and has a mirror surface (306) with an aperture greater than or equal to the baseline length (L) of the stereo camera (200), generates a virtual image of the third chart (401B), and the virtual image of the third chart (401B) is captured by a plurality of cameras (201) of the stereo camera (200), and parameters corresponding to the rotation and translation of the camera (201) images are corrected using the image of the first chart (304) and the image of the third chart (401B), [0100]).
Murata et al. and Nishihara and Imagawa et al. are in the same art of stereo camera calibration (Murata et al., [0001]; Nishihara, [0045]; Imagawa et al., [0002]). The combination of Imagawa et al. with Murata et al. and Nishihara will enable using a chart. It would have been obvious at the time of filing to combine the chart of Imagawa et al. with the invention of Murata et al. and Nishihara as this was known at the time of filing, the combination would have predictable results, and as Imagawa et al. state “Therefore, the present invention has been made in view of the above problems, and aims to correct stereo cameras with high accuracy and low cost” ([0012]) and “Therefore, the present invention can provide charts of multiple distances in a space-saving and inexpensive manner, and can also perform high-precision correction of stereo cameras while easing the requirements for vehicle installation accuracy” ([0014]) providing a cost benefit to combining inventions.
Regarding claims 3 and 10, Murata et al. and Nishihara and Imagawa et al. disclose the imaging device according to claims 2 and 9. Murata et al. and Imagawa et al. further indicate the correction unit calculates a difference between parallax deviation amounts based on the first parallax calculated by the first parallax calculator, the second parallax calculated by the second parallax calculator, and the parallax calculated based on the known distance, and calculates an installation deviation amount of the chart based on a relation among the calculated difference between the parallax deviation amounts, a preset parallax deviation amount, and an installation deviation amount of the chart (Murata et al., The angle of deviation Δθ in the camera's installation direction can be calculated using the measured values dA, dB, and Z as follows. The actual distance DA + ΔDA to the stationary object at point A is expressed by the following equation (2). Here, (dA - Δd) is the parallax value that should be observed when the installation directions of both cameras 1 and 2 are not misaligned, [0034], one of α or β can be identified as the parallax error Δd, [0039], “If a stereo camera is equipped with a drive mechanism for correcting the camera's orientation, the camera can be calibrated by rotating it by an angle Δθ to correct its orientation. Alternatively, the measurement results may be corrected using Δd as a camera calibration. In other words, by substituting the value obtained by subtracting Δd from the parallax observed during distance measurement into equation (1), the correct measurement value can be obtained. In this case, camera rotation is not necessary”, [0040]; Imagawa et al., This document describes a method of photographing multiple charts placed at different distances using a stereo camera, correcting for camera optical distortion, and calibrating the translational misalignment of one camera relative to the other, [0002], “This stereo camera adjustment device includes a correction unit and a calibration unit. Here, the correction unit processes a pair of image data output by capturing a chart having a predetermined pattern with a stereo camera, and calculates correction parameters for each image data to transform the image data so that the pattern projected onto the image plane defined by the image data approaches the predetermined pattern of the chart. Here, this correction parameter corrects at least the distortion of image data caused by the camera's optical distortion. Furthermore, the calibration unit calculates calibration parameters that correct the misalignment of a pair of image data caused by the positional misalignment of a pair of cameras, using the characteristics of the images related to the pair of image data transformed by the calculated correction parameters as constraint”, [0003], “Therefore, when mounted on the vehicle 101, variations in windshield thickness and other factors can cause image rotation and image plane shift, which are corrected to update the parameter memory 205”, [0059], “As described above, the correction device of this embodiment can provide charts of multiple distances in a space-saving and inexpensive manner, and can correct the stereo camera 200 with high precision while relaxing the installation accuracy requirements for the vehicle 101”, [0064], The infinite distance chart 401B is rewritten (606) by taking into account the correction amount obtained in steps 603 and 604 above, using the data in the correction parameter memory 205 shown in Figure 10, [0083], As described above, by arranging two optical devices 301 and 4011A that illuminate the infinitely distant charts 304 and 401B and performing imaging with the stereo camera 200, the relative translation and rotation of the left and right images can be corrected, [0084]).
Regarding claims 4 and 11, Murata et al. and Nishihara and Imagawa et al. disclose the imaging device and method according to claims 2 and 9. Murata et al. further indicate the correction unit calculates a parallax deviation amount between the first imaging unit and the second imaging unit based on the calculated installation distance of the chart and the first parallax calculated by the first parallax calculator, and corrects the parallax deviation between the first imaging unit and the second imaging unit based on the calculated parallax deviation amount (The angle of deviation Δθ in the camera's installation direction can be calculated using the measured values dA, dB, and Z as follows. The actual distance DA + ΔDA to the stationary object at point A is expressed by the following equation (2). Here, (dA - Δd) is the parallax value that should be observed when the installation directions of both cameras 1 and 2 are not misaligned, [0034], one of α or β can be identified as the parallax error Δd, [0039], “If a stereo camera is equipped with a drive mechanism for correcting the camera's orientation, the camera can be calibrated by rotating it by an angle Δθ to correct its orientation. Alternatively, the measurement results may be corrected using Δd as a camera calibration. In other words, by substituting the value obtained by subtracting Δd from the parallax observed during distance measurement into equation (1), the correct measurement value can be obtained. In this case, camera rotation is not necessary”, [0040]).
Claim(s) 5, 6, 7, 12, 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Murata et al. (JPH10341458A [Machine Translation]) and Nishihara (US 20090116742 A1) as applied to claims 1 and 8 above, further in view of Matono et al. (US 20170180701 A1).
Regarding claim 5 and 12, Murata et al. and Nishihara disclose the imaging device and method according to claims 1 and 8. Murata et al. partly indicate the first parallax calculator uses the image of the first pattern captured by the first imaging unit as a standard image and the image of the first pattern captured by the second imaging unit as a reference image to calculate the first parallax by template matching (Then, the shape of the candidate object is compared with the template, and it is determined whether or not the candidate object is a feature to be extracted, [0045]) however another reference is added to make this point more explicit.
Matono et al. teach the first parallax calculator uses the image of the first pattern captured by the first imaging unit as a standard image and the image of the first pattern captured by the second imaging unit as a reference image to calculate the first parallax by template matching (“FIG. 5 shows a configuration example mainly showing the detailed configuration of the stereo camera 102 and the correspondence between the stereo camera 102 and the arbitration unit 206. The stereo camera 102 can calculate positional aberration (parallax) of the same object (three-dimensional object) such as a pedestrian on plural images imaged at the same time by template matching and can measure a distance of the object (three-dimensional object) based on the calculated parallax, which includes a camera 501 as a first imaging unit and a camera 502 as a second imaging unit to obtain plural images”, [0027]).
Murata et al. and Nishihara and Matono et al. are in the same art of stereo cameras (Murata et al., [0001]; Nishihara, [0045]; Matono et al., [0027]). The combination of Matono et al. with Murata et al. and Nishihara will enable using a template. It would have been obvious at the time of filing to combine the template of Matono et al. with the invention of Murata et al. and Nishihara as this was known at the time of filing, the combination would have predictable results, and as Matono et al. state “According to the process, another sensor can substitute for part of functions, for example, when the sensor is out of order, therefore, the reliability of the entire system is improved” ([0024]) and “As the collision determination result is particularly important information in the system, the bus 30 is provided between the three-dimensional object detection unit 504 and the arbitration unit 206 for changing the priority more efficiently, thereby receiving information indicating that the distance with respect to the obstacle is close immediately by the three-dimensional object detection unit 504 through the bus 30” ([0029]) thereby providing a safety benefit to combining inventions.
Regarding claims 6 and 13, Murata et al. and Nishihara disclose the imaging device and method according to claims 1 and 8. Murata et al. and Nishihara do not disclose when the first pattern and the second pattern have a same pattern, the second parallax calculator uses the image of the first pattern captured by the first imaging unit as a standard image and the image of the second pattern captured by the second imaging unit as a reference image to calculate the second parallax by template matching.
Matono et al. teach when the first pattern and the second pattern have a same pattern, the second parallax calculator uses the image of the first pattern captured by the first imaging unit as a standard image and the image of the second pattern captured by the second imaging unit as a reference image to calculate the second parallax by template matching (“FIG. 5 shows a configuration example mainly showing the detailed configuration of the stereo camera 102 and the correspondence between the stereo camera 102 and the arbitration unit 206. The stereo camera 102 can calculate positional aberration (parallax) of the same object (three-dimensional object) such as a pedestrian on plural images imaged at the same time by template matching and can measure a distance of the object (three-dimensional object) based on the calculated parallax, which includes a camera 501 as a first imaging unit and a camera 502 as a second imaging unit to obtain plural images”, [0027]).
Murata et al. and Nishihara and Matono et al. are in the same art of stereo cameras (Murata et al., [0001]; Nishihara, [0045]; Matono et al., [0027]). The combination of Matono et al. with Murata et al. and Nishihara will enable using a template. It would have been obvious at the time of filing to combine the template of Matono et al. with the invention of Murata et al. and Nishihara as this was known at the time of filing, the combination would have predictable results, and as Matono et al. state “According to the process, another sensor can substitute for part of functions, for example, when the sensor is out of order, therefore, the reliability of the entire system is improved” ([0024]) and “As the collision determination result is particularly important information in the system, the bus 30 is provided between the three-dimensional object detection unit 504 and the arbitration unit 206 for changing the priority more efficiently, thereby receiving information indicating that the distance with respect to the obstacle is close immediately by the three-dimensional object detection unit 504 through the bus 30” ([0029]) thereby providing a safety benefit to combining inventions.
Regarding claims 7 and 14, Murata et al. and Nishihara disclose the imaging device and method according to claims 1 and 8. Murata et al. and Nishihara further indicate when the first pattern and the second pattern have different patterns, the second parallax calculator: uses a template image of the first pattern preliminarily prepared as a standard image and the image of the first pattern captured by the first imaging unit as a reference image to detect a position of the first pattern on the image by template matching; uses a template image of the second pattern preliminarily prepared as a standard image and the image of the second pattern captured by the second imaging unit as a reference image to detect a position of the second pattern on the image by template matching; and calculates a difference between the detected position of the first pattern on the image and the detected position of the second pattern on the image as the second parallax (Murata et al., The angle of deviation Δθ in the camera's installation direction can be calculated using the measured values dA, dB, and Z as follows. The actual distance DA + ΔDA to the stationary object at point A is expressed by the following equation (2). Here, (dA - Δd) is the parallax value that should be observed when the installation directions of both cameras 1 and 2 are not misaligned, [0034], one of α or β can be identified as the parallax error Δd, [0039], “If a stereo camera is equipped with a drive mechanism for correcting the camera's orientation, the camera can be calibrated by rotating it by an angle Δθ to correct its orientation. Alternatively, the measurement results may be corrected using Δd as a camera calibration. In other words, by substituting the value obtained by subtracting Δd from the parallax observed during distance measurement into equation (1), the correct measurement value can be obtained. In this case, camera rotation is not necessary”, [0040]; Nishihara, FIG. 3 illustrates an example of a calibration pattern 100 for the gesture recognition interface system 10 in accordance with an aspect of the invention. The calibration pattern 100 includes a non-continuous border 102 that can be located near the edges of the projection area of the projector 26, and also includes a first gap 104, a second gap 106, and a third gap 108, [0030], As an example, FIG. 4 illustrates another example of a calibration pattern 150 for a gesture recognition interface system in accordance with an aspect of the invention. The calibration pattern 150 includes a repeated pattern of features that contrast with the background surface 20, demonstrated in the example of FIG. 4 as a plurality of black dots 152, [0033], “At 256, at least one calibration pattern is projected onto the background surface in the second light spectrum. The second light spectrum can be visible light. At 258, features of the at least one calibration pattern are associated with known physical locations. The at least one calibration pattern can include a first calibration pattern with features that define the projection boundaries of a projector that projects the calibration pattern relative to the background surface. The at least one calibration pattern can also include a second calibration pattern with features that are associated with the known physical locations in two-dimensional space on the background surface. The association can occur based on a parallax separation of the features in images received at each of multiple stereo cameras”, [0045]).
Murata et al. and Nishihara do not use the term template.
Matono et al. teach when the first pattern and the second pattern have different patterns, the second parallax calculator: uses a template image of the first pattern preliminarily prepared as a standard image and the image of the first pattern captured by the first imaging unit as a reference image to detect a position of the first pattern on the image by template matching; uses a template image of the second pattern preliminarily prepared as a standard image and the image of the second pattern captured by the second imaging unit as a reference image to detect a position of the second pattern on the image by template matching; and calculates a difference between the detected position of the first pattern on the image and the detected position of the second pattern on the image as the second parallax (“FIG. 5 shows a configuration example mainly showing the detailed configuration of the stereo camera 102 and the correspondence between the stereo camera 102 and the arbitration unit 206. The stereo camera 102 can calculate positional aberration (parallax) of the same object (three-dimensional object) such as a pedestrian on plural images imaged at the same time by template matching and can measure a distance of the object (three-dimensional object) based on the calculated parallax, which includes a camera 501 as a first imaging unit and a camera 502 as a second imaging unit to obtain plural images”, [0027]).
Murata et al. and Nishihara and Matono et al. are in the same art of stereo cameras (Murata et al., [0001]; Nishihara, [0045]; Matono et al., [0027]). The combination of Matono et al. with Murata et al. and Nishihara will enable using a template. It would have been obvious at the time of filing to combine the template of Matono et al. with the invention of Murata et al. and Nishihara as this was known at the time of filing, the combination would have predictable results, and as Matono et al. state “According to the process, another sensor can substitute for part of functions, for example, when the sensor is out of order, therefore, the reliability of the entire system is improved” ([0024]) and “As the collision determination result is particularly important information in the system, the bus 30 is provided between the three-dimensional object detection unit 504 and the arbitration unit 206 for changing the priority more efficiently, thereby receiving information indicating that the distance with respect to the obstacle is close immediately by the three-dimensional object detection unit 504 through the bus 30” ([0029]) thereby providing a safety benefit to combining inventions.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: US 20240404109 A1 [does not predate]: Accordingly, the calibration device 6 according to the present embodiment eliminates the parallax difference Go through a software-wise approach based on the correspondence relationship between the position coordinates of an image IMG and the position coordinates of a calibration pattern PTNc in a light illumination range IRR. Specifically, the generating unit 14, based on the amount of parallax difference acquired from the calculating unit 12, corrects the correspondence relationship between the position coordinates of the image IMG and the position coordinates of the calibration pattern PTNc in the light illumination range IRR acquired from the acquiring unit 10. Then, the generating unit 14 stores the corrected correspondence relationship into the memory 20 as mutual position information of the light illumination range IRR and the imaging range. The generating unit 14 according to the present embodiment generates first mutual position information by adding the amount of parallax difference to the position coordinates of the first pattern PTNc1. The generating unit 14 also generates second mutual position information by adding the amount of parallax difference to the position coordinates of the second pattern PTNc2. Then, the generating unit 14 stores the first mutual position information and the second mutual position information into the memory 20.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M ENTEZARI HAUSMANN whose telephone number is (571)270-5084. The examiner can normally be reached 10-7 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent M Rudolph can be reached at (571) 272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE M ENTEZARI HAUSMANN/Primary Examiner, Art Unit 2671
Prosecution Insights