Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 4/3/2025 and 1/10/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 10, 11, 13-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Goevert el al (US 2009/0129666) in view of Albeck (US 2007/0057946).
As to claims 1, 10, and 11, Goevert teaches a method (and the system) for three-dimensional reconstruction, comprising:
acquiring a first image and a second image, wherein the first image and the second image are respectively obtained by means of different image collection devices collecting a predetermined code element image projected onto the surface of an object to be measured (Para. 0010, “a method comprising the steps of providing at least one camera for recording a plurality of images of the scene including the object, recording a first sequence of first images of the scene from a first perspective relative to the scene, and recording a second sequence of second images of the scene from a second perspective relative to the scene, the first and second perspectives being different from one another, determining a plurality of first image areas within the first images and determining a plurality of second image areas within the second images, identifying a plurality of correspondences between the first and second image areas, and reconstructing the scene based on the correspondences between the first and second image areas, wherein the correspondences are identified by matching a parameterized function to each image area in order to obtain a plurality of first and second function parameters representing the first and second image areas, and by comparing respective first and second function parameters, and wherein the first and second sequences each comprise a plurality of first and second images so that a spatial position of the image areas and any movement over time of the image areas are used in order to identify the correspondences);
acquiring a first target pixel point of each code element in the first image, and determining, in the second image, a matched pixel point for the first target pixel point of each code element in the first image (Para. 0012, “The spatial correspondences represent assignments of image areas within the number of images of at least two image sequences, with an image area from a first image sequence being assigned to an image area from a second image sequence and, if appropriate, to further image areas from further image sequences. The term image area in this case includes extended areas in the image sequences, i.e. a plurality of pixels, and/or a single pixel”); and
determining the three-dimensional coordinates of the first target pixel point at least according to the predetermined pixel point coordinates of the first target pixel point in the first image and the predetermined pixel point coordinates of the matched pixel point in the second image, so as to complete three-dimensional reconstruction (Para. 0011, 0012).
Goevert teaches collecting a predetermined code element image projected onto the surface of an object to be measured, but not clearly points out that the predetermined code element image comprising a plurality of target code elements that are randomly distributed in a preset direction, and the target code elements being line segment stripes, but Albeck teaches these limitations (Albeck discloses projection of stripe-like structured/coded patterns onto an object surface, see Albeck, Fig1 and Fig4, also para 0014-0016);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of the references in order to improve robustness and accuracy of stereo reconstruction by projecting a coded stripe pattern and using correspondence between the two images to compute depth or 3D coordinates.
As to claims 2 and 16, Goevert as modified teaches the method as claimed in claim 1, wherein determining, in the second image, the matched pixel point for the first target pixel point of each code element in the first image (Para. 0011, "wherein the correspondences are identified by matching a parameterized function to each image area in order to obtain a plurality of first and second function parameters representing the first and second image areas") comprises:
determining the pixel point coordinates of the first target pixel point and a gray-scale value of the first target pixel point (Para. 0038, "difference images are generated by respectively subtracting reference images from the image values, in particular from the gray values, of the individual images of the image sequences");
determining a first region with a preset area in the first image, and taking a central point of the first region as the first target pixel point (Para. 0003, "This determination of correspondences corresponds to an identification of pixel positions or pixel areas in the images with points or objects or object sections in the scene to be reconstructed";
determining a second region with the preset area from the second image, wherein the longitudinal coordinate of the center of the second region in the second image is the same as the longitudinal coordinate of the center of the first region in the first image (para 0014); and
determining, from the second region, matched pixel points matching the first target pixel point (para 0011).
As to claims 3 and 17, Goevert as modified teaches the method as claimed in claim 2, wherein determining, from the second region, matched pixel points matching the first target pixel point comprises:
determining the correlation coefficient of each pixel point in the second region one by one, the correlation coefficient being used for representing a correlation between a pixel point in the second region and the first target pixel point; and
determining a pixel point having the maximum correlation coefficient in the second region as the matched pixel point(Para. 0005, "for example, cross correlation coefficients, sum of the squared differences, or sum of the absolute differences. This method is temporally effective, in particular for calibrated stereo image pairs, i.e. image pairs for which only pixels lying on common epipolar lines can be in correspondence")..
As to claims 4 and 18, Goevert as modified teaches the method as claimed in claim 3, wherein determining the correlation coefficient of each pixel point in the second region one by one comprises:
determining an average gray-scale value of all pixel points in the first region as a first average gray-scale value, and determining an average gray-scale value of all pixel points in the second region as a second average gray-scale value; and
determining a correlation coefficient between each pixel point in the second region and the first target pixel point according to a difference between the gray-scale value of the first target pixel point in the first region and the first average gray-scale value and a difference between the gray-scale value of each pixel point in the second region and the second average gray-scale value (Para. 0005 and 0026, "This difference is a measure of the absolute mean image value and/or gray value in the respective image area").
As to claims 5 and 19, Goevert as modified teaches the method as claimed in claim 3, wherein determining the pixel point having the maximum correlation coefficient in the second region as the matched pixel point comprises:
determining a pixel point having the maximum correlation coefficient in the second region as a candidate matched point;
in cases where the candidate matched point overlaps the second target pixel point in the second image, determining the candidate matched point as the matched pixel point; and
in cases where the candidate matched point does not overlap the second target pixel point in the second image, determining the second target pixel in a preset range around the candidate matched point as the matched pixel point (para 0005).
As to claim 6, Goevert as modified teaches the method as claimed in claim 1, wherein the method further comprises:
determining, on the basis of the length of the target code element, the width of the target code element, the spacing between the target code elements, and the pixel point coordinates of a central pixel point of the target code element, a target region where the target code element is located, wherein the pixel point coordinates of the central pixel point of the target code element are randomly generated in the region where the code element image is located;
traversing all pixel points in the target region, and generating the target code element in the target region in cases where there is no described target code element in the target region, wherein the target code element at least comprises a line segment of a preset length and two end points corresponding to the line segment of the preset length; and
generating the target code element at all the target regions within the code element image region (Para. 0042, "In a step 6, a parameterized function h(u,v,t) is adapted to each individual interest pixel and the local environment thereof, preferably on the basis of the original image and/or of the difference image").
As to claims 7 and 20, Goevert as modified teaches the method as claimed in claim 1, wherein the method further comprises:
determining a first neighborhood code element set of any code element in the first image and a plurality of second neighborhood code element set of a plurality of candidate code elements in the second image;
determining the number of matched neighborhood code elements in the plurality of second adjacent code element sets and the number of matched neighborhood code elements in the first neighborhood code element set, and determining, as a target second neighborhood code element set, a second neighborhood code element set having the maximum number of matched neighborhood code elements in the plurality of second neighborhood code element sets; and
determining the candidate code element corresponding to the target second neighborhood code element set as the target code element (para 0010-0012).
As to claim 8, Goevert as modified teaches the method as claimed in claim 1, wherein the method further comprises:
determining a plurality of first target pixel points in the first image and a plurality of second target pixel points in the second image; and
matching the plurality of first target pixel points with the plurality of second target pixel points in the second image on a one-to-one basis (para 0012).
As to claim 13, Goevert as modified teaches the method as claimed in claim 1, wherein the orientations of a plurality of target code elements in the first image are the same, and the spacing between the code elements is randomly determined (Para. 0013, Image sequences are recorded in order to implement the method, with an image sequence consisting of a succession of individual images of the scene, which images are recorded from an observation perspective and preferably have an equidistant temporal spacing. The image sequences are recorded from different observation perspectives, preferably from different observation directions and/or observation distances and/or using various optical imaging devices. Preferably, the images in the respective number of image sequences each are recorded at the same instants of time).
As to claim 14, Goevert as modified teaches the method as claimed in claim 1, wherein there is an epipolar constraint relationship between the first image and the second image (Para. 0023, "There is a particularly advantageous embodiment when the similarity measure is formed exclusively between image areas of the number of image sequences that are assigned to the same epipolars").
As to claim 15, Goevert as modified teaches the method as claimed in claim 3, wherein the correlation coefficient comprises a zero mean normalized cross correlation (ZNCC) determined by using a ZNCC method for matching (para 0005, “A customary method for determining the spatial correspondences is the correlation of the image contents on a local plane, wherein contents of spatial windows are mutually compared by applying suitable error measures as similarity measure such as, for example, cross correlation coefficients, sum of the squared differences, or sum of the absolute differences. This method is temporally effective, in particular for calibrated stereo image pairs, i.e. image pairs for which only pixels lying on common epipolar lines can be in correspondence”).
As to claim 21, Goevert as modified teaches the method as claimed in claim 10, wherein the image collection devices comprise a gray-scale camera and a color camera (Para. 0063, "The cameras are designed as digital cameras, in particular as CCD cameras, by means of which images can be recorded in gray value format. In the case of other applications, it is also possible to use color cameras or thermal imaging cameras or UV cameras. The cameras 10a, b, C are synchronized with one another so that a tuple of three with three individual images is simultaneously recorded.").
Claim 22 is similar to claim 13, thus see rejection above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Liu (US 2024/0378737) discloses a method includes: projecting, by a pattern projector, a plurality of lines to a surface of an object-to-be-measured; collecting, by three cameras, two-dimensional images on the surface of the object-to-be-measured to correspondingly obtain three frames of two-dimensional images; determining matching point pairs between every two of the three frames of two-dimensional images to correspondingly obtain three sets of matching point pairs; verifying matching consistency between the matching point pairs; and performing three-dimensional reconstruction on the matching point pairs with the matching consistency to obtain three-dimensional points on the surface of the object-to-be-measured.
Chen (US 2022/0375164) discloses a method for 3D reconstruction includes: acquiring an image sequence of an object to be reconstructed continuously acquired by a monocular image collector; extracting depth information of an image to be processed in the image sequence; estimating translation attitude information of the image to be processed based on world coordinate information of each feature point in a reference image, image coordinate information of each feature point in the image to be processed, and rotation attitude information of the image to be processed, the reference image being an adjacent image whose acquisition time point in the image sequence is located before the image to be processed; generating a point cloud image based on the depth information, the rotation attitude information and the translation attitude information of each image; and performing 3D reconstruction on the object to be reconstructed based on the point cloud image.
Contact information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENT WU CHANG whose telephone number is (571)272-7667. The examiner can normally be reached Monday to Friday between 8:00am – 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights