DETAILED ACTION
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3 and 6-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Geiger et. al. “Automatic Camera and Range Sensor Calibration using a single Shot” hereinafter referred to as Geiger in view of Arora et al. US 10,839,557 hereinafter referred to as Arora.
In regards to claim 1, Geiger teaches:
“A method for automatically recognizing correspondence between calibration pattern feature points in camera calibration”
Geiger section III Camera-To-Camera Calibration teaches planar checkerboard patterns as calibration targets for multiple reasons: They are cheap to employ, corners can be localized with high sub-pixel accuracy and structure recovery profits from strong edges between corners.
“comprising: generating calibration pattern candidates from an input two-dimensional (2D) image using a point-line-face ... hierarchical structure; and performing verification on the calibration pattern candidates based on a preset verification condition”
According to Applicant’s disclosure paragraph [0033] In an image 100 according to an embodiment, as illustrated in FIG. 2, a 3D pattern structure is set to a polyhedron. Each face is implemented as a 2D plane pattern of typical camera calibration. Therefore, it is assumed that the entire structure is implemented as a hierarchical structure of points, lines, a face, and multiple faces (i.e., a point-line-face-multi-face hierarchical structure). The Examiner interprets based on this portion of the disclosure that a 2-D structure would equate to a point-line-face hierarchical structure. Geiger Section III A. Corner detection and Figure 2(c) teaches input image I which is a 2-D image. Geiger further teaches in section III A. Corner detection to produce a list of corner candidates, we apply conservative non-maxima-suppression (with parameters nnms and τnms) [18] on C, followed by verifying the candidates by their gradient statistics in a local n × n pixel neighborhood. Geiger section III C Structure recovery teaches Given a seed corner, we search for its closest neighbors in the direction of its edges e1 and e2, yielding an initial 2 2 checkerboard hypothesis with an associated energy value E(XY). To optimize E(XY), we propose expansion moves on Y, which expand any of the checkerboard borders by a single row or column. Amongst all four possibilities, we select the proposal, which reduces E(XY) the most. Fig. 3(a) illustrates the expansion moves exemplarily. The Examiner interprets that starting from a corner and moving along edges (lines) to recover the face of the checker board is equivalent to point-line-face hierarchical structure. See recovered face in Figure 3(c).
“wherein the generating of the calibration pattern candidates comprises: analyzing a 2D coordinate point on a captured 2D image plane, and determine the point, the line, the face, … of a three- dimensional pattern which the 2D coordinate point of the 2D image plane corresponds to”
Geiger further teaches in section III A. Corner detection to produce a list of corner candidates, we apply conservative non-maxima-suppression (with parameters nnms and τnms) [18] on C, followed by verifying the candidates by their gradient statistics in a local n × n pixel neighborhood. Geiger section III C Structure recovery teaches Given a seed corner, we search for its closest neighbors in the direction of its edges e1 and e2, yielding an initial 2 2 checkerboard hypothesis with an associated energy value E(XY). To optimize E(XY), we propose expansion moves on Y, which expand any of the checkerboard borders by a single row or column. Amongst all four possibilities, we select the proposal, which reduces E(XY) the most. Fig. 3(a) illustrates the expansion moves exemplarily. The Examiner interprets that the input image of Figure 2(c) is a captured 2d image and performing mathematical processes like non-maxima suppression and gradient statistics requires a coordinate system. The Examiner further interprets that the checkerboards are three-dimensional objects and placed at an angle to the camera such that they are perceived as three-dimensional space as well. Additionally, the Examiner interprets that Figure 3 illustrates the recovery of the point-line and face of each checkerboard.
Geiger does not explicitly teach:
“[point-line-face]-multi-face” and “and the multi-face structure”
However, this would be considered a routine implementation. Geiger teaches in Figure 3C detecting multiple 2-D calibration patterns in one image. It is clear that each checkerboard pattern has its own face and therefore the image as a whole detects multiple faces. Those of ordinary skill would readily understand that the different calibration patterns could be placed on different faces of a singular object for the same results. For example, Arora Figures 4A-D and column 8 and lines 14-28 teach an approach in accordance with various embodiments uses a three-dimensional chessboard or checkerboard style calibration object 400. In an example, the calibration object 400 may have all markers as unique, such as from a predefined ArUco dictionary, which provides a library for augmented reality applications based in part on OpenCv support (Open Source Computer Vision Library). Such a dictionary may rely on black and white markers with codes that are detected by calling a singular functions. For robustness, each camera may include views to multiple faces of the calibration object 400. As a result, the calibration object 400 may be tilted to have a diagonal vertical while being placed on a surface. The pattern corners are detected in each image and tuples of 2D and 3D correspondences become the input to the calibration process herein. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Geiger in view of Arora to have included the features of “[point-line-face]-multi-face” because The physical shape and the size integrated into an augmented reality implementation of a live environment may still not resolve a user's expectation of the item's view and aesthetics in the two dimensional integration in the live environment (Arora column 19-23).
In regards to claim 2, Geiger/Arora teach all the limitations of claim 1 and further teach:
“wherein the input image is an image acquired by capturing a subject while moving around the subject”
Arora Figure 5 and column 9 lines 20-28 teach cameras perform a rotation by a common angle around a common axis, and the intrinsic parameters of each camera can be considered constant. In this way, instead of treating successive views of the turntable acquired by a give camera as if these have been in fact acquired each from an independent camera, it can be recognized that each camera performs, relative to the turntable, a rotational motion around a fixed axis of rotation of the turntable. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Geiger in view of Arora to have included the features of “wherein the input image is an image acquired by capturing a subject while moving around the subject” because The physical shape and the size integrated into an augmented reality implementation of a live environment may still not resolve a user's expectation of the item's view and aesthetics in the two dimensional integration in the live environment (Arora column 19-23).
In regards to claim 3, Geiger/Arora teach all the limitations of claim 2 and further teach:
“wherein the input image is represented by at least one of a checkerboard, an ellipse, and a concentric circle or a combination thereof, and a location of each point is determined by analyzing a characteristic of at least one of the checkerboard, the ellipse, and the concentric circle or a combination thereof”
Geiger section III Camera-To-Camera Calibration teaches planar checkerboard patterns as calibration targets for multiple reasons: They are cheap to employ, corners can be localized with high sub-pixel accuracy and structure recovery profits from strong edges between corners.
In regards to claim 6, Geiger/Arora teach all the limitations of claim 1 and claim 6 contains similar limitations written in apparatus form. It would have been obvious to practice the invention as an apparatus. Therefore, claim 6 is rejected for similar reasoning as applied to claim 1.
In regards to claim 7, Geiger/Arora teach all the limitations of claim 6 and claim 7 contains similar limitations as in claim 2. Therefore, claim 7 is rejected for similar reasoning as plied to claim 2.
In regards to claim 8, Geiger/Arora teach all the limitations of claim 7 and claim 6 contains similar limitations as in claim 3. Therefore, claim 8 is rejected for similar reasoning as applied to claim 3.
Claim(s) 4-5 and 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Geiger in view of Arora in view of Yan et al. US 2021/0051317 hereinafter referred to as Yan.
In regards to claim 4, Geiger/Arora teach all the limitations of claim 2 but do not explicitly teach:
“wherein each line is generated using a preset number of points”
Yan paragraph [0038] teaches an image sensor with a pattern/image/feature recognition system running on a computer system (e.g., the local computing system 110) can detect the pattern 204, and identify points representing vertices between the dark and light chessboard squares. By drawing lines connecting these points, the computer system can generate a matrix or grid. The Examiner interprets that the preset number of points is known when the target is generated because the target pattern defines the points at the vertices. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Geiger/Arora in view of Yan to have included the features of “wherein each line is generated using a preset number of points” because as passengers are becoming increasingly reliant on vehicle sensors for safe and efficient operation of motor vehicles, it is imperative that motor vehicles have as robust of an understanding of their environment as possible (Yan [0004]).
In regards to claim 5, Geiger/Arora teach all the limitations of claim 1 and further teach:
“three- dimensional (3D) [structure]”
Arora Figures 4A-D and column 8 and lines 14-28 teach an approach in accordance with various embodiments uses a three-dimensional chessboard or checkerboard style calibration object 400. In an example, the calibration object 400 may have all markers as unique, such as from a predefined ArUco dictionary, which provides a library for augmented reality applications based in part on OpenCv support (Open Source Computer Vision Library). Such a dictionary may rely on black and white markers with codes that are detected by calling a singular functions. For robustness, each camera may include views to multiple faces of the calibration object 400. As a result, the calibration object 400 may be tilted to have a diagonal vertical while being placed on a surface. The pattern corners are detected in each image and tuples of 2D and 3D correspondences become the input to the calibration process herein. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Geiger in view of Arora to have included the features of “three- dimensional (3D) [structure]” because The physical shape and the size integrated into an augmented reality implementation of a live environment may still not resolve a user's expectation of the item's view and aesthetics in the two dimensional integration in the live environment (Arora column 19-23).
Geiger/Arora do not explicitly teach:
“wherein each face is determined based on finite sets of lines, and a maximum number of finite sets of lines is determined when a [calibration] structure is configured”
Yan paragraph [0038] teaches an image sensor with a pattern/image/feature recognition system running on a computer system (e.g., the local computing system 110) can detect the pattern 204, and identify points representing vertices between the dark and light chessboard squares. By drawing lines connecting these points, the computer system can generate a matrix or grid. The Examiner interprets that the preset number of points is known when the target is generated because the target pattern defines the points at the vertices. It would have been obvious for a person with ordinary skill in the art before the invention was effectively filed to have modified Geiger/Arora in view of Yan to have included the features of “wherein each face is determined based on finite sets of lines, and a maximum number of finite sets of lines is determined when a [calibration] structure is configured” because as passengers are becoming increasingly reliant on vehicle sensors for safe and efficient operation of motor vehicles, it is imperative that motor vehicles have as robust of an understanding of their environment as possible (Yan [0004]).
In regards to claim 9, Geiger/Arora teach all the limitations of claim 6 and claim 9 contains similar limitations as in claim 4. Therefore, claim 9 is rejected for similar reasoning as applied to claim 4.
In regards to claim 10, Geiger/Arora teach all the limitations of claim 6 and claim 10 contains similar limitations as in claim 5. Therefore, claim 10 is rejected for similar reasoning as applied to claim 5.
Response to Arguments
Applicant's arguments filed 4/30/2025 have been fully considered but they are not persuasive.
As indicated above, Geiger further teaches in section III A. Corner detection to produce a list of corner candidates, we apply conservative non-maxima-suppression (with parameters nnms and τnms) [18] on C, followed by verifying the candidates by their gradient statistics in a local n × n pixel neighborhood. Geiger section III C Structure recovery teaches Given a seed corner, we search for its closest neighbors in the direction of its edges e1 and e2, yielding an initial 2 2 checkerboard hypothesis with an associated energy value E(XY). To optimize E(XY), we propose expansion moves on Y, which expand any of the checkerboard borders by a single row or column. Amongst all four possibilities, we select the proposal, which reduces E(XY) the most. Fig. 3(a) illustrates the expansion moves exemplarily. The Examiner interprets that the input image of Figure 2(c) is a captured 2d image and performing mathematical processes like non-maxima suppression and gradient statistics requires a coordinate system. The Examiner further interprets that the checkerboards are three-dimensional objects and placed at an angle to the camera such that they are perceived as three-dimensional space as well. Additionally, the Examiner interprets that Figure 3 illustrates the recovery of the point-line and face of each checkerboard.
Geiger teaches in Figure 3C detecting multiple 2-D calibration patterns in one image. It is clear that each checkerboard pattern has its own face and therefore the image as a whole detects multiple faces. Those of ordinary skill would readily understand that the different calibration patterns could be placed on different faces of a singular object for the same results.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL E TEITELBAUM, Ph.D. whose telephone number is (571)270-5996. The examiner can normally be reached 8:30AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Miller can be reached at 571-272-7353. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL E TEITELBAUM, Ph.D./Primary Examiner, Art Unit 2422