Prosecution Insights
Last updated: November 10, 2026
Application No. 17/644,269

METHOD AND SYSTEM FOR EXTRINSIC CAMERA CALIBRATION

Non-Final OA §103
Filed
Dec 14, 2021
Priority
Jun 14, 2019 — CA 3046609 +1 more
Examiner
GOEBEL, EMMA ROSE
Art Unit
2662
Tech Center
2600 — Communications
Assignee
Hinge Health Inc.
OA Round
7 (Non-Final)
53%
Grant Probability
Moderate
7-8
OA Rounds
0m
Est. Remaining
87%
With Interview

Examiner Intelligence

Grants 53% of resolved cases
53%
Career Allowance Rate
37 granted / 70 resolved
-9.1% vs TC avg
Strong +34% interview lift
Without
With
+34.5%
Interview Lift
resolved cases with interview
Typical timeline
3y 0m
Avg Prosecution
21 currently pending
Career history
107
Total Applications
across all art units

Statute-Specific Performance

§101
18.7%
-21.3% vs TC avg
§103
59.5%
+19.5% vs TC avg
§102
11.4%
-28.6% vs TC avg
§112
8.8%
-31.2% vs TC avg
Black line = Tech Center average estimate • Based on career data from 70 resolved cases

Office Action

§103
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Continued Examination Under 37 CFR 1.114 A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on August 10, 2026 has been entered. Priority Acknowledgement is made of Applicant’s claim of priority from Foreign Application No. CA3046609 filed June 14, 2019 and as a Continuation of PCT Application No. PCT/IB2020/052938, filed March 27, 2020. Status of Claims Claims 1-3, 5-17, 19 and 21 are pending. Claims 4, 18 and 20 have been cancelled. Response to Arguments Applicant's arguments filed April 29, 2026 have been fully considered but they are not persuasive. Applicant argues that the rejection does not adequately articulate why a person having ordinary skill in the art would combine the proposed references. Specifically, because the references operate using materially different inputs, representations, architectures, and objectives, Applicant argues that a person having ordinary skill in the art would not select the teachings of the prior art references. Examiner respectfully disagrees. Applicant is reminded that obviousness can be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so (see MPEP 2143.01). As stated in MPEP 2145(X)(C), a teaching, suggestion, or motivation to combine references that is found in the prior art is an appropriate rationale for determining obviousness. The motivations to combine, as outlined in the rejection below, are found in the references themselves. Addressing the example provided in Applicant’s arguments of the Ramnath reference, the motivation to combine the Ramnath reference with the Mallet, Chang and Sun references is that adding the teachings of Ramnath would allow for verifying that a potential match is a best match, as described in the Ramnath reference itself. That is, a person having ordinary skill in the art would turn to the teaching of Ramnath used in the herein rejection because they desire to more accurately verify that the matching features between the sets of features are the best match. While it is true that if a proposed modification would render the prior art invention being modified unsatisfactory for its intended purpose, there may be no suggestion or motivation to make the proposed modification, Examiner asserts that the combination of references would not render any of the inventions unsatisfactory despite the differences among the references. Staying with the Ramnath example, using Ramnath’s teaching of a best match verification process with the match determination of Chang would not render the invention of Chang unusable, but rather would allow for verifying that the determined matching features are the best possible match. Thus, Examiner asserts that there is sufficient motivation to combine the references presented in the herein rejection. Applicant further argues that the asserted combination requires material changes to the cited systems and that the rejection is not sufficient because arriving at the claimed invention requires successive changes to the roles, inputs and operations of the cited systems. Specifically, Applicant argues against the separation of the "providing" step by relying on Sun for generating synthetic views and Wang for virtual-camera parameters associated with those views. However, as was previously discussed in the Non-Final Office Action mailed March 18, 2026, Sun teaches generating 3D synthetic views from a 2D image representation (2.5D, which is just 2D with depth measurements, so is sufficient to teach 2D) in combination with camera parameters to determine poses of the object relative to a camera (see Sun, Para. [0068]) (emphasis added). Although Sun does not describe the specific camera parameters used, Examiner asserts that a person skilled in the art would find it obvious to use the synthetic view generation of Wang to cure the deficiencies of Sun. Wang teaches that each synthetic view is associated with virtual camera parameters including a set of translation and rotation coordinates (see Wang, Para. [0065] and [0136]) and teaches determining camera extrinsic parameters including a translation component and rotation component (see Wang, Para. [0101] and [0079]). Nothing in the two references indicates that the teachings of the two cannot be combined, or that the combination would render either invention unsatisfactory for the intended purpose. Rather, one having ordinary skill in the art would be motivated to combine these references because doing so would allow for using virtual cameras to determine camera extrinsic parameters, as recognized by Wang. Applicant further argues that the Office separates the feature-matching and verification operations of claim 17 by relying on Chang for feature comparisons and Ramnath for verification and that the rejection does not adequately explain why a person having ordinary skill in the art would modify Chang using the teachings of Ramnath. The combination of these references is discussed in the above response to arguments. Examiner reiterates that Chang is relied upon to teach "finding correspondences by comparing each feature in the set of features with each feature in each set of the multiple sets of features of the multiple synthetic views" and "for each feature…identifying a best match….". Chang teaches each training image represents a different respective view of the reference image using computer simulation (i.e., the training images are multiple synthetic views). Then, a best match is found between a reference feature and candidate feature using a metric such as weighted average or probabilities (see Chang, Col. 4 line 54-Col. 5 line 14). Chang does not teach the specific cross-checking function once the best match is chosen. However, this process is taught by Ramnath who teaches passing potential matches through a verifier process that verifies the match as the best match (see Ramnath, Para. [0033]). Applicant argues that the combination appears to depend upon first knowing the claimed sequence and then selecting aspects of Chang and Ramnath to map to different portions of that sequence. However, Examiner asserts that the "best match" of Chang can be passed down the workflow to the Ramnath reference for best match verification, as described. This does not constitute mapping portions of Chang and Ramnath to different portions of the claim, but rather combining two references in a way that allows for an improvement upon the invention without rendering it unsatisfactory for its intended purpose. Additionally, as described in the above response to arguments, one having ordinary skill in the art would find it obvious to combine the Chang and Ramnath references because doing so would provide an additional "best match" verification step, as is taught by Ramnath. Thus, the references are sufficient to teach the claimed invention and the 35 USC 103 rejection of the claims is upheld. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. Claims 1-3, 5-6, 9-13 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Mallet et al. (US 2015/0288951 A1) in view of Chang et al. (US 9412164 B2) further in view of Sun et al. (WO 2018/190805 A1), Ramnath et al. (US 2020/01432328 A1, filed November 7, 2018), and Wang et al. (US 2019/0012804 A1). Regarding claim 1, Mallet teaches a method of determining extrinsic parameters of a camera, the method comprising: obtaining a two-dimensional (2D) digital representation of a calibration design (Mallet, Para. [0045], line 2, Mallet teaches a computer graphics model of calibration target 810. Para. [0046], the images may be captured on film, and then scanned into digital form. In digital form the pixel coordinates of an image point can be determined. Para. [0047], Each image captured for calibrating a camera may include a 2D array of pixels, and may be enumerated using pixel coordinates); obtaining a digital camera image of a physical representation of the calibration design (Mallet, Para. [0044], lines 11-12, Mallet teaches a camera captures images of the scene including images of a calibration target); identifying a set of features in the digital camera image (Mallet, Para. [0047], an identification procedure is performed on an image in order to identify parts of the image that correspond to particular fiducial markings), wherein each feature in the set of features is either (i) a portion of the digital camera image or (ii) a descriptor that is representative of information derived from the portion of the digital camera image (Mallet, Para. [0047], an identification procedure is performed on an image in order to identify parts of the image that correspond to particular fiducial markings. That is, the parts or segments of the image are identified for which a fiducial marking on the calibration target was the source in the physical scene). Although Mallet teaches a computer graphics model of calibration target and a processor that can compare information from the captured images to the computer graphics model (Mallet, Para. [0045]), Mallet does not explicitly teach “identifying the set of features in each synthetic view of the multiple synthetic views, so as to identify multiple sets of features, wherein because the multiple synthetic views depict the calibration design under the different camera projective transforms, at least one feature in the set of features is different in each of the multiple synthetic views”, “finding correspondences by – comparing each feature in the set of features of the digital camera image with each feature in each set of the multiple sets of features of the multiple synthetic views” and “for each feature in the set of features of the digital camera image, identifying a best match by selecting, from among the multiple sets of features of the multiple synthetic views, a first feature that is determined to have a highest similarity and that is in a given synthetic view of the multiple synthetic views”. However, in an analogous field of endeavor, Chang teaches that each training image represents a different respective view of the reference image. The different views may be generated by capturing training images of the reference image from different perspectives or by simulating the different views of the reference image using a computer simulation process. The image processing system matches features in the reference and candidate sets of the image features by determining at least one metric that characterizes the matching image features in the reference and candidate sets (e.g., weighted average or estimated probabilities) and then selects ones of the image features in the reference set from which the calibration-enabling data is derived based on the determined metric (i.e., highest similarity) (Chang, Col. 4 line 54-Col. 5 line 14). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet with the teachings of Chang by including that each synthetic image represents a different respective view and generating a candidate set of image features for the synthetic image and determining a best match between the synthetic image and the digital camera image by selecting the image feature based on the highest similarity. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for calibrating an imaging system based on matched image features, as recognized by Chang. Although Mallet in view of Chang teaches a computer graphics model of calibration target and a processor that can compare information from the captured images to the computer graphics model (Mallet, Para. [0045]), they do not explicitly teach “providing the 2D digital representation of the calibration design as input, to a three-dimensional (3D) rendering framework that produces, as output, multiple synthetic views that depict the calibration design under different camera projective transforms that are calculated based on a plurality of virtual camera parameters”. However, in an analogous field of endeavor, Sun teaches a 3D CAD model or 3D data is used to render or generate synthetic orthographic views from any potential viewpoint a user or operator may look at the object in a real scene. The strategy for creating synthetic views may be random or may be based on planned sampling the 3D space (Sun, Para. [0075]). The depth measurements in combination with the camera parameters allows the 2.5D image representation to be converted to a 3 point cloud (Sun, Para. [0068]). The image processor is configured to determine a pose of the object relative to the camera and depth sensor based on poses of the object in the 3D orthographic projections. Where a best match is found, the pose of the orthographic projection or representation derived from the 3D data is determined as the pose of the object relative to the camera and depth sensor. By using image representations derived from the orthographic projections, the comparison of the orthographic projections may be more rapidly performed (Sun, Para. [0093]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang with the teachings of Sun by including using a 3D CAD model or 3D data to render multiple synthetic orthographic views from any potential viewpoint a user or operator may look at the object in a real scene of the digital representation of the calibration design and the view is used to determine the pose of the object relative to the camera (i.e., the calibration design under different camera projective transforms). One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for determining the extrinsic camera parameters based on the plurality of generated synthetic views, as recognized by Sun. Although Mallet in view of Chang further in view of Sun teaches determining matching features from a reference and candidate set based on the highest similarity (Chang, Col. 4 line 63-Col. 5 line 14), they do not explicitly teach “comparing the first feature against each feature in the set of features in the digital camera image, so as to identify a second feature in the digital camera image that has a highest similarity as a cross-check feature” and “accepting the first feature as the best match in response to a determination that the second feature is the same as that feature for which the first feature was identified as the best match”. However, in an analogous field of endeavor, Ramnath teaches passing potential matches through a verifier process that compares the local-feature descriptors of the image against the local-feature descriptors of the one or more potential matches. If there were only one potential match, the verifier process may verify that the potential match is a matching target (Ramnath, Para. [0033]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang further in view of Sun with the teachings of Ramnath by including a verifier process that compares the potential best match feature to the to the features in the image and verify that the potential match is a matching target based on the comparison. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for verifying that a potential match is the best match, as recognized by Ramnath. Although Mallet in view of Chang further in view of Sun and Ramnath teaches determining, among other parameters, the position, rotation, distortion and focal length of the camera with respect to the calibration target (Mallet, Para. [0045]), they do not explicitly teach that for the synthetic views, “wherein each of the synthetic views is associated with a different one of the plurality of virtual camera parameters that comprises a set of translation and rotation coordinates that is representative of a corresponding camera projective transform that indicates placement of a corresponding one of multiple virtual cameras relative to the 2D digital representation of the calibration design” and “calculating, based on two-dimensional coordinates and three-dimensional coordinates of the features associated with the best matches, the extrinsic parameters of the camera by establishing (i) a translation component that is represented as a 3-vector with separate x, y, and z values in 3D space and (ii) a rotation component that is represented as a vector of Euler angles”. However, in an analogous field of endeavor, Wang teaches a virtual camera may be defined by virtual camera parameters which represent the configuration of the virtual camera required in order to have captured the second image. For example, each virtual camera has, among other virtual camera parameters, a position and orientation which can be determined (Wang, Para. [0065]). K, R, and t are the camera intrinsic (K) and extrinsic (R, t) parameters, respectively, of each virtual camera. Extrinsic (R, t) parameters of each virtual camera. R | t = r 11 r 12 r 13 t 1 r 21 r 22 r 23 t 2 r 31 r 32 r 33 t 3 (i.e., translation component is a 3-vector with x, y, and z values in 3D space) (Wang, Para. [0136]). The 2D coordinates of a projection p are (u,v) (Wang, Para. [0138]). Wang further teaches positions and orientations (i.e., extrinsic parameters) of the plurality of multi-directional image capture apparatuses (i.e., camera) may be determined based on the positions and orientations of the virtual cameras (Wang, Para. [0101]). If there are twelve virtual cameras (six from each panoramic image of the stereo-pair of panoramic images) corresponding to the multi-directional image capture apparatus, then twelve rotation matrices are obtained for the orientation of the multi-directional image capture apparatus. Each of these rotation matrices may then be converted into corresponding Euler angles to obtain a set of Euler angles for the multi-directional image capture apparatus (i.e., a rotation component that is represented as a vector of Euler angles) (Wang, Para. [0079]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang further in view of Sun and Ramnath with the teachings of Wang by including for each synthetic view of the virtual camera, intrinsic and extrinsic parameters included a translation component that is represented as a 3-vector with separate x, y, and z values in 3D space and a rotation component that is represented as a vector of Euler angles and determining the camera extrinsic parameters based on the extrinsic parameters of the virtual camera. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for determining camera extrinsic parameters based on virtual cameras, as recognized by Wang. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date. Regarding claim 2, Mallet in view of Chang further in view of Sun, Ramnath and Wang teaches the method of claim 1, and further teaches wherein the calibration design is asymmetric in at least one dimension (Mallet, Figs. 2-5, calibration targets 200, 300, 400, and 500, Mallet teaches calibration targets that have unique and asymmetrical fiducial marker patterns). Regarding claim 3, Mallet in view Chang further in view of Sun, Ramnath and Wang teaches the method of claim 2, and further teaches wherein the calibration design is a logo (Mallet, Para. [0025], Mallet teaches that calibrating a camera involves using a test object, or a calibration target, that can contain marks known as fiducial markings). As indicated by your specification (Para. [0013]: “the planar calibration pattern 30 may be a logo, background image or other design that may already normally appear in the camera’s 10 field of view”), the indication of the calibration image as a logo adds no special characteristics or unique improvements to the method, and therefore could suitably be any test object or calibration target, as taught by Mallet. Regarding claim 5, Mallet in view Chang further in view of Sun, Ramnath and Wang teaches the method of claim 1, and further teaches wherein the multiple synthetic views are selected from a space of virtual camera parameters where the calibration image is within a field of view of the given synthetic view (Wang, Para. [0065], a virtual camera may be defined by virtual camera parameters which represent the configuration of the virtual camera required in order to have captured the second image). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, Ramnath, and Wang references presented in the rejection of Claim 1, apply to Claim 5 and are incorporated herein by reference. Thus, the method recited in Claim 5 is met by Mallet in view of Chang further in view of Sun, Ramnath, and Wang. Regarding claim 6, Mallet in view Chang further in view of Sun, Ramnath and Wang teaches the method of claim 1, and further teaches wherein identifying a set of features from each of the plurality of synthetic views and identifying the set of features in the digital camera image is performed using a feature detection module (Para. [0045], lines 7-10, Mallet teaches that without receiving input from a user, processor 804 can compare information from the captured images to the computer graphics model. The processor taught by Mallet can be a feature detection module). Regarding claim 9, Mallet teaches a camera calibration module for determining the translation and rotation of a camera using a physical, planar calibration pattern, the camera calibration module comprising: a feature detector for extracting a set of features from an image captured from the camera (Mallet, Para. [0047], an identification procedure is performed on an image in order to identify parts of the image that correspond to particular fiducial markings), wherein each feature in the set of features is either (i) a predetermined portion of the image or one of the plurality of synthetic views or (ii) a descriptor that is representative of information derived from the predetermined portion (Mallet, Para. [0047], an identification procedure is performed on an image in order to identify parts of the image that correspond to particular fiducial markings. That is, the parts or segments of the image are identified for which a fiducial marking on the calibration target was the source in the physical scene); Although Mallet teaches a computer graphics model of calibration target and a processor that can compare information from the captured images to the computer graphics model (Mallet, Para. [0045]), Mallet does not explicitly teach extracting a set of features from “each of the plurality of synthetic views”, “a feature matching module for finding correspondences by comparing each feature in the set of features of the image with each feature in each set of features of each synthetic view of the plurality of synthetic views” and “for each feature in the set of features of the image, identifying a best match by selecting, from among the set of features of the plurality of synthetic views, a first feature that is determined to have a highest similarity and that is in a given synthetic view of the plurality of synthetic views”. However, in an analogous field of endeavor, Chang teaches the image processing system matches features in the reference and candidate sets of the image features by determining at least one metric that characterizes the matching image features in the reference and candidate sets (e.g., weighted average or estimated probabilities) and then selects ones of the image features in the reference set from which the calibration-enabling data is derived based on the determined metric (i.e., highest similarity) (Chang, Col. 4 line 63-Col. 5 line 14). The proposed combination as well as the motivation for combining the Mallet and Chang references presented in the rejection of Claim 1, apply to Claim 9 and are incorporated herein by reference. Although Mallet in view of Chang teaches a computer graphics model of calibration target and a processor that can compare information from the captured images to the computer graphics model (Mallet, Para. [0045]), they do not explicitly teach “a synthetic pattern generator for generating a plurality of synthetic views of a digital calibration image corresponding to the physical calibration pattern”. However, in an analogous field of endeavor, Sun teaches a 3D CAD model or 3D data is used to render or generate synthetic orthographic views from any potential viewpoint a user or operator may look at the object in a real scene. The strategy for creating synthetic views may be random or may be based on planned sampling the 3D space (Sun, Para. [0075]). The proposed combination as well as the motivation for combining the Mallet, Chang and Sun references presented in the rejection of Claim 1, apply to Claim 9 and are incorporated herein by reference. Although Mallet in view of Chang further in view of Sun teaches determining matching features from a reference and candidate set based on the highest similarity (Chang, Col. 4 line 63-Col. 5 line 14), they do not explicitly teach “comparing the first feature against each feature in the set of features in the image, so as to identify a second feature in the image that has a highest similarity as a cross-check feature” and “accepting the first feature as the best match in response to a determination that the second feature is the same as that feature for which the first feature was identified as the best match”. However, in an analogous field of endeavor, Ramnath teaches passing potential matches through a verifier process that compares the local-feature descriptors of the image against the local-feature descriptors of the one or more potential matches. If there were only one potential match, the verifier process may verify that the potential match is a matching target (Ramnath, Para. [0033]). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, and Ramnath references presented in the rejection of Claim 1, apply to Claim 9 and are incorporated herein by reference. Although Mallet in view of Chang further in view of Sun and Ramnath teaches determining, among other parameters, the position, rotation, distortion and focal length of the camera with respect to the calibration target (Mallet, Para. [0045]), they do not explicitly teach “wherein each synthetic view of the plurality of synthetic views is associated with a different one of a plurality of virtual camera parameters comprising a set of translation and rotation coordinates” and “a calibration solver for calculating the translation and the rotation of the camera using two-dimensional and three-dimensional coordinates of the features associated with the best matches” and “wherein the translation of the camera is represented as a 3-vector with separate x, y, and z values in 3D space, and wherein the rotation of the camera that is represented as (i) a vector of Euler angles, (ii) a 3x3 rotation matrix, or (iii) an angle-axis vector”. However, in an analogous field of endeavor, Wang teaches a virtual camera may be defined by virtual camera parameters which represent the configuration of the virtual camera required in order to have captured the second image. For example, each virtual camera has, among other virtual camera parameters, a position and orientation which can be determined (Wang, Para. [0065]). K, R, and t are the camera intrinsic (K) and extrinsic (R, t) parameters, respectively, of each virtual camera. Extrinsic (R, t) parameters of each virtual camera. R | t = r 11 r 12 r 13 t 1 r 21 r 22 r 23 t 2 r 31 r 32 r 33 t 3 (i.e., translation component is a 3-vector with x, y, and z values in 3D space, rotation of the camera represented by a 3x3 rotation matrix) (Wang, Para. [0136]). The 2D coordinates of a projection p are (u,v) (Wang, Para. [0138]). Wang further teaches positions and orientations (i.e., extrinsic parameters) of the plurality of multi-directional image capture apparatuses (i.e., camera) may be determined based on the positions and orientations of the virtual cameras (Wang, Para. [0101]). If there are twelve virtual cameras (six from each panoramic image of the stereo-pair of panoramic images) corresponding to the multi-directional image capture apparatus, then twelve rotation matrices are obtained for the orientation of the multi-directional image capture apparatus. Each of these rotation matrices may then be converted into corresponding Euler angles to obtain a set of Euler angles for the multi-directional image capture apparatus (i.e., a rotation component that is represented as a vector of Euler angles) (Wang, Para. [0079]). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, Ramnath and Wang references presented in the rejection of Claim 1, apply to Claim 9 and are incorporated herein by reference. Thus, the camera calibration module recited in claim 9 is met by Mallet in view of Chang further in view of Sun, Ramnath, and Wang. Regarding claim 10, Mallet in view of Chang further in view of Sun, Ramnath, and Wang teaches the camera calibration module of claim 9, and further teaches wherein the digital calibration image is asymmetric in at least one dimension (Mallet, Figs. 2-5, calibration targets 200, 300, 400, and 500, Mallet teaches calibration targets that have unique and asymmetrical fiducial marker patterns). Regarding claim 11, Mallet in view of Chang further in view of Sun, Ramnath, and Wang teaches the camera calibration module of claim 10, and further teaches wherein each synthetic view of the plurality of synthetic views is associated with the set of translation and rotation coordinates that indicates placement of a corresponding one of a plurality of virtual cameras relative to the digital calibration image (Wang, Para. [0065], a virtual camera may be defined by virtual camera parameters which represent the configuration of the virtual camera required in order to have captured the second image. For example, each virtual camera has, among other virtual camera parameters, a position and orientation which can be determined. Para. [0136], K, R, and t are the camera intrinsic (K) and extrinsic (R, t) parameters, respectively, of each virtual camera). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, Ramnath and Wang references presented in the rejection of Claim 1, apply to Claim 11 and are incorporated herein by reference. Thus, the camera calibration module recited in claim 11 is met by Mallet in view of Chang further in view of Sun, Ramnath, and Wang. Regarding claim 12, Mallet in view of Chang further in view of Sun, Ramnath, and Wang teaches the camera calibration module of claim 9, and further teaches wherein a plurality of extrinsic parameters comprise translation and rotation coordinates (Wang, Para. [0065], a virtual camera may be defined by virtual camera parameters which represent the configuration of the virtual camera required in order to have captured the second image. For example, each virtual camera has, among other virtual camera parameters, a position and orientation which can be determined. Para. [0136], K, R, and t are the camera intrinsic (K) and extrinsic (R, t) parameters, respectively, of each virtual camera). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, Ramnath and Wang references presented in the rejection of Claim 1, apply to Claim 12 and are incorporated herein by reference. Thus, the camera calibration module recited in claim 12 is met by Mallet in view of Chang further in view of Sun, Ramnath, and Wang. Regarding claim 13, Mallet in view of Chang further in view of Sun, Ramnath, and Wang teaches the camera calibration module of claim 9, and further teaches wherein the plurality of synthetic views are selected from a space of virtual camera parameters where the digital calibration image is within a field of view of the given synthetic view (Wang, Para. [0065], a virtual camera may be defined by virtual camera parameters which represent the configuration of the virtual camera required in order to have captured the second image). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, Ramnath and Wang references presented in the rejection of Claim 1, apply to Claim 13 and are incorporated herein by reference. Thus, the camera calibration module recited in claim 13 is met by Mallet in view of Chang further in view of Sun, Ramnath, and Wang. Regarding claim 16, Mallet in view of Chang further in view of Sun, Ramnath, and Wang teaches a camera calibration system comprising: the camera calibration module of claim 9 (as described above); the camera (Para. [0044], line 4, Mallet teaches a physical camera 802); and the physical planar calibration pattern (Para. [0044], line 10, Mallet teaches a calibration target 810); wherein output from the camera is embedded with the translation and the rotation of the camera (Para. [0010], lines 8-9, Mallet teaches using the processor to determine the position, rotation, and focal length for the physical camera). Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Mallet (US 2015/0288951 A1) in view of Chang (US 9412164 B2) further in view of Sun (WO 2018/190805 A1), Ramnath (US 2020/01432328 A1, filed November 7, 2018) and Wang et al. (US 2019/0012804 A1), as applied to claims 1-6, 9-13, and 16 above, and further in view of Gossow (US 2017/0098395 A1). Regarding claim 7, Mallet in view of Chang further in view of Sun, Ramnath and Wang teaches the method of claim 1, as described above. Although Mallet in view of Chang further in view of Sun, Ramnath and Wang teaches calculating the extrinsic parameters of the camera using the virtual camera parameters of the features associated with the best matches (Mallet, Para. [0045]), they do not explicitly teach “wherein finding the correspondences further comprises computing an elementwise difference between each feature and minimizing this difference for both the given synthetic view to the digital camera image and the digital camera image to the given synthetic view”. However, in an analogous field of endeavor, Gossow teaches identifying best matches by computing the elementwise difference between each feature (Para. [0039] lines 6-7, Gossow teaches a difference between intensities of the pixels in the synthetic images and intensities of pixels in the actual images) and minimizing this difference for both the synthetic view to captured image and captured image to synthetic view (Para. [0039] lines 5-7, Gossow teaches a calibration device that determines the value of the parameter vector that minimizes a difference between intensities of the pixels in the synthetic images and intensities of pixels in the actual images). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the method of Mallet in view of Chang further in view of Sun, Ramnath and Wang with the teachings of Gossow by including identifying best matches by computing the elementwise difference between each feature and minimizing the difference for both the synthetic view to captured image and captured image to synthetic view. One having ordinary skill in the art would be motivated to combine the references since doing so would improve the speed and accuracy of camera calibration and extrinsic parameter determination, as recognized by Gossow. Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Claim 14 recites a camera calibration module with elements corresponding to the steps recited in Claim 7. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Mallet, Chang, Sun, Ramnath, Wang and Gossow references, presented in rejection of Claim 7, apply to this claim. Claims 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Mallet (US 2015/0288951 A1) in view of Chang (US 9412164 B2) further in view of Sun (WO 2018/190805 A1), Ramnath (US 2020/01432328 A1, filed November 7, 2018) and Wang et al. (US 2019/0012804 A1), as applied to claims 1-6, 9-13, 16, and 20 above, and further in view of Hong (US 8861864 B2). Regarding claim 8, Mallet in view of Chang further in view of Sun, Ramnath and Wang teaches the method of claim 1, as described above. Although Mallet in view of Chang further in view of Sun, Ramnath and Wang teaches calculating the extrinsic parameters of the camera using the virtual camera parameters of the features associated with the best matches (Mallet, Para. [0045]), they do not explicitly teach “identifying a region of interest of the digital camera image wherein the identifying the set of features in the digital camera image is performed only on the region of interest”. However, in an analogous field of endeavor, Hong teaches identifying a region of interest of the digital camera image (Hong, Col. 1, line 59, Hong teaches a user selecting a desired region of interest), and wherein identifying the set of features in the digital camera image is performed only on the region of interest (Hong, Col. 1, lines 59-61, Hong teaches that after a user selects a desired region of interest, a feature detector, such as SIFT, is applied to the region of interest). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang further in view of Sun, Ramnath, and Wang with the teachings of Hong by including identifying a region of interest of the digital camera image and perform feature identification on this region. One having ordinary skill in the art would be motivated to combine the references since doing so would improve the speed and accuracy of camera calibration and extrinsic parameter determination. Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Claim 15 recites a camera calibration module with elements corresponding to the steps recited in Claim 8. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Mallet, Chang, Sun, Ramnath, Wang and Hong references, presented in rejection of Claim 8, apply to this claim. Claims 17, 19 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Mallet et al. (US 2015/0288951 A1) in view of Chang et al. (US 9412164 B2) further in view of Sun et al. (WO 2018/190805 A1), Ramnath et al. (US 2020/01432328 A1, filed November 7, 2018), Wang et al. (US 2019/0012804 A1) and Shen et al. (US 2018/0260668 A1). Regarding claim 17, Mallet teaches a method comprising: obtaining a graphic pattern in digital form (Mallet, Para. [0045], a computer graphics model of calibration target 810 including the positions and unique patterns of the fiducial markings on the target); obtaining an image of a physical representation of the graphic pattern that is generated by a camera (Para. [0044], lines 11-12, Mallet teaches a camera captures images of the scene including images of a calibration target); identifying the set of features in the image (Para. [0045], lines 8-9, Mallet teaches a processor that can compare information from the captured images). Although Mallet teaches a computer graphics model of calibration target and a processor that can compare information from the captured images to the computer graphics model (Mallet, Para. [0045]), Mallet does not explicitly “identifying a set of features in each synthetic view of the multiple synthetic views, so as to identify multiple sets of features” and “for each feature in the set of features of the digital camera image, identifying a best match by selecting, from among the multiple sets of features identified in the multiple synthetic views, a given first feature that is determined to be most similar to that feature and that is in a given one of the multiple synthetic views”. However, in an analogous field of endeavor, Chang teaches the image processing system determines a reference set of image features from the electronic data file which specifies the reference image in a reference coordinate space (Chang, Col. 3, lines 38-44) and that the image processing system matches features in the reference and candidate sets of the image features by determining at least one metric that characterizes the matching image features in the reference and candidate sets (e.g., weighted average or estimated probabilities) and then selects ones of the image features in the reference set from which the calibration-enabling data is derived based on the determined metric (i.e., highest similarity) (Chang, Col. 4 line 63-Col. 5 line 14). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet with the teachings of Chang by including determining set of features in the synthetic view for each synthetic view and determining a best match between the synthetic image and the digital camera image by selecting the image feature based on the highest similarity. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for calibrating an imaging system based on matched image features, as recognized by Chang. Although Mallet in view of Chang teaches a computer graphics model of calibration target and a processor that can compare information from the captured images to the computer graphics model (Mallet, Para. [0045]), they do not explicitly teach “providing the graphic pattern, as input, to a three-dimensional (3D) rendering framework that renders the graphic pattern into multiple synthetic views, wherein each synthetic view is representative of an image that depicts the graphic pattern under a projective transform that is calculable from a corresponding virtual camera parameter set (Sun, Para. [0075]). The depth measurements in combination with the camera parameters allows the 2.5D image representation to be converted to a 3 point cloud (Sun, Para. [0068]). The image processor is configured to determine a pose of the object relative to the camera and depth sensor based on poses of the object in the 3D orthographic projections. Where a best match is found, the pose of the orthographic projection or representation derived from the 3D data is determined as the pose of the object relative to the camera and depth sensor. By using image representations derived from the orthographic projections, the comparison of the orthographic projections may be more rapidly performed (Sun, Para. [0093]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang with the teachings of Sun by including using a 3D CAD model or 3D data to render multiple synthetic orthographic views from any potential viewpoint a user or operator may look at the object in a real scene. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for determining the extrinsic camera parameters based on the plurality of generated synthetic views, as recognized by Sun. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date. Although Mallet in view of Chang further in view of Sun teaches determining matching features from a reference and candidate set based on the highest similarity (Chang, Col. 4 line 63-Col. 5 line 14), they do not explicitly teach “comparing the first feature against each feature in the set of features in the digital camera image, so as to identify a second feature in the digital camera image that is determined to be the most similar to the first feature” and “accepting the first feature as the best match in response to a determination that the second feature is the same as that feature for which the first feature was identified as the best match”. However, in an analogous field of endeavor, Ramnath teaches passing potential matches through a verifier process that compares the local-feature descriptors of the image against the local-feature descriptors of the one or more potential matches. If there were only one potential match, the verifier process may verify that the potential match is a matching target (Ramnath, Para. [0033]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang further in view of Sun with the teachings of Ramnath by including a verifier process that compares the potential best match feature to the to the features in the image and verify that the potential match is a matching target based on the comparison. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for verifying that a potential match is the best match, as recognized by Ramnath. Although Mallet in view of Chang further in view of Sun and Ramnath teaches determining, among other parameters, the position, rotation, distortion and focal length of the camera with respect to the calibration target (Mallet, Para. [0045]), they do not explicitly teach “a corresponding set of translation and rotation coordinates, which is indicative of placement of a corresponding virtual camera with respect to the graphic pattern” and “calculating a translation and a rotation component of the camera based on two-dimensional coordinates and three-dimensional coordinates of the features identified in the image”. However, in an analogous field of endeavor, Wang teaches a virtual camera may be defined by virtual camera parameters which represent the configuration of the virtual camera required in order to have captured the second image. For example, each virtual camera has, among other virtual camera parameters, a position and orientation which can be determined (Wang, Para. [0065]). K, R, and t are the camera intrinsic (K) and extrinsic (R, t) parameters, respectively, of each virtual camera (Wang, Para. [0136]). The 2D coordinates of a projection p are (u,v) (Wang, Para. [0138]). Wang further teaches positions and orientations (i.e., extrinsic parameters) of the plurality of multi-directional image capture apparatuses (i.e., camera) may be determined based on the positions and orientations of the virtual cameras (Wang, Para. [0101]). If there are twelve virtual cameras (six from each panoramic image of the stereo-pair of panoramic images) corresponding to the multi-directional image capture apparatus, then twelve rotation matrices are obtained for the orientation of the multi-directional image capture apparatus. Each of these rotation matrices may then be converted into corresponding Euler angles to obtain a set of Euler angles for the multi-directional image capture apparatus (i.e., a rotation component that is represented as a vector of Euler angles) (Wang, Para. [0079]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang further in view of Sun and Ramnath with the teachings of Wang by including for each synthetic view of the virtual camera, intrinsic and extrinsic parameters including translation and rotation coordinates and determining the camera extrinsic parameters based on the extrinsic parameters of the virtual camera. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for determining camera extrinsic parameters based on virtual cameras, as recognized by Wang. Although Mallet in view of Chang further in view of Sun, Ramnath and Wang teaches a patch associated with a point of interest may be used to generate a local-feature descriptor (e.g., using a Scale Invariant Feature Transform (SIFT), using Speed Up Robust Feature (SURF)) that may function as a representation of the patch (Ramnath, Para. [0028]), they do not explicitly teach “wherein each feature in the set of features is representative of a patch of a corresponding one of the multiple synthetic views that is accompanied by a descriptor, the descriptor being representative of an encoding of information related to the patch in a lower-level dimensional space”. However, in an analogous field of endeavor, Shen teaches an encoder that encodes the content, or feature vectors, of the input image into a low-dimensional space (Shen, Para. [0073]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Mallet in view of Chang further in view of Sun, Ramnath and Wang with the teachings of Shen by including encoding the local-feature descriptor representative of a patch as taught by Ramnath using the encoder of Shen that encodes the feature vector into a low-dimensional space. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for encoding feature representations that capture image contexts, as recognized by Shen. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date. Regarding claim 19, Mallet in view of Chang further in view of Sun, Ramnath, Wang and Shen teaches the method of claim 17, and further teaches wherein the translation component is represented as a vector with x-, y-, and z-values, and wherein the rotation component is represented as a vector of Euler angles, an angle-axis vector, or a 3x3 matrix (Wang, Para. [0136], extrinsic (R, t) parameters of each virtual camera. R | t = r 11 r 12 r 13 t 1 r 21 r 22 r 23 t 2 r 31 r 32 r 33 t 3 ). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, Ramnath, Wang and Shen references presented in the rejection of Claim 17, apply to Claim 19 and are incorporated herein by reference. Thus, the method recited in claim 19 is met by Mallet in view of Chang further in view of Sun, Ramnath, Wang and Shen. Regarding claim 21, Mallet in view of Chang further in view of Sun, Ramnath, Wang and Shen teaches the method of claim 17, wherein to identify the set of features in each synthetic view of the multiple synthetic views, a Speeded Up Robust Features (SURF) algorithm or a Maximally Stable Extremal Regions (MSER) algorithm is executed that identifies the set of features in a fashion invariant to scaling and rotation (Ramnath, Para. [0028], a patch associated with a point of interest may be used to generate a local-feature descriptor (e.g., using a Scale Invariant Feature Transform (SIFT), using Speed Up Robust Feature (SURF)) that may function as a representation of the patch. The local-feature descriptor may include location information (e.g., an (x, y) coordinate) for the point of interest within the image). The proposed combination as well as the motivation for combining the Mallet, Chang, Sun, Ramnath, Wang and Shen references presented in the rejection of Claim 17, apply to Claim 21 and are incorporated herein by reference. Thus, the method recited in claim 21 is met by Mallet in view of Chang further in view of Sun, Ramnath, Wang and Shen. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Emma Rose Goebel/Examiner, Art Unit 2662 /AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Read full office action

Prosecution Timeline

Show 13 earlier events
Apr 20, 2026
Examiner Interview Summary
Apr 29, 2026
Response Filed
Jun 08, 2026
Final Rejection mailed — §103
Aug 05, 2026
Applicant Interview (Telephonic)
Aug 05, 2026
Examiner Interview Summary
Aug 10, 2026
Request for Continued Examination
Aug 12, 2026
Response after Non-Final Action
Oct 19, 2026
Non-Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12784412
MODEL DESIGN AND EXECUTION FOR EMPTY SHELF DETECTION
3y 8m to grant Granted Nov 03, 2026
Patent 12784139
CONTEXTUAL MEDIA GENERATION
3y 8m to grant Granted Nov 03, 2026
Patent 12737860
IMAGE PROCESSING FOR EXPOSURE BRACKETING
3y 0m to grant Granted Sep 15, 2026
Patent 12711574
METHOD AND APPARATUS WITH SUPERSAMPLING
3y 4m to grant Granted Aug 18, 2026
Patent 12694646
STATE DETECTION APPARATUS
3y 2m to grant Granted Jul 28, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

7-8
Expected OA Rounds
53%
Grant Probability
87%
With Interview (+34.5%)
3y 0m (~0m remaining)
Median Time to Grant
High
PTA Risk
Based on 70 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month