Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/26/26 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
The examiner notes the independent claims have been amended to include the following:
PNG
media_image1.png
191
591
media_image1.png
Greyscale
The examiner notes when performing camera calibration there are a variety of algorithms which may be used, such as Bundle Adjustment, which is disclosed by prior art, Bleyer (20200111232) used in the rejection, (see para 8, 32, 104, 105), which is a technique to align the virtual camera with the physical camera. See cited US 20180074519 (para 69) for varies camera calibration algorithms.
Additionally other known algorithms such as Zhang’s Algorithm which uses calibration targets (checkerboards) to calibrate the camera.
Another method/algorithm is SLAM which tracks the headsets position while building a map for calibration
The examiner has updated the rejection below by incorporating Chaurasia et al., US 11,451,758 which discloses using the SLAM algorithm to enhance a VR view (col 14, line 36 to col 15, line 24) which uses the pose (position and orientation) of the camara (circuits) to the virtual camera using the pixel relationships of the images. As disclosed by Chaurasia this allows the user to view the physical surrounding in addition to the virtual content.
In the event the applicant deems unexpected results are derived/obtained by the use of a conventional algorithm “SLAM” the examiner notes IAW MPEP 716.02, where applicant can clarify/explain such. It is noted that the only mention of SLAM in the disclosure filed (para 58), which appears to repeat what is claimed.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6-8, 13-15 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bleyer et al., US 20200111232 in view of DOS SANTOS MENDONCA et al., US 20220292718 and Grossman, US 20130182083 and Lahr et al., US 10,775,881 in view of Chaurasia et al., US 11,451,758.
PNG
media_image2.png
681
616
media_image2.png
Greyscale
PNG
media_image3.png
134
609
media_image3.png
Greyscale
The claimed a plurality of characteristic patterns is met by calibration marker images (Fig 14, 15, para 8, 25-27, 32, 34, 66-83, 85-89, 94-95, 98, 101-104)
The claimed plurality of camera circuits…is met by the HMD camera system using the one or more cameras (abstract, 007-008, 30, 32, 39, 57, 70, 72, 76, 78, 82, 96-97, 100-101, 104-105, 108-110, 114-115), where the HMD may render a virtual object for alignment (para 57, 73, 87. 93, 95, 99). Bleyer disclose that the calibration may be performed using a display screen or projected and reflected through any number of mirror or other reflective surfaces (para 85).
The claimed processor is met where the computer system, including processor 2005 (para 107-11, 116, 120-121) may be in the form of HMD, to calibrate the HMD cameras including intrinsic and/or extrinsic parameters (abstract, para 007-8, 30-32, 104-105)
Bleyer discloses the “markers” used for calibration appear on the display of the HMD (para 34, 69, 78) and display screen or other reflective surface.
Regarding the physical pattern on the external of the device, the examiner incorporates DOS SANTOS MENDONCA et al., US 20220292718.
DOS SANTOS MENDONCA et al., US 20220292718 discloses the physical appearance on the outside of the eyewear including a fiducial to perform calibration of the device, where (par 50) the features of includes an appearance (physical pattern) fiducial (112L/R) which maybe infrared, visible, where they may be etched onto the frame, attached including adhesive (sticker).
The motivation to modify Bleyer with DO would ensure the calibration of the cameras of the HMD, using fiducials installed on the device which allows calibration of the device without requiring a calibration plate/system, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention to quickly perform calibration.
Regarding FOV: it is noted above the combination does not explicitly recite the FOV of the camera is outside the FOV of the patterns thus requiring the reflective surface (mirror as disclosed).
The examiner incorporates Grossman, US 20130182083 (Fig 4, para 69) where the camera which has a FOV outside of the display, however with the reflection of the display via mirror (4), the camera can be calibrated.
PNG
media_image4.png
369
480
media_image4.png
Greyscale
Thus even if the FOV of the camera is outside the image/pattern which is needed to calibrate the camera, a mirror can be placed in the path of the camera and thus view the pattern/image as noted above in order to calibrate the camera.
The motivation to modify the above combination allows the prior art to place the cameras and calibration pattern in any FOV and ensures the capturing of such pattern using the reflective property of a mirror as taught by Grossman, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
The examiner also further provides additional evidence regarding the placement of the patterns/fiducials which may be on the side or atop the helmet/head-up display as shown by Fig 3 (sensors 308, 310) of Lahr et al., US 10,775,881, this reference is being to show the placement of the fiducials/marking can be anywhere.
The motivation to modify the above combination with Lahr allows the system to ensure proper calibration using a mirror/reflection (Grossman) to ensure the fiducials/markings can be used for calibration as would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding the newly claimed SLAM features:
The examiner notes the combination doesn’t disclose the use of the well-known algorithm, which is a known algorithm to calibrate/reconfigure a camera(s) in it’s environment.
The examiner incorporate Chaurasia et al., US 11,451,758 which discloses using the SLAM algorithm to enhance a VR view (col 14, line 36 to col 15, line 24) which uses the pose (position and orientation) of the camara (circuits) to the virtual camera using the pixel relationships of the images. As disclosed by Chaurasia this allows the user to view the physical surrounding in addition to the virtual content.
The motivation to modify the above combination with Chaurasia provides the advantages as noted above, notably to allow the user to view the physical as well as the virtual environment by ensuring the virtual images and cameras relationship are localized/know to provide a physical surrounding display when needed, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 6,
Bleyer disclose a storage 202 (Fig 20) which may be included into the HMD, including a database (para 89, 97, 109-113, 118, 120).
In considering claim 7,
Bleyer does not explicitly recite match each of the plurality of virtual images to one of the patterns, however Bleyer does disclose calibration of the one or more cameras based upon the virtual image with respect to the plurality of marker patterns.
The incorporated DOS SANTOS MENDONCA discloses a system which compares the reference image and the field image to perform calibration (para 16-18, 30, 33, 39, 71 and 76), which performs calibration extrinsic/intrinsic base on the change of appearance of the fiducial markers between the reference image and field images (para 42-43).
Thus it would have been obvious to one of ordinary skill in the art to modify Bleyer with DOS SANTOS MENDONCA to ensure calibration by matching the images to the patterns (fiducials) to ensure proper calibration of the device, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 8,
Refer to claim 1.
In considering claim 13,
Refer to claim 6.
In considering claim 14,
Refer to claim 7.
In considering claim 15,
Refer to claim 1, where the system can be carried out by computer executable instructions via GP computer (para 117-120).
In considering claim 20,
The claimed storing…refer to claim 13.
The claimed matching…refer to claim 14.
Claim(s) 3-5, 10-12 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bleyer et al., US 20200111232, DOS SANTOS MENDONCA et al., US 20220292718, Grossman, US 20130182083 and Lahr et al., US 10,775,881. Chaurasia et al., US 11,451,758 in view Hyttinen et al., US 20160227127.
In considering claim 3,
As noted in claim 1, Bleyer discloses HMD system which includes one or more cameras including multiple marker (characteristic) patterns which are used to calibrate the cameras/HMD. Bleyer does disclose where each image can be displayed at different angles or distance (para 008) and the distance between markers and determining the position/orientation of the display/reflection (para 67) is used to determine the distance (para 73, 77, 84-85, 101-102) to the captured perspective distances which is used to carry out intrinsic calibration (para 08, 100, 104-15). Bleyer discloses the system may include stereoscopic camera (para 110), which includes different images for different cameras with an overlapped region (para 004).
Although Bleyer disclose the perspective distance (not explicitly virtual distance) between images/patterns, DOS SANTOS MENDOCA disclose distance regarding the fiducial mark and cameras (para 29).
The examiner evidences Hyttinen et al., US 20160227127 which discloses such practice (para 104). Where the system includes a plurality of cameras and the virtual distance is compared to the actual distance in order to calibrate the cameras, in order to calibrate the one or more cameras using the one or more images, and updating the intrinsic and/or extrinsic properties.
The motivation to modify Bleyer/DOS SANTOS MENDOCA with Hyttinen provides the known measures to ensure calibration of the cameras and thus ensure proper operation, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
I considering claim 4,
As noted above Bleyer discloses a plurality of cameras including stereoscopic camera which are calibrated using multiple image patterns (markers) as noted in claim 1 and 3.
Regarding the newly claimed 1st and 2nd relationship: (also see claim 5).
Bleyer discloses that the pitch, roll and yaw (para 45, 56 and 73) which are all rotations around the respective axis (x, y and z) for the angular alignment of the HMD relative to the displayed pattern are to perform calibration.
Hyttinen was evidenced to show the calibration between multiple cameras, and the position/alignment with respect to cameras have different camera views to ensure direction and distance between the cameras for proper coverage.
The motivation to modify Bleyer/DOS SANTOS MENDOCA/Grossman/Lahr combination with Hyttinen provides the well-known feature of ensuring alignment between multiple cameras which capture different view of a virtual display, thus ensuring a proper display/capture and calibration thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 5,
The incorporated Hyttinen discloses the position (distances and directions) for each of the plurality of cameras with respect to other camera (para 48, 64,-67, 69, 96-99, 103, 111, 113-116, 119, 128-131. 134 and 143 to ensure calibration/coverage.
The motivation to modify Bleyer/DOS SANTOS MENDOCA/Grossman/Lahr combination with Hyttinen provides the well-known feature of ensuring alignment between multiple cameras which capture different view of a virtual display, thus ensuring a proper display/capture and calibration thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 10,
Refer to claim 3.
In considering claim 11,
Refer to claim 4.
In considering claim 12,
Refer to claim 5.
In considering claim 17,
Refer to claim 3.
In considering claim 18,
Refer to claim 4.
In considering claim 19,
Refer to claim 5.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure—see newly cited references on attached form PTO-892.
The below are additional references that disclose the use of SLAM when calibrating cameras.
US 20230400327 (para 108) discloses SLAM for camera calibration.
US 20210287385 (para 61) discloses the use of SLAM to perform camera calibration.
Previously cited; US 20210124174 (para 95, 10, 190, 277, 296, 334) discloses the use of SLAM algorithm for camera calibration.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN P YENKE whose telephone number is (571)272-7359. The examiner can normally be reached Typically 8:00am-4:30pm (M-F).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Miller can be reached on 571-272-7353. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN P YENKE/Primary Examiner, Art Unit 2422
Prosecution Insights