Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to the amended claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Examiner’s Response:
Based upon the amendment the examiner has updated the rejection.
The basis for the rejection is that it is known to calibrate a camera via reflection of an image/barcode/id, notably using a mirror (disclosed by not explicitly claimed).
The examiner notes newly cited:
US 20130182083 (Fig 4, para 69) where the camera which has a FOV outside of the display, however with the reflection of the display via mirror (4), the camera can be calibrated.
PNG
media_image1.png
369
480
media_image1.png
Greyscale
The concept is even if the FOV of the camera is outside the image/pattern which is needed to calibrate the camera, a mirror can be placed in the path of the camera and thus view the pattern/image as noted above.
Additionally, the use of a mirror (which reflects) is a known device to show/reflect anything outside the FOV of a person for example.
In the even the applicant deems the reflection (mirror) as disclosed includes unexpected results and features which were unable to be done by those of ordinary skill in the art, the examiner requests applicant clarify/respond as such in order to expedite prosecution IAW MPEP 716.02.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6-8, 13-15 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bleyer et al., US 20200111232 in view of DOS SANTOS MENDONCA et al., US 20220292718 and Grossman, US 20130182083 and Lahr et al., US 10,775,881.
PNG
media_image2.png
613
723
media_image2.png
Greyscale
The claimed a plurality of characteristic patterns is met by calibration marker images (Fig 14, 15, para 8, 25-27, 32, 34, 66-83, 85-89, 94-95, 98, 101-104)
The claimed plurality of camera circuits…is met by the HMD camera system using the one or more cameras (abstract, 007-008, 30, 32, 39, 57, 70, 72, 76, 78, 82, 96-97, 100-101, 104-105, 108-110, 114-115), where the HMD may render a virtual object for alignment (para 57, 73, 87. 93, 95, 99). Bleyer disclose that the calibration may be performed using a display screen or projected and reflected through any number of mirror or other reflective surfaces (para 85).
The claimed processor is met where the computer system, including processor 2005 (para 107-11, 116, 120-121) may be in the form of HMD, to calibrate the HMD cameras including intrinsic and/or extrinsic parameters (abstract, para 007-8, 30-32, 104-105)
Bleyer discloses the “markers” used for calibration appear on the display of the HMD (para 34, 69, 78) and display screen or other reflective surface.
Regarding the newly amended physical pattern on the external of the device, the examiner incorporates DOS SANTOS MENDONCA et al., US 20220292718.
DOS SANTOS MENDONCA et al., US 20220292718 discloses the physical appearance on the outside of the eyewear including a fiducial to perform calibration of the device, where (par 50) the features of includes an appearance (physical pattern) fiducial (112L/R) which maybe infrared, visible, where they may be etched onto the frame, attached including adhesive (sticker).
The motivation to modify Bleyer with DO would ensure the calibration of the cameras of the HMD, using fiducials installed on the device which allows calibration of the device without requiring a calibration plate/system, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention to quickly perform calibration.
Regarding the newly amended features: it is noted above the combination does not explicitly recite the FOV of the camera is outside the FOV of the patterns thus requiring the reflective surface (mirror as disclosed).
The examiner incorporates Grossman, US 20130182083 (Fig 4, para 69) where the camera which has a FOV outside of the display, however with the reflection of the display via mirror (4), the camera can be calibrated.
PNG
media_image1.png
369
480
media_image1.png
Greyscale
Thus even if the FOV of the camera is outside the image/pattern which is needed to calibrate the camera, a mirror can be placed in the path of the camera and thus view the pattern/image as noted above in order to calibrate the camera.
The motivation to modify the above combination allows the prior art to place the cameras and calibration pattern in any FOV and ensures the capturing of such pattern using the reflective property of a mirror as taught by Grossman, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
The examiner also further provides additional evidence regarding the placement of the patterns/fiducials which may be on the side or atop the helmet/head-up display as shown by Fig 3 (sensors 308, 310) of Lahr et al., US 10,775,881, this reference is being to show the placement of the fiducials/marking can be anywhere.
The motivation to modify the above combination with Lahr allows the system to ensure proper calibration using a mirror/reflection (Grossman) to ensure the fiducials/markings can be used for calibration as would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 6,
Bleyer disclose a storage 202 (Fig 20) which may be included into the HMD, including a database (para 89, 97, 109-113, 118, 120).
In considering claim 7,
Bleyer does not explicitly recite match each of the plurality of virtual images to one of the patterns, however Bleyer does disclose calibration of the one or more cameras based upon the virtual image with respect to the plurality of marker patterns.
The incorporated DOS SANTOS MENDONCA discloses a system which compares the reference image and the field image to perform calibration (para 16-18, 30, 33, 39, 71 and 76), which performs calibration extrinsic/intrinsic base on the change of appearance of the fiducial markers between the reference image and field images (para 42-43).
Thus it would have been obvious to one of ordinary skill in the art to modify Bleyer with DOS SANTOS MENDONCA to ensure calibration by matching the images to the patterns (fiducials) to ensure proper calibration of the device, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 8,
Refer to claim 1.
In considering claim 13,
Refer to claim 6.
In considering claim 14,
Refer to claim 7.
In considering claim 15,
Refer to claim 1, where the system can be carried out by computer executable instructions via GP computer (para 117-120).
In considering claim 20,
The claimed storing…refer to claim 13.
The claimed matching…refer to claim 14.
Claim(s) 3-5, 10-12 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bleyer et al., US 20200111232, DOS SANTOS MENDONCA et al., US 20220292718, Grossman, US 20130182083 and Lahr et al., US 10,775,881.
in view Hyttinen et al., US 20160227127.
In considering claim 3,
As noted in claim 1, Bleyer discloses HMD system which includes one or more cameras including multiple marker (characteristic) patterns which are used to calibrate the cameras/HMD. Bleyer does disclose where each image can be displayed at different angles or distance (para 008) and the distance between markers and determining the position/orientation of the display/reflection (para 67) is used to determine the distance (para 73, 77, 84-85, 101-102) to the captured perspective distances which is used to carry out intrinsic calibration (para 08, 100, 104-15). Bleyer discloses the system may include stereoscopic camera (para 110), which includes different images for different cameras with an overlapped region (para 004).
Although Bleyer disclose the perspective distance (not explicitly virtual distance) between images/patterns, DOS SANTOS MENDOCA disclose distance regarding the fiducial mark and cameras (para 29).
The examiner evidences Hyttinen et al., US 20160227127 which discloses such practice (para 104). Where the system includes a plurality of cameras and the virtual distance is compared to the actual distance in order to calibrate the cameras, in order to calibrate the one or more cameras using the one or more images, and updating the intrinsic and/or extrinsic properties.
The motivation to modify Bleyer/DOS SANTOS MENDOCA with Hyttinen provides the known measures to ensure calibration of the cameras and thus ensure proper operation, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
I considering claim 4,
As noted above Bleyer discloses a plurality of cameras including stereoscopic camera which are calibrated using multiple image patterns (markers) as noted in claim 1 and 3.
Regarding the newly claimed 1st and 2nd relationship: (also see claim 5).
Bleyer discloses that the pitch, roll and yaw (para 45, 56 and 73) which are all rotations around the respective axis (x, y and z) for the angular alignment of the HMD relative to the displayed pattern are to perform calibration.
Hyttinen was evidenced to show the calibration between multiple cameras, and the position/alignment with respect to cameras have different camera views to ensure direction and distance between the cameras for proper coverage.
The motivation to modify Bleyer/DOS SANTOS MENDOCA/Grossman/Lahr combination with Hyttinen provides the well-known feature of ensuring alignment between multiple cameras which capture different view of a virtual display, thus ensuring a proper display/capture and calibration thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 5,
The incorporated Hyttinen discloses the position (distances and directions) for each of the plurality of cameras with respect to other camera (para 48, 64,-67, 69, 96-99, 103, 111, 113-116, 119, 128-131. 134 and 143 to ensure calibration/coverage.
The motivation to modify Bleyer/DOS SANTOS MENDOCA/Grossman/Lahr combination with Hyttinen provides the well-known feature of ensuring alignment between multiple cameras which capture different view of a virtual display, thus ensuring a proper display/capture and calibration thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 10,
Refer to claim 3.
In considering claim 11,
Refer to claim 4.
In considering claim 12,
Refer to claim 5.
In considering claim 17,
Refer to claim 3.
In considering claim 18,
Refer to claim 4.
In considering claim 19,
Refer to claim 5.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure—see newly cited references on attached form PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN P YENKE whose telephone number is (571)272-7359. The examiner can normally be reached Typically 8:00am-4:30pm (M-F).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Miller can be reached on 571-272-7353. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN P YENKE/Primary Examiner, Art Unit 2422