DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 9, 12 – 20, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Himane (U.S. PG Pub 2020/0089313) in view of Kudirka et al. (U.S. PG Pub 2018/0261010).
Regarding Claim 1, Himane teaches a method for calibrating a transparent wearable display (Figure 1, Element 120. Paragraph 34) configured to be used in parallel with an external display (Figure 1, Element 135. Paragraph 41), the method being performed by a calibration device, the method comprising the steps of:
performing a first calibration (Figure 7, Element 720. Paragraphs 91 - 92) to define an indicated position on the external display (Figure 1, Element 135. Paragraph 41) at least partly based on a direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34);
determining an indicated position on the external display (Figure 1, Element 135. Paragraph 41), based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34), the indicated position corresponding to a user interface element (Figure 7, Element Marker. Paragraph 90) shown on the external display (Figure 1, Element 135. Paragraph 41);
determining an element position on the external display (Figure 1, Element 135. Paragraph 41) corresponding to a centre position (Seen in Figures 3A and 3B) of the user interface element (Figure 7, Element Marker. Paragraph 90); and
performing a translational adjustment of calibration (Figure 4, Element 446. Paragraph 64) of how the indicated position is determined on the external display (Figure 1, Element 135. Paragraph 41) based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34).
Himane is silent with regards to move the indicated position closer to the element position in response to activation of a user interface element using the direction of the wearable display.
Kudirka et al. teach to move the indicated position (Element the virtual position. Paragraph 61) closer to the element position (Element background position. Paragraph 61) in response to activation of a user interface element (Element selecting the calibration function. Paragraph 61) using the direction of the wearable display (Figure 7, Element 100. Paragraph 23).
It would have been obvious to a person of ordinary skill in the art to modify the teachings of the heads-up display of Himane with the teachings of the calibration of Kudirka et al. The motivation to modify the teachings of Himane with the teachings of Kudirka et al. is to provide a calibration of the HUD while the user is looking in a specific direction, as taught by Kudirka et al. (Paragraph 61).
Regarding Claim 2, Himane in view of Kudirka et al. teach the method according to claim 1 (See Above). Himane teaches wherein the step of performing a translational calibration (Figure 4, Element 446. Paragraph 64) comprises moving the indicated position towards the element position, by a distance being the distance between the indicated position and the element position multiplied by a nudging factor (Paragraph 64).
Regarding Claim 3, Himane in view of Kudirka et al. teach the method according to claim 1 (See Above). Himane teaches further comprising: recalibrating (Paragraph 98) how the indicated position on the external display (Figure 1, Element 135. Paragraph 41) is based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34).
Regarding Claim 4, Himane in view of Kudirka et al. teach the method according to claim 3 (See Above). Himane teaches wherein the step of recalibrating is repeated (Paragraph 98), and wherein, in each iteration of the step of recalibrating, a time interval (Element time interval. Paragraph 98) until the next iteration of the step of recalibrating is determined based on a most recently determined sensor drift (Paragraph 98).
Regarding Claim 5, Himane in view of Kudirka et al. teach the method according to claim 3 (See Above). Himane teaches wherein the step of recalibrating comprises the sub-steps of:
rendering, on the external display (Figure 1, Element 135. Paragraph 41), an image (Figure 3A, Element 310. Paragraph 51) comprising a high-luminance area and a low-luminance area (Seen in Figure 3A);
detecting, using a narrow-beam light sensor, fixedly mounted in relation to the wearable display (Figure 1, Element 120. Paragraph 34), whether the wearable display (Figure 1, Element 120. Paragraph 34) is directed to the high-luminance area or the low-luminance area (Seen in Figure 3A); and
repeating the rendering, with a different image (Paragraph 98), and detecting until the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34) is determined with sufficient accuracy.
Regarding Claim 6, Himane in view of Kudirka et al. teach the method according to claim 5 (See Above). Himane teaches wherein for each iteration, the step of rendering an image (Figure 3A, Element 310. Paragraph 51) comprises rendering, over time (Paragraph 98), a smaller area (Paragraph 91. Himane discloses that the marker can change size.), being a high-luminance area or a low-luminance area (Seen in Figure 3A) in the direction (Element position and orientation. Paragraph 91) that the wearable display (Figure 1, Element 120. Paragraph 34) is directed.
Regarding Claim 7, Himane in view of Kudirka et al. teach the method according to claim 5 (See Above). Himane teaches wherein the step of rendering an image (Figure 3A, Element 310. Paragraph 51) comprises interjecting the image (Figure 3A, Element 310. Paragraph 51) as a single frame (Paragraph 89. Himane discloses that each scene will replace the previous frame) in main content shown on the external display (Figure 1, Element 135. Paragraph 41).
Regarding Claim 8, Himane in view of Kudirka et al. teach the method according to claim 3 (See Above). Himane teaches wherein the step of recalibrating comprises the sub-steps of:
rendering, on the external display (Figure 1, Element 135. Paragraph 41), a calibration marker (Figure 7, Element Marker. Paragraph 90);
determining, using a camera (Figure 4, Element 414. Paragraph 59), fixedly mounted in relation to the wearable display (Figure 1, Element 120. Paragraph 34), a position of the calibration marker (Figure 7, Element Marker. Paragraph 90) in relation to the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34); and
recalibrating how the indicated position on the external display (Figure 1, Element 135. Paragraph 41) is based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34) based on the position of the calibration marker (Figure 7, Element Marker. Paragraph 90) in relation to the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34).
Regarding Claim 9, Himane in view of Kudirka et al. teach the method according to claim 8 (See Above). Himane teaches wherein the step of rendering a calibration marker (Figure 7, Element Marker. Paragraph 90) comprises overlaying the calibration marker (Figure 7, Element Marker. Paragraph 90) in main content shown on the external display (Figure 1, Element 135. Paragraph 41).
Regarding Claim 12, Himane teaches a calibration device for calibrating a transparent wearable display (Figure 1, Element 120. Paragraph 34) configured to be used in parallel with an external display (Figure 1, Element 135. Paragraph 41), the calibration device comprising:
a processor (Figure 4, Element 402. Paragraph 56); and
a memory storing instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to:
perform a first calibration (Figure 7, Element 720. Paragraphs 91 - 92) to define an indicated position on the external display (Figure 1, Element 135. Paragraph 41) at least partly based on a direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34);
determine an indicated position on the external display (Figure 1, Element 135. Paragraph 41), based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34), the indicated position corresponding to a user interface element (Figure 7, Element Marker. Paragraph 90) shown on the external display (Figure 1, Element 135. Paragraph 41);
determine an element position on the external display (Figure 1, Element 135. Paragraph 41) corresponding to a centre position (Seen in Figures 3A and 3B) of the user interface element (Figure 7, Element Marker. Paragraph 90); and
perform a translational adjustment of calibration (Figure 4, Element 446. Paragraph 64) of how the indicated position is determined on the external display (Figure 1, Element 135. Paragraph 41) based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34).
Himane is silent with regards to move the indicated position closer to the element position in response to activation of a user interface element using the direction of the wearable display.
Kudirka et al. teach to move the indicated position (Element the virtual position. Paragraph 61) closer to the element position (Element background position. Paragraph 61) in response to activation of a user interface element (Element selecting the calibration function. Paragraph 61) using the direction of the wearable display (Figure 7, Element 100. Paragraph 23).
It would have been obvious to a person of ordinary skill in the art to modify the teachings of the heads-up display of Himane with the teachings of the calibration of Kudirka et al. The motivation to modify the teachings of Himane with the teachings of Kudirka et al. is to provide a calibration of the HUD while the user is looking in a specific direction, as taught by Kudirka et al. (Paragraph 61).
Regarding Claim 13, Himane in view of Kudirka et al. teach the calibration device according to claim 12 (See Above). Himane teaches wherein the instructions to perform a translational calibration (Figure 4, Element 446. Paragraph 64) comprise instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to move the indicated position towards the element position, by a distance being the distance between the indicated position and the element position multiplied by a nudging factor (Paragraph 64).
Regarding Claim 14, Himane in view of Kudirka et al. teach the calibration device according to claim 12 (See Above). Himane teaches further comprising instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to: recalibrate (Paragraph 98) how the indicated position on the external display (Figure 1, Element 135. Paragraph 41) is based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34).
Regarding Claim 15, Himane in view of Kudirka et al. teach the calibration device according to claim 14 (See Above). Himane teaches wherein the instructions to recalibrate are repeated (Paragraph 98), and wherein, in each iteration of the instructions to recalibrate, a time interval (Element time interval. Paragraph 98) until the next iteration of the instructions to recalibrate is determined based on a most recently determined sensor drift (Paragraph 98).
Regarding Claim 16, Himane in view of Kudirka et al. teach the calibration device according to claim 14 (See Above). Himane teaches wherein the instructions to recalibrate comprises instructions to:
render, on the external display (Figure 1, Element 135. Paragraph 41), an image (Figure 3A, Element 310. Paragraph 51) comprising a high-luminance area and a low-luminance area (Seen in Figure 3A);
detect, using a narrow-beam light sensor, fixedly mounted in relation to the wearable display (Figure 1, Element 120. Paragraph 34), whether the wearable display (Figure 1, Element 120. Paragraph 34) is directed to the high-luminance area or the low-luminance area (Seen in Figure 3A); and
repeat the instructions to render, with a different image (Paragraph 98), and to detect until the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34) is determined with sufficient accuracy.
Regarding Claim 17, Himane in view of Kudirka et al. teach the calibration device according to claim 16 (See Above). Himane teaches wherein for each iteration, the instructions to render an image (Figure 3A, Element 310. Paragraph 51) comprise instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to render, over time, a smaller area (Paragraph 91. Himane discloses that the marker can change size.), being a high-luminance area or a low-luminance area (Seen in Figure 3A) in the direction (Element position and orientation. Paragraph 91) that the wearable display (Figure 1, Element 120. Paragraph 34) is directed.
Regarding Claim 18, Himane in view of Kudirka et al. teach the calibration device according to claim 16 (See Above). Himane teaches wherein the instructions to render an image (Figure 3A, Element 310. Paragraph 51) comprises instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to interject the image (Figure 3A, Element 310. Paragraph 51) as a single frame (Paragraph 89. Himane discloses that each scene will replace the previous frame) in main content shown on the external display (Figure 1, Element 135. Paragraph 41).
Regarding Claim 19, Himane in view of Kudirka et al. teach the calibration device according to claim 14 (See Above). Himane teaches wherein the instructions to recalibrate comprise the instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to:
render, on the external display (Figure 1, Element 135. Paragraph 41), a calibration marker (Figure 7, Element Marker. Paragraph 90);
determine, using a camera (Figure 4, Element 414. Paragraph 59), fixedly mounted in relation to the wearable display (Figure 1, Element 120. Paragraph 34), a position of the calibration marker (Figure 7, Element Marker. Paragraph 90) in relation to the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34); and
recalibrate how the indicated position on the external display (Figure 1, Element 135. Paragraph 41) is based on the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34) based on the position of the calibration marker (Figure 7, Element Marker. Paragraph 90) in relation to the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34).
Regarding Claim 20, Himane in view of Kudirka et al. teach the calibration device according to claim 19 (See Above). Himane teaches wherein the instructions to render a calibration marker (Figure 7, Element Marker. Paragraph 90) comprise instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to overlay the calibration marker (Figure 7, Element Marker. Paragraph 90) in main content shown on the external display (Figure 1, Element 135. Paragraph 41).
Regarding Claim 22, Himane in view of Kudirka et al. teach the calibration device according to claim 14 (See Above). Himane teaches wherein the instructions to recalibrate comprise instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to:
determine, using an inertial measurement unit (Figure 4, Element 406. Paragraph 57), IMU, fixedly mounted in relation to the wearable display (Figure 1, Element 120. Paragraph 34), a translational movement of the wearable display (Figure 1, Element 120. Paragraph 34) in relation to the external monitor; and
recalibrate how the indicated position on the external display (Figure 1, Element 135. Paragraph 41) is defined by the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34) based on the translational movement.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Himane (U.S. PG Pub 2020/0089313) in view of Kudirka et al. (U.S. PG Pub 2018/0261010) in view of Wodrich et al. (U.S. PG Pub 2020/0364381).
Regarding Claim 21, Himane in view of Kudirka et al. teach the calibration device according to claim 14 (See Above). Himane teaches wherein the instructions to recalibrate comprise instructions that, when executed by the processor (Figure 4, Element 402. Paragraph 56), cause the calibration device to: determine, between emitters/receivers (Paragraph 36), fixedly mounted in relation to the wearable display (Figure 1, Element 120. Paragraph 34), and a second emitters/receivers (Paragraphs 36 and 47) fixedly mounted in relation to the external display (Figure 1, Element 135. Paragraph 41) a deviation of how the indicated position on the external display (Figure 1, Element 135. Paragraph 41) is defined by the direction (Element position and orientation. Paragraph 91) of the wearable display (Figure 1, Element 120. Paragraph 34) compared to a previously calibrated state; and recalibrate (Paragraph 98) to eliminate the deviation.
Himane is silent with regards to determine, using an determination of angle-of-arrival and/or angle-of-departure between a first radio transceiver and a second radio transceiver.
Wodrich et al. teach determine, using an determination of angle-of-arrival and/or angle-of-departure between a first radio transceiver and a second radio transceiver (Paragraphs 91 – 92).
Himane teaches a device which is different from the claimed interface apparatus by the substitution of the step(s) of distance determination. Wodrich et al. teach the substituted step(s) of distance determination and their functions were known in the art to provide distance determination, using an determination of angle-of-arrival and/or angle-of-departure between a first radio transceiver and a second radio transceiver.
The distance determination of Himane could have been substituted with distance determination as taught by Wodrich et al. and the results would have been predictable and resulted in distance determination, using an determination of angle-of-arrival and/or angle-of-departure between a first radio transceiver and a second radio transceiver. Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art at the time the invention was made.
Response to Arguments
Regarding the first argument, in which the applicant asserts that Himane is fundamentally different than the instant claimed invention. The applicant argues that Himane discloses that all positions, selectable elements and calibrations are on the HMD and not the external display. The applicant further argues that Himane fails to disclose the claimed determining steps of at least Claim 1. The applicant further argues that the markers shown in Figures 3A and 3B are not user interface elements. The applicant lastly argues that Himane fails to disclose the last limitation of at least Claim 1.
The examiner respectfully disagrees with the applicant’s assertions. Himane discloses “Accordingly, in some implementations, the physical second device 130 displays marker 310 on the physical display 135 of the physical second device 130 to facilitate tracking of the physical second device 130 by the physical first device 120 (Paragraph 51. Emphasis Added).” Himane further discloses “In some implementations, the physical second device 130 serves as a controller of the virtual experience, e.g., touch screen inputs to physical display 135 are detected by physical second device 130 and sent to physical first device 120 as inputs. For example, the user 110 may interact with the virtual scene 205 via an input interface of the physical second device 130 (Paragraph 54. Emphasis Added).” Himane lastly discloses “In particular, a user 110 may be able to look at the virtual second device 230 when physically interacting with the physical second device 130 and the user 110 can expect that their input through the virtual second device 230 will correspond to similar input (or interaction) at the physical second device 130. In addition, because each position on the virtual display 235 of the virtual second device 230 may correspond to a single position on the physical display 135 of the physical second device 130, the user 110 may navigate the virtual scene 205 using the virtual controller 320 presented on the virtual display 235 of the virtual second device 230 (e.g., up to and including the borders of the virtual representation) (Paragraph 55. Emphasis Added).” Therefore, Himane teaches that all positions and selectable elements are presented on the external display.
Himane discloses “In some implementations, the drift correction unit 446 is configured to use image tracking data to correlate inertial tracking data and determine location and rotation corrections for the physical first device 120, a physical second device 130, or the relative position of the physical first device 120 and the physical second device 130 (Paragraph 64. Emphasis Added).” Therefore, Himane is tracking and correcting locations and rotations of the first device and the second device. The Office is unmoved by the applicant’s argument, and the rejection is maintained.
Regarding the second argument, in which the applicant asserts that Kudirka et al. fails to disclose selecting a calibration function based on a user interface element on the external display. The examiner respectfully disagrees with the applicant’s assertion. Himane discloses “In addition, because each position on the virtual display 235 of the virtual second device 230 may correspond to a single position on the physical display 135 of the physical second device 130, the user 110 may navigate the virtual scene 205 using the virtual controller 320 presented on the virtual display 235 of the virtual second device 230 (e.g., up to and including the borders of the virtual representation) (Paragraph 55. Emphasis Added).” The examiner notes that Himane is relied upon for teaching that the user interface element on the external display also corresponds to the display on the HMD. Therefore Kudirka et al. does not need to teach the user interface element on the external display, since it is already taught by Himane. Kudirka et al. discloses “The user can calibrate, i.e. center, the mixed reality HUD by looking in the direction in which the ball is intended to be hit and selecting the calibration function, e.g., by using either a switch, voice commands, or head/eye movement input (Paragraph 61. Emphasis Added).” It is then Himane, as modified by Kudirka et al. that will teach the whole of the limitation. The Office is unmoved by the applicant’s argument, and the rejection is maintained.
All other arguments are held moot in light of the above rejection and/or the response to the first and/or second arguments.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Tichenor et al. (U.S. PG Pub 2016/0187971) teaches a heads-up display that senses a pitch angle defining the gaze vector for a display, similar to the instant invention.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW B SCHNIREL whose telephone number is (571)270-7690. The examiner can normally be reached Monday - Friday, 10 - 6 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Boddie can be reached at 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.B.S./Examiner, Art Unit 2625
/WILLIAM BODDIE/Supervisory Patent Examiner, Art Unit 2625
Prosecution Insights