Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Acknowledgement is made of remarks filed 30 March 2026 in which no claims were amended. Claims 1-20 are currently pending and an office action on the merits follows.
Inventorship
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Pub. No. 2022/0377306 by Borys et al. (“Borys”) in view of U.S. Pub. No. 2022/0391013 by Vlaskamp (“Vlaskamp”).
As to claim 1, Borys discloses a method for calibrating a sensor alignment offset on a display device comprising a stereoscopic sensor system and a stereoscopic display system (Borys, the optical sensors 102, 104 used for stereo vision may become misaligned due to bending and/or other physical manipulation of an AR device 100. ¶ [0035]), the method comprising:
obtaining a left sensed image from a left image sensor of the stereoscopic sensor system and a right sensed image from a right image sensor of the stereoscopic sensor system (Borys, The misaligned vantage point mitigation system 112 determines whether the vantage points of the optical sensors 102, 104 are misaligned based on the location of matching features identified in images captured by both optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 accesses a pair of corresponding images captured by each of the left optical sensor 102 and the right optical sensor 104. The corresponding images are images that were captured by the left optical sensor 102 and the right optical sensor 104 simultaneously or near-simultaneously. Figure 1, ¶ [0036])(Borys, The feature identification component 204 identifies matching features in a pair of corresponding images. A feature is identifiable portion or component of an image. Examples of features that may be identified in an image include an edge, corner, point of interest, blob, ridge, and the like. Matching features in the two images are features identified in each image that are determined to be the same feature. ¶ [0045]),
based at least upon the user input, calibrating the sensor alignment offset of the left image sensor and the right image sensor in the alignment direction (Borys, The misaligned vantage point mitigation system 112 mitigates a detected misalignment of the optical sensors 102, 104 based on the determined variance between the expected and actual locations of the feature in the images. For example, the misaligned vantage point mitigation system 112 uses the variance to calculate an adjustment variable to account for the determined variance…The adjustment variable determined by the misaligned vantage point mitigation system 112 is subsequently used to account for the determined misalignment of the optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 may provide the adjustment variable to the AR processing system 108 for use when subsequently calculating depth data from images captured by the left optical sensor 102 and the right optical sensor 104. ¶ [0039]), and
applying the sensor alignment offset to adjust the display of images from the stereoscopic sensor system (Borys, The misaligned vantage point mitigation system 112 mitigates a detected misalignment of the optical sensors 102, 104 based on the determined variance between the expected and actual locations of the feature in the images. For example, the misaligned vantage point mitigation system 112 uses the variance to calculate an adjustment variable to account for the determined variance…The adjustment variable determined by the misaligned vantage point mitigation system 112 is subsequently used to account for the determined misalignment of the optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 may provide the adjustment variable to the AR processing system 108 for use when subsequently calculating depth data from images captured by the left optical sensor 102 and the right optical sensor 104. ¶ [0039])(Borys, The misaligned vantage point mitigation system 112 may repeat this process to further refine the adjustment variable to account for the misalignment of the optical sensors 102, 104 and/or changes to the alignment of the optical sensors 102, 104. For example, the alignment of the optical sensors 102, 104 may change due to the AR device 100 being used by a different user. ¶ [0040]).
Borys does not expressly teach
forming a pair of images with binocular divergence based at least upon the left sensed image and the right sensed image,
displaying the pair of images with binocular divergence using the stereoscopic display system,
receiving a user input in response to displaying the pair of images with binocular divergence, the user input relating to a vertical disparity of the pair of images with binocular divergence in an alignment direction,
based at least upon the user input, calibrating the sensor alignment offset of the left image sensor and the right image sensor in the alignment direction,
Vlaskamp teaches a display system for determining alignment
forming a pair of images with binocular divergence based at least upon the left sensed image and the right sensed image (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]),
displaying the pair of images with binocular divergence using the stereoscopic display system (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302])(Vlaskamp, As shown in FIG. 16, the HMD may provide different respective alignment markers to the user's left and right eyes in order to demonstrate any left-right vertical misalignment. For example, the HMD may display screen 1600a to a user's left eye and may display screen 1600b to the user's right eye. Screen 1600a may include a left-eye horizontal alignment marker 1502 and a vertical alignment marker 1506, while screen 1600b may include a right-eye horizontal alignment marker 1504 and a vertical alignment marker 1506. Figure 16, ¶ [0303]),
receiving a user input in response to displaying the pair of images with binocular divergence, the user input relating to a vertical disparity of the pair of images with binocular divergence in an alignment direction (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]),
based at least upon the user input, calibrating the sensor alignment offset of the left image sensor and the right image sensor in the alignment direction (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]),
At the time before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Borys’s misalignment correction to include Vlaskamp’s user input alignment because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, Borys’s misalignment correction as modified by Vlaskamp’s user input alignment is known to yield a predictable result of providing a head-mounted display which is aligned to the specific user since this permits the user to individually correct the alignment based on their eye positions. Thus, a person of ordinary skill would have appreciated including in Borys’s misalignment correction the ability to do Vlaskamp’s user input alignment since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Thus, Borys, as modified by Vlaskamp, teaches the displayed alignment lines and user input for misalignment correction.
As to claim 2, Borys, as modified by Vlaskamp, teaches the method wherein forming the pair of images with binocular divergence comprises identifying a feature oriented in a non-alignment direction and highlighting the feature in a left offset image and a right offset image of the pair of images with binocular divergence (Borys, The feature identification component 204 identifies matching features in a pair of corresponding images. A feature is identifiable portion or component of an image. Examples of features that may be identified in an image include an edge, corner, point of interest, blob, ridge, and the like. Matching features in the two images are features identified in each image that are determined to be the same feature. ¶ [0045]).
As to claim 3, Borys, as modified by Vlaskamp, teaches the method wherein highlighting the feature comprises displaying a left alignment line in the left offset image and a right alignment line in the right offset image (Borys, The feature identification component 204 identifies matching features in a pair of corresponding images. A feature is identifiable portion or component of an image. Examples of features that may be identified in an image include an edge, corner, point of interest, blob, ridge, and the like. Matching features in the two images are features identified in each image that are determined to be the same feature. ¶ [0045]) (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). In addition, the motivation used is the same as in the rejection of claim 1.
As to claim 4, Borys, as modified by Vlaskamp, teaches the method wherein displaying the pair of images with binocular divergence comprises displaying the left alignment line over the left sensed image and displaying the right alignment line over the right sensed image (Borys, The misaligned vantage point mitigation system 112 determines whether the vantage points of the optical sensors 102, 104 are misaligned based on the location of matching features identified in images captured by both optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 accesses a pair of corresponding images captured by each of the left optical sensor 102 and the right optical sensor 104. The corresponding images are images that were captured by the left optical sensor 102 and the right optical sensor 104 simultaneously or near-simultaneously. Figure 1, ¶ [0036]) (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). In addition, the motivation used is the same as in the rejection of claim 1.
As to claim 5, Borys, as modified by Vlaskamp, teaches the method wherein applying the sensor alignment offset to adjust the display of images from the stereoscopic sensor system comprises
obtaining, from the stereoscopic sensor system, a second left sensed image and a second right sensed image for reprojection (Borys, The misaligned vantage point mitigation system 112 determines whether the vantage points of the optical sensors 102, 104 are misaligned based on the location of matching features identified in images captured by both optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 accesses a pair of corresponding images captured by each of the left optical sensor 102 and the right optical sensor 104. The corresponding images are images that were captured by the left optical sensor 102 and the right optical sensor 104 simultaneously or near-simultaneously. Figure 1, ¶ [0036]),
adjusting, based at least upon the sensor alignment offset, a display location of one or more of the second left sensed image or the second right sensed image to form adjusted stereoscopic images (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]), and
displaying the adjusted stereoscopic image using the stereoscopic display system (Borys, The misaligned vantage point mitigation system 112 mitigates a detected misalignment of the optical sensors 102, 104 based on the determined variance between the expected and actual locations of the feature in the images. For example, the misaligned vantage point mitigation system 112 uses the variance to calculate an adjustment variable to account for the determined variance…The adjustment variable determined by the misaligned vantage point mitigation system 112 is subsequently used to account for the determined misalignment of the optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 may provide the adjustment variable to the AR processing system 108 for use when subsequently calculating depth data from images captured by the left optical sensor 102 and the right optical sensor 104. ¶ [0039])(Borys, The misaligned vantage point mitigation system 112 may repeat this process to further refine the adjustment variable to account for the misalignment of the optical sensors 102, 104 and/or changes to the alignment of the optical sensors 102, 104. For example, the alignment of the optical sensors 102, 104 may change due to the AR device 100 being used by a different user. ¶ [0040]). In addition, the motivation used is the same as in the rejection of claim 1.
As to claim 6, Borys, as modified by Vlaskamp, teaches the method wherein the user input is a first user input, wherein the stereoscopic display system includes a left projector and a right projector (Borys, The output components 526 may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, ¶ [0076]) (Vlaskamp, head-mounted display system may be configured to determine whether its left-eye and right-eye displays are level with the user's eyes (e.g., their interocular axis) and provide appropriate user feedback encouraging the user to adjust an image displayed on a left or right eyepiece, with the ultimate goal of correcting a calibration metric of a left or right display sub system (such as a projector input for a corresponding left or right eyepiece) so that images projected through left-eye and right-eye displays are level with an interocular axis between the eyes. ¶ [0296]), and wherein the method further comprises calibrating the stereoscopic display system by presenting dichoptic lines using the stereoscopic display system and receiving another user input relating to a vertical disparity of the dichoptic lines to calibrate a display alignment offset (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]). In addition, the motivation used is the same as in the rejection of claim 1.
As to claim 7, Borys, as modified by Vlaskamp, teaches the method wherein displaying the pair of images with binocular divergence using the stereoscopic display system comprises displaying the pair of images with binocular divergence based at least upon the display alignment offset (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]). In addition, the motivation used is the same as in the rejection of claim 1.
As to claim 8, Borys discloses a display device (Borys, AR device 100, Figure 1) comprising:
a stereoscopic sensor system including a left image sensor and a right image sensor (Borys, the optical sensors 102, 104 used for stereo vision may become misaligned due to bending and/or other physical manipulation of an AR device 100. ¶ [0035]);
a stereoscopic display system (Borys, the optical sensors 102, 104 used for stereo vision may become misaligned due to bending and/or other physical manipulation of an AR device 100. ¶ [0035]);
a logic subsystem (Borys, processors 504, Figure 5); and
a storage subsystem (Borys, memory 506, Figure 5) comprising instructions executable by the logic subsystem (Borys, FIG. 5 shows a diagrammatic representation of the machine 500 in the example form of a computer system, within which instructions 510 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 500 to perform any one or more of the methodologies discussed herein may be executed. Figure 5, ¶ [0074]) to
obtain a left sensed image from the left image sensor and a right sensed image from the right image sensor (Borys, The left optical sensor 102 and right optical sensor 104 may be any type of sensor capable of capturing image data. For example, the left optical sensor 102 and the right optical sensor 104 may be cameras configured to capture images and/or video. The images captured by the left optical sensor 102 and the right optical sensor 104 are provided to the AR processing system 108 via the communication links 108. Figure 1, ¶ [0028]),
based at least upon the user input, calibrate a sensor alignment offset of the left image sensor and the right image sensor in the alignment direction (Borys, The misaligned vantage point mitigation system 112 mitigates a detected misalignment of the optical sensors 102, 104 based on the determined variance between the expected and actual locations of the feature in the images. For example, the misaligned vantage point mitigation system 112 uses the variance to calculate an adjustment variable to account for the determined variance…The adjustment variable determined by the misaligned vantage point mitigation system 112 is subsequently used to account for the determined misalignment of the optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 may provide the adjustment variable to the AR processing system 108 for use when subsequently calculating depth data from images captured by the left optical sensor 102 and the right optical sensor 104. ¶ [0039]),
apply the sensor alignment offset to adjust the display of images from the stereoscopic sensor system (Borys, The misaligned vantage point mitigation system 112 mitigates a detected misalignment of the optical sensors 102, 104 based on the determined variance between the expected and actual locations of the feature in the images. For example, the misaligned vantage point mitigation system 112 uses the variance to calculate an adjustment variable to account for the determined variance…The adjustment variable determined by the misaligned vantage point mitigation system 112 is subsequently used to account for the determined misalignment of the optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 may provide the adjustment variable to the AR processing system 108 for use when subsequently calculating depth data from images captured by the left optical sensor 102 and the right optical sensor 104. ¶ [0039])(Borys, The misaligned vantage point mitigation system 112 may repeat this process to further refine the adjustment variable to account for the misalignment of the optical sensors 102, 104 and/or changes to the alignment of the optical sensors 102, 104. For example, the alignment of the optical sensors 102, 104 may change due to the AR device 100 being used by a different user. ¶ [0040]).
Borys does not expressly disclose
form a pair of images with binocular divergence based at least upon the left sensed image and the right sensed image,
display the pair of images with binocular divergence using the stereoscopic display system,
receive a user input in response to displaying the pair of images with binocular divergence, the user input relating to a vertical disparity of the pair of images with binocular divergence in an alignment direction,
based at least upon the user input, calibrate a sensor alignment offset of the left image sensor and the right image sensor in the alignment direction,
Vlaskamp teaches a display system for determining alignment
form a pair of images with binocular divergence based at least upon the left sensed image and the right sensed image (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]),
display the pair of images with binocular divergence using the stereoscopic display system (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302])(Vlaskamp, As shown in FIG. 16, the HMD may provide different respective alignment markers to the user's left and right eyes in order to demonstrate any left-right vertical misalignment. For example, the HMD may display screen 1600a to a user's left eye and may display screen 1600b to the user's right eye. Screen 1600a may include a left-eye horizontal alignment marker 1502 and a vertical alignment marker 1506, while screen 1600b may include a right-eye horizontal alignment marker 1504 and a vertical alignment marker 1506. Figure 16, ¶ [0303]),
receive a user input in response to displaying the pair of images with binocular divergence, the user input relating to a vertical disparity of the pair of images with binocular divergence in an alignment direction (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]),
based at least upon the user input, calibrate a sensor alignment offset of the left image sensor and the right image sensor in the alignment direction (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]),
At the time before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Borys’s misalignment correction to include Vlaskamp’s user input alignment because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, Borys’s misalignment correction as modified by Vlaskamp’s user input alignment is known to yield a predictable result of providing a head-mounted display which is aligned to the specific user since this permits the user to individually correct the alignment based on their eye positions. Thus, a person of ordinary skill would have appreciated including in Borys’s misalignment correction the ability to do Vlaskamp’s user input alignment since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Thus, Borys, as modified by Vlaskamp, teaches the displayed alignment lines and user input for misalignment correction.
As to claim 9, Borys, as modified by Vlaskamp, teaches the display device wherein the instructions executable to obtain the left sensed image from the left image sensor and the right sensed image from the right image sensor comprise instructions executable to identify a feature oriented in a non-alignment direction and highlight the feature in a left offset image and a right offset image of the pair of images with binocular divergence (Borys, The feature identification component 204 identifies matching features in a pair of corresponding images. A feature is identifiable portion or component of an image. Examples of features that may be identified in an image include an edge, corner, point of interest, blob, ridge, and the like. Matching features in the two images are features identified in each image that are determined to be the same feature. ¶ [0045]).
As to claim 10, Borys, as modified by Vlaskamp, teaches the display device wherein the instructions executable to highlight the feature in the left offset image and the right offset image of the pair of images with binocular divergence comprise instructions executable to display a left alignment line in the left offset image and a right alignment line in the right offset image (Borys, The feature identification component 204 identifies matching features in a pair of corresponding images. A feature is identifiable portion or component of an image. Examples of features that may be identified in an image include an edge, corner, point of interest, blob, ridge, and the like. Matching features in the two images are features identified in each image that are determined to be the same feature. ¶ [0045]) (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). In addition, the motivation used is the same as in the rejection of claim 8.
As to claim 11, Borys, as modified by Vlaskamp, teaches the display device wherein the instructions executable to display the pair of images with binocular divergence comprise instructions executable to display the left alignment line over the left sensed image and to display the right alignment line over the right sensed image (Borys, The misaligned vantage point mitigation system 112 determines whether the vantage points of the optical sensors 102, 104 are misaligned based on the location of matching features identified in images captured by both optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 accesses a pair of corresponding images captured by each of the left optical sensor 102 and the right optical sensor 104. The corresponding images are images that were captured by the left optical sensor 102 and the right optical sensor 104 simultaneously or near-simultaneously. Figure 1, ¶ [0036]) (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). In addition, the motivation used is the same as in the rejection of claim 8.
As to claim 12, Borys, as modified by Vlaskamp, teaches the display device wherein the instructions executable to apply the sensor alignment offset to adjust the display of images from the stereoscopic sensor system comprise instructions executable to
obtain, from the stereoscopic sensor system, a second left sensed image and a second right sensed image for reprojection (Borys, The misaligned vantage point mitigation system 112 determines whether the vantage points of the optical sensors 102, 104 are misaligned based on the location of matching features identified in images captured by both optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 accesses a pair of corresponding images captured by each of the left optical sensor 102 and the right optical sensor 104. The corresponding images are images that were captured by the left optical sensor 102 and the right optical sensor 104 simultaneously or near-simultaneously. Figure 1, ¶ [0036]),
adjust, based at least upon the sensor alignment offset, a display location of one or more of the second left sensed image or the second right sensed image to form adjusted stereoscopic images (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]), and
display the adjusted stereoscopic images using the stereoscopic display system (Borys, The misaligned vantage point mitigation system 112 mitigates a detected misalignment of the optical sensors 102, 104 based on the determined variance between the expected and actual locations of the feature in the images. For example, the misaligned vantage point mitigation system 112 uses the variance to calculate an adjustment variable to account for the determined variance…The adjustment variable determined by the misaligned vantage point mitigation system 112 is subsequently used to account for the determined misalignment of the optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 may provide the adjustment variable to the AR processing system 108 for use when subsequently calculating depth data from images captured by the left optical sensor 102 and the right optical sensor 104. ¶ [0039])(Borys, The misaligned vantage point mitigation system 112 may repeat this process to further refine the adjustment variable to account for the misalignment of the optical sensors 102, 104 and/or changes to the alignment of the optical sensors 102, 104. For example, the alignment of the optical sensors 102, 104 may change due to the AR device 100 being used by a different user. ¶ [0040]). In addition, the motivation used is the same as in the rejection of claim 8.
As to claim 13, Borys, as modified by Vlaskamp, teaches the display device wherein the stereoscopic display system includes a left projector and a right projector (Borys, The output components 526 may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, ¶ [0076]) (Vlaskamp, head-mounted display system may be configured to determine whether its left-eye and right-eye displays are level with the user's eyes (e.g., their interocular axis) and provide appropriate user feedback encouraging the user to adjust an image displayed on a left or right eyepiece, with the ultimate goal of correcting a calibration metric of a left or right display sub system (such as a projector input for a corresponding left or right eyepiece) so that images projected through left-eye and right-eye displays are level with an interocular axis between the eyes. ¶ [0296]), and wherein the instructions are further executable to present dichoptic lines using the stereoscopic display system and receive another user input relating to a vertical disparity of the dichoptic lines to calibrate a display alignment offset (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]). In addition, the motivation used is the same as in the rejection of claim 8.
As to claim 14, Borys, as modified by Vlaskamp, teaches the display device wherein the instructions executable to display the pair of images with binocular divergence using the stereoscopic display system comprise instructions executable to display the pair of images with binocular divergence based at least upon the display alignment offset (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]). In addition, the motivation used is the same as in the rejection of claim 8.
As to claim 15, Borys discloses a head mounted display (HMD) device (Borys, AR device 100, Figure 1) comprising:
a stereoscopic sensor system including a left image sensor and a right image sensor (Borys, the optical sensors 102, 104 used for stereo vision may become misaligned due to bending and/or other physical manipulation of an AR device 100. ¶ [0035]);
a logic subsystem (Borys, processors 504, Figure 5); and
a storage subsystem (Borys, memory 506, Figure 5) comprising instructions executable by the logic subsystem (Borys, FIG. 5 shows a diagrammatic representation of the machine 500 in the example form of a computer system, within which instructions 510 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 500 to perform any one or more of the methodologies discussed herein may be executed. Figure 5, ¶ [0074]) to
obtain a left sensed image from the left image sensor and a right sensed image from the right image sensor (Borys, The left optical sensor 102 and right optical sensor 104 may be any type of sensor capable of capturing image data. For example, the left optical sensor 102 and the right optical sensor 104 may be cameras configured to capture images and/or video. The images captured by the left optical sensor 102 and the right optical sensor 104 are provided to the AR processing system 108 via the communication links 108. Figure 1, ¶ [0028]);
apply the sensor alignment offset to adjust the display of images from the stereoscopic sensor system (Borys, The misaligned vantage point mitigation system 112 mitigates a detected misalignment of the optical sensors 102, 104 based on the determined variance between the expected and actual locations of the feature in the images. For example, the misaligned vantage point mitigation system 112 uses the variance to calculate an adjustment variable to account for the determined variance…The adjustment variable determined by the misaligned vantage point mitigation system 112 is subsequently used to account for the determined misalignment of the optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 may provide the adjustment variable to the AR processing system 108 for use when subsequently calculating depth data from images captured by the left optical sensor 102 and the right optical sensor 104. ¶ [0039])(Borys, The misaligned vantage point mitigation system 112 may repeat this process to further refine the adjustment variable to account for the misalignment of the optical sensors 102, 104 and/or changes to the alignment of the optical sensors 102, 104. For example, the alignment of the optical sensors 102, 104 may change due to the AR device 100 being used by a different user. ¶ [0040]).
Borys does not expressly disclose
a stereoscopic display system including a left projector and a right projector;
present dichoptic lines using the stereoscopic display system;
receive a user input relating to a vertical disparity of the dichoptic lines to calibrate a display alignment offset of a left display image using the left projector, and a right display image using the right projector;
form a pair of images with binocular divergence based at least upon the left sensed image and the right sensed image;
display the pair of images with binocular divergence using the stereoscopic display system;
receive another user input relating to a vertical disparity of the pair of images with binocular divergence in an alignment direction;
in response, calibrate a sensor alignment offset of the left image sensor and the right image sensor in the alignment direction;
Vlaskamp teaches a display system for determining alignment
a stereoscopic display system including a left projector and a right projector (Vlaskamp, head-mounted display system may be configured to determine whether its left-eye and right-eye displays are level with the user's eyes (e.g., their interocular axis) and provide appropriate user feedback encouraging the user to adjust an image displayed on a left or right eyepiece, with the ultimate goal of correcting a calibration metric of a left or right display sub system (such as a projector input for a corresponding left or right eyepiece) so that images projected through left-eye and right-eye displays are level with an interocular axis between the eyes. ¶ [0296]);
present dichoptic lines using the stereoscopic display system (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]);
receive a user input relating to a vertical disparity of the dichoptic lines to calibrate a display alignment offset of a left display image using the left projector, and a right display image using the right projector (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]);
form a pair of images with binocular divergence based at least upon the left sensed image and the right sensed image (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]);
display the pair of images with binocular divergence using the stereoscopic display system (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302])(Vlaskamp, As shown in FIG. 16, the HMD may provide different respective alignment markers to the user's left and right eyes in order to demonstrate any left-right vertical misalignment. For example, the HMD may display screen 1600a to a user's left eye and may display screen 1600b to the user's right eye. Screen 1600a may include a left-eye horizontal alignment marker 1502 and a vertical alignment marker 1506, while screen 1600b may include a right-eye horizontal alignment marker 1504 and a vertical alignment marker 1506. Figure 16, ¶ [0303]);
receive another user input relating to a vertical disparity of the pair of images with binocular divergence in an alignment direction (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]);
in response, calibrate a sensor alignment offset of the left image sensor and the right image sensor in the alignment direction (Vlaskamp, The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Figures 15 and 16, ¶ [0306]);
At the time before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Borys’s misalignment correction to include Vlaskamp’s user input alignment because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, Borys’s misalignment correction as modified by Vlaskamp’s user input alignment is known to yield a predictable result of providing a head-mounted display which is aligned to the specific user since this permits the user to individually correct the alignment based on their eye positions. Thus, a person of ordinary skill would have appreciated including in Borys’s misalignment correction the ability to do Vlaskamp’s user input alignment since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Thus, Borys, as modified by Vlaskamp, teaches the displayed alignment lines and user input for misalignment correction.
As to claim 16, Borys, as modified by Vlaskamp, teaches the HMD device wherein the instructions executable to obtain the left sensed image from the left image sensor and the right sensed image from the right image sensor comprise instructions executable to identify a feature oriented in a non-alignment direction and highlight the feature in a left offset image and a right offset image of the pair of images with binocular divergence (Borys, The feature identification component 204 identifies matching features in a pair of corresponding images. A feature is identifiable portion or component of an image. Examples of features that may be identified in an image include an edge, corner, point of interest, blob, ridge, and the like. Matching features in the two images are features identified in each image that are determined to be the same feature. ¶ [0045]).
As to claim 17, Borys, as modified by Vlaskamp, teaches the HMD device wherein the instructions executable to highlight the feature in the left offset image and the right offset image of the pair of images with binocular divergence comprise instructions executable to display a left alignment line in the left offset image and a right alignment line in the right offset image (Borys, The feature identification component 204 identifies matching features in a pair of corresponding images. A feature is identifiable portion or component of an image. Examples of features that may be identified in an image include an edge, corner, point of interest, blob, ridge, and the like. Matching features in the two images are features identified in each image that are determined to be the same feature. ¶ [0045]) (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). In addition, the motivation used is the same as in the rejection of claim 15.
As to claim 18, Borys, as modified by Vlaskamp, teaches the HMD device wherein the instructions executable to display the pair of images with binocular divergence comprise instructions executable to display the left alignment line over the left sensed image and to display the right alignment line over the right sensed image (Borys, The misaligned vantage point mitigation system 112 determines whether the vantage points of the optical sensors 102, 104 are misaligned based on the location of matching features identified in images captured by both optical sensors 102, 104. For example, the misaligned vantage point mitigation system 112 accesses a pair of corresponding images captured by each of the left optical sensor 102 and the right optical sensor 104. The corresponding images are images that were captured by the left optical sensor 102 and the right optical sensor 104 simultaneously or near-simultaneously. Figure 1, ¶ [0036]) (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). In addition, the motivation used is the same as in the rejection of claim 15.
As to claim 19, Borys, as modified by Vlaskamp, teaches the HMD device wherein the instructions executable to display the pair of images with binocular divergence comprise instructions executable to display the left alignment line and to display the right alignment line in absence of the left sensed image and the right sensed image (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). Vlaskamp teaches display of alignment markers alone without a sensed image. In addition, the motivation used is the same as in the rejection of claim 15.
As to claim 20, Borys, as modified by Vlaskamp, teaches the HMD device wherein the instructions executable to receive the user input relating to the vertical disparity of the pair of images with binocular divergence in the alignment direction are executable to receive a gesture input using the stereoscopic sensor system (Vlaskamp, FIGS. 15 and 16 illustrate example display screens that may be provided to a user of a head-mounted display (HMD) (e.g., display 220, FIG. 3) as part of identifying vertical misalignment remaining after leveling. As shown in FIG. 15, a display screen such as screen 1500 may include user instructions as well as alignment markers such as markers 1502, 1504, and 1506 that highlight any vertical misalignment to the user. In some embodiments, the alignment markers make take the form of the letter “T” laying sideways. Figures 15 and 16, ¶ [0302]). Vlaskamp teaches receiving of user gesture input to correct the alignment. In addition, the motivation used is the same as in the rejection of claim 15.
Response to Arguments
Applicant's arguments filed 30 March 2026 have been fully considered but they are not persuasive.
Applicant’s representative asserts, in regards to independent claim 1, the cited prior art does not teach the “displaying the pair of image with binocular divergence using the stereoscopic display system, receiving a user input in response to displaying the pair of images with binocular divergence, based at least upon the user input, calibrating the sensor alignment offset of the left image sensor and the right image sensor, and applying the sensor alignment offset to adjust the display of images from the stereoscopic sensor system”. Specifically, Borys is silent regarding stereoscopic display systems and utilizing user input for stereoscopic sensor vertical alignment and Vlaskamp is silent regarding a stereoscopic sensor system as well as displaying image data therefrom. Further, neither Borys and Vlaskamp discloses adjusting display locations of left and/or right images from stereo cameras when displaying the images with stereo displays.
The Office respectfully disagrees with this assertion and submits the rejection above. As shown in the rejection Borys teaches a stereo vision system and a misalignment being corrected based on left and right sensed images (Borys, ¶¶ [0035, 0039, and 0040]). Vlaskamp teaches the vertical alignment markers, with markers provided in images for each of the user’s eyes, for which the user may provide input in order to align the head mounted display system (¶¶ 0302-0306]). Vlaskamp additionally teaches the HMD as presenting stereoscopic images (¶ [0157]) and the adjustments for at least one of the displayed images on the left or right eye displays (Vlaskamp, ¶ [0306]). As shown in the rejection above, Borys is combined with Vlaskamp to teach the user input correcting the stereo vision system for vertical alignment. The combination of Borys and Vlaskamp teaches the stereoscopic display sensing misalignment, presenting alignment markers, receiving user input on the alignment markers, and adjusting the offset of the stereoscopic image display based on that user input. Thus, the combination of Borys and Vlaskamp teaches the limitations of claim 1.
Applicant’s representative asserts, in regards to independent claims 8 and 15, these claims are not taught by the cited prior art of Borys and Vlaskamp for the same reasons as claim 1.
The Office respectfully disagrees with this assertion and submits the rejection above for these claims and arguments for claim 1 above.
Applicant’s representative asserts the claims are allowable and a notice of allowance should be sent.
The Office respectfully disagrees with this assertion and submits the rejection and response above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRENT D CASTIAUX whose telephone number is (571)272-5143. The examiner can normally be reached Mon-Fri 7:30 AM- 4:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chanh Nguyen can be reached at (571)272-7772. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRENT D CASTIAUX/ Primary Examiner, Art Unit 2623
Prosecution Insights