DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 6-15 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Edwin et al. (US 12,717,404, hereinafter Edwin), and further in view of Juenger (US 2018/0035102).
Regarding claim 1, Edwin teaches a head-wearable device (head-mounted device 10, fig. 2; col. 5 lines 30-32: devices 10 include a head-mounted device such as a pair of glasses (sometimes referred to as augmented reality glasses)), configured to be worn by a user (fig. 2 shows a pair of augmented reality glasses that could be worn by a user), for presenting an artificial- reality environment (col. 8 lines 26-43: In an augmented reality configuration, first waveguide 24B may also transmit (pass) real-world light from the scene/environment in front of (facing) device 10. The real-world light (sometimes referred to herein as world light, scene light, or environmental light) may include light emitted and/or reflected by objects in the scene/environment in front of device 10 ... This may allow images in image light 38B to be overlaid with world light 36 of real-world objects 34 (e.g., to overlay virtual objects from image data in image light 38B as displayed by first projector 22B with real-world objects 34 in front of the user when viewed at first eye box 20A)), the head-wearable device comprising:
one or more image-projection systems (fig. 2 projectors 22A and 22B; col. 5 lines 45-48: Projector 22 may include a first projector 22B (sometimes referred to herein as left projector 22B) and a second projector 22A (sometimes referred to herein as right projector 22A));
memory (storage such as nonvolatile memory or other electrically-programmable-read-only memory) and one or more processors (processing circuitry may be based on one or more processors; col. 3 lines 42-52: The control circuitry may include storage and processing circuitry for supporting the operation of system 8. The storage and processing circuitry may include storage such as nonvolatile memory (e.g., flash memory or other electrically-programmable-read-only memory configured to form a solid state drive), volatile memory (e.g., static or dynamic random-access-memory), etc. Processing circuitry in the control circuitry may be used to gather input from sensors and other input devices and may be used to control output devices. The processing circuitry may be based on one or more processors):
while an image is being presented to a user's first eye using a first image- projection system of the head-wearable device (col. 6 lines 53-56: first projector 22B may emit (e.g., produce, generate, project, or display) image light that is coupled into first waveguide 24B (e.g., by a first input coupler on first waveguide 24B); col. 9 lines 5-10: As shown in FIG. 4, image data 40B (e.g., a left image) may be produced by first projector 22B and may be directed to first eye box 20B by first waveguide 24B) and the image is being presented to a user's second eye using a second image-projection system of the head-wearable device (col. 6 lines 61-65: second projector 22A may emit (e.g., produce, generate, project, or display) image light that is coupled into second waveguide 24A (e.g., by a second input coupler on second waveguide 24A); col. 6 lines 41-49: Waveguides 24 may have input couplers that receive light from projectors 22. This image light is then guided laterally (along the X axis) within waveguides 24 in accordance with the principal of total internal reflection. Each waveguide 24 may have an output coupler in front of a respective eye box 20. The output coupler couples the image light out of the waveguide 24 and directs an image towards the associated eye box 20 for viewing by a user (e.g., a user whose eyes are located in eye boxes 20); col. 9 lines 5-10: Image data 40A (e.g., a right image) may be produced by second projector 22A and may be directed to second eye box 20A by second waveguide 24A):
making a determination, by the head-wearable device, that presenting a realignment pattern (col. 21 lines 58-61: Projector 22 may include a calibration pattern 184 in image light 38 that is displayed for one or more frames (e.g., in one or more frames of the image light 38 generated by projector 22)) at one or both of (i) a candidate point in time (time when the blink or saccade is about to begin or is currently in progress; col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye) and (ii) a candidate location within the image (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye; col. 23 lines 4-10: At operation 192, projector 22 may transmit calibration pattern 184 within a region 180 of FOV 176 that is different from (e.g., separated from, non-overlapping with, away from, etc.) the identified location 178 of the user's gaze within FOV 176. This may help to obfuscate the presence of calibration pattern 184 within image light 38 from being observed by the user) would result in the user not perceiving the realignment pattern due to one or both of the candidate point in time and the candidate location (while displaying the images, if the system determines a misalignment, then it is determined to display a hidden calibration pattern in the image light at a particular time or at a particular location in the image; col. 10 lines 23-30: Device 10 may perform in-field calibration operations using a set of sensors. In performing in-field calibration operations, the set of sensors may gather (e.g., measure, sense, or generate) sensor data that identifies the amount of optical misalignment in device 10. Control circuitry in device 10 may then perform adjustments to device 10 based on the identified amount of optical misalignment (e.g., to mitigate the identified amount of optical misalignment); col. 19 lines 16-23: In performing calibration operations using optical bridge sensor 112, projectors 22 (FIG. 2) may transmit a sequence (series) of one or more predetermined calibration patterns of image data in image light 38 (e.g., while processing operations 132-134 of FIG. 8). These calibration patterns may be used specifically for the purpose of calibrating optical alignment (e.g., without including other virtual objects intended for the user's view); col. 19 lines 34-43: If care is not taken, the calibration pattern can undesirably obscure the user's view of real-world objects in world light transmitted to eye box 20 through the waveguide and/or can undesirably distract the user from other virtual objects in image light 38 that are intended for the user to see (e.g., virtual objects associated with an application running on the device). It may therefore be desirable to be able to hide or obfuscate the calibration pattern within image light 38 such that the calibration pattern is unnoticeable or invisible to the user when the user's eyes are at eye boxes 20A and 20B; col. 19 lines 44-47: Device 10 may implement one or more gaze-based hiding/obfuscation techniques to help hide the calibration pattern from view based on sensor data captured by a gaze tracking sensor on device 10; col. 23 lines 11-15: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by transmitting calibration pattern 184 within the blind spot of eye 156; col. 23 lines 34-45: At operation 200, eye tracking sensor 150 may perform a pre-calibration operation to identify the location of blind spot 196 for the current user of device 10. This may involve, for example, gathering IR sensor data at one or more times (e.g., while the user looks in one or more predetermined directions) to identify the location of the user's blind spot 196. The control circuitry on device 10 may identify a relationship or mapping between the user's gaze direction (e.g., gaze vector 166) and the location of their blind spot 196. In this way, subsequent measurements of the user's gaze direction may be used to identify the location of the user's blind spot (e.g., in angle space) within the eye box; col. 23 lines 46-50: At operation 202, the control circuitry may identify the location of the user's blind spot 196 based on the sensor data generated by eye tracking sensor 150 (e.g., using the calibrated relationship between the user's gaze direction and the location of their blind spot 196); col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times);
selecting, based on the determination, one or both of (i) the candidate point in time as a selected point in time (time when the blink or saccade is about to begin or is currently in progress) at which to present the realignment pattern via the head-wearable device (col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye; col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times) and (ii) the candidate location as a selected location within the image (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye) at which the realignment pattern should be presented (col. 23 lines 11-15: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by transmitting calibration pattern 184 within the blind spot of eye 156; col. 23 lines 34-45: At operation 200, eye tracking sensor 150 may perform a pre-calibration operation to identify the location of blind spot 196 for the current user of device 10. This may involve, for example, gathering IR sensor data at one or more times (e.g., while the user looks in one or more predetermined directions) to identify the location of the user's blind spot 196. The control circuitry on device 10 may identify a relationship or mapping between the user's gaze direction (e.g., gaze vector 166) and the location of their blind spot 196. In this way, subsequent measurements of the user's gaze direction may be used to identify the location of the user's blind spot (e.g., in angle space) within the eye box; col. 23 lines 46-50: At operation 202, the control circuitry may identify the location of the user's blind spot 196 based on the sensor data generated by eye tracking sensor 150 (e.g., using the calibrated relationship between the user's gaze direction and the location of their blind spot 196); col. 23 lines 51-58: At operation 204, projector 22 may transmit some or all of the calibration dots 186 of calibration pattern 184 within a region of FOV 176 (FIG. 12) that is located at, within, or overlapping (e.g., partially or completely overlapping) the identified location of blind spot 196. As the user's eye has no sensitivity within this region, hiding the calibration pattern within blind spot 196 may prevent the user from being able to perceive or see calibration pattern 184; col. 24 lines 17-25: any desired adjustments to the timing and/or positioning of some or all of calibration pattern 184 may be performed in response to any desired features of eyes 156 as captured (imaged) by eye tracking sensor 150. If desired, image data provided to projectors 22 for generating image light 38 may be adjusted based on any desired features of eyes 156 as captured by eye tracking sensor 150. If desired, two or more of the implementations of FIGS. 12-16 may be combined);
presenting, via the head-wearable device, the realignment pattern (calibration pattern) at one or both of the selected point in time (time when the blink or saccade is about to begin or is currently in progress) and the selected location (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye) in accordance with the determining, such that the user does not perceive the realignment pattern (col. 23 lines 51-58: At operation 204, projector 22 may transmit some or all of the calibration dots 186 of calibration pattern 184 within a region of FOV 176 (FIG. 12) that is located at, within, or overlapping (e.g., partially or completely overlapping) the identified location of blind spot 196. As the user's eye has no sensitivity within this region, hiding the calibration pattern within blind spot 196 may prevent the user from being able to perceive or see calibration pattern 184; At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade)); and
modifying presentation characteristics (calibrating optical alignment) for the first image-projection system or the second image-projection system based on the presenting of the realignment pattern (col. 10 lines 48-58: The in-field calibration operations may serve to mitigate (e.g., calibrate, compensate for, or correct) optical misalignment that may be present in device 10, as shown by arrow 56. Such calibration may, for example, compensate for left-right binocular misalignment between the left and right displays (e.g., aligning image data 40A in second eye box 20A with nominal location 42) and/or may allow for proper registration of virtual objects with real-world objects (e.g., by properly registering virtual object 44 to real-world object 46, by properly registering virtual object 52 to real-world object 50, etc.); col. 15 lines 39-52: At operation 134, device 10 may adjust (e.g., correct, calibrate, alter, etc.) optical alignment between first projector 22B, second projector 22A, first waveguide 24B, and/or second waveguide 24A based on the position measurements and/or the optical bridge sensor image data. The adjustments may include adjustments to the image data displayed at first eye box 20B using the image light 38B produced by first projector 22B and/or adjustments to the image data displayed at second eye box 20A using the image light 38A produced by second projector 22A (e.g., image warping, geometric transforms, image distortion, image translations, etc.) and/or may include mechanical adjustments to one or more of first projector 22B, second projector 22A, first waveguide 24B, and/or second waveguide 24A; col. 19 lines 16-23: In performing calibration operations using optical bridge sensor 112, projectors 22 (FIG. 2) may transmit a sequence (series) of one or more predetermined calibration patterns of image data in image light 38 (e.g., while processing operations 132-134 of FIG. 8). These calibration patterns may be used specifically for the purpose of calibrating optical alignment (e.g., without including other virtual objects intended for the user's view)).
Although Edwin teaches storage and processing circuitry, wherein storage comprises memory such as nonvolatile memory or other electrically-programmable-read-only memory, Edwin does not explicitly teach one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, the one or more programs including instructions for providing calibration patterns to modify misalignment.
Juenger teaches one or more programs ([0059] and [0061]), wherein the one or more programs are stored in memory (computer-readable media 706, fig. 7 and [0061]) and configured to be executed by one or more processors (processing system 704, fig. 7 and [0056]), the one or more programs including instructions for providing alignment patterns for realignment of digital content ([0016], [0018] and [0093]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Juenger’s knowledge of using a processor to execute a program stored on a memory to realign stereoscopic digital data as taught and modify the system of Edwin because such a system is portable and reusable and enhances a user’s experience by increasing visual comfort ([0002]).
Regarding claim 2, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein: the head-wearable device further comprises one or more imaging devices (Juenger – fig. 1 camera(s) 120 and [0023]: sensors 114 are illustrated as including one or more cameras 120. In one or more implementations, the cameras 120 include at least a first camera mounted on a first side (e.g., the left side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a left image of the digital content 106, and a second camera mounted on a second side (e.g., a right side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a right image of the digital content 106), and the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head- wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected location as a location at which the realignment pattern would be within peripheral vision of the user (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV and is more than 80% away from the gaze direction within the FOV is functionally analogous to a location within the peripheral vision of the user; Edwin - col. 22 lines 23-39: For example, as shown in FIG. 12, control circuitry may identify, from the sensor data, that the user's eye has a gaze (e.g., gaze vector) oriented towards location 178 in FOV 176. Projector 22 may place calibration pattern 184 within a region 180 of FOV 176 that is located away from location 178 (the detected gaze direction). Region 180 may, for example, be non-overlapping with respect to location 178. If desired, region 180 may be separated from location 178 by at least distance 182 (e.g., a minimum angular separation) across FOV 176. Distance 182 may span 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, more than 80%, more than 50%, more than 20% or other percentages of the width of FOV 176, for example. By separating calibration pattern 180 from the direction of the user's gaze, the user will be less likely to notice the presence of calibration pattern 184 and thus calibration pattern 184 may be obfuscated for the user; Edwin - col. 23 lines 4-10: At operation 192, projector 22 may transmit calibration pattern 184 within a region 180 of FOV 176 that is different from (e.g., separated from, non-overlapping with, away from, etc.) the identified location 178 of the user's gaze within FOV 176. This may help to obfuscate the presence of calibration pattern 184 within image light 38 from being observed by the user; Juenger – [0016]: an alignment pattern that is configured to enable automatic realignment of the stereoscopic digital content may be generated within the stereoscopic digital content such that the alignment pattern is within the field of view of the viewer).
Regarding claim 3, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein: the head-wearable device further comprises one or more imaging devices (Juenger – fig. 1 camera(s) 120 and [0023]: sensors 114 are illustrated as including one or more cameras 120. In one or more implementations, the cameras 120 include at least a first camera mounted on a first side (e.g., the left side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a left image of the digital content 106, and a second camera mounted on a second side (e.g., a right side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a right image of the digital content 106), and the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head- wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected location as a location at which the realignment pattern would be within a blind spot of the user (location based on blind spot of eye is functionally analogous to a location within the peripheral vision of the user; Edwin - col. 23 lines 11-15: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by transmitting calibration pattern 184 within the blind spot of eye 156; Edwin - col. 23 lines 34-45: At operation 200, eye tracking sensor 150 may perform a pre-calibration operation to identify the location of blind spot 196 for the current user of device 10. This may involve, for example, gathering IR sensor data at one or more times (e.g., while the user looks in one or more predetermined directions) to identify the location of the user's blind spot 196. The control circuitry on device 10 may identify a relationship or mapping between the user's gaze direction (e.g., gaze vector 166) and the location of their blind spot 196. In this way, subsequent measurements of the user's gaze direction may be used to identify the location of the user's blind spot (e.g., in angle space) within the eye box; Edwin - col. 23 lines 46-50: At operation 202, the control circuitry may identify the location of the user's blind spot 196 based on the sensor data generated by eye tracking sensor 150 (e.g., using the calibrated relationship between the user's gaze direction and the location of their blind spot 196); Edwin - col. 23 lines 51-58: At operation 204, projector 22 may transmit some or all of the calibration dots 186 of calibration pattern 184 within a region of FOV 176 (FIG. 12) that is located at, within, or overlapping (e.g., partially or completely overlapping) the identified location of blind spot 196. As the user's eye has no sensitivity within this region, hiding the calibration pattern within blind spot 196 may prevent the user from being able to perceive or see calibration pattern 184).
Regarding claim 4, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein: the head-wearable device further comprises one or more imaging devices (Juenger – fig. 1 camera(s) 120 and [0023]: sensors 114 are illustrated as including one or more cameras 120. In one or more implementations, the cameras 120 include at least a first camera mounted on a first side (e.g., the left side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a left image of the digital content 106, and a second camera mounted on a second side (e.g., a right side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a right image of the digital content 106), and the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head- wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected point in time as a point in time during which the user's first eye or the user's second eye is blinking (time when the blink is about to begin or is currently in progress; Edwin - col. 20 lines 58-64: If desired, the control circuitry may identify when a user is blinking, beginning to blink, or is about to blink based on the sensor data (e.g., by comparing the sensor data to predetermined sensor data associated with the user blinking, beginning a blink, or a physiology when the user is about to blink). The control circuitry may also identify a duration of the blink or a typical duration with which the user blinks; Edwin - col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye; Edwin - col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times).
Regarding claim 6, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein: the head-wearable device further comprises one or more imaging devices (Juenger – fig. 1 camera(s) 120 and [0023]: sensors 114 are illustrated as including one or more cameras 120. In one or more implementations, the cameras 120 include at least a first camera mounted on a first side (e.g., the left side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a left image of the digital content 106, and a second camera mounted on a second side (e.g., a right side) of the housing 112 of computing device 102 that is configured to detect an alignment pattern in a right image of the digital content 106), and the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head- wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected point in time as a point in time during which the user's first eye and the user's second eye are performing a saccade (time when the saccade is about to begin or is currently in progress; Edwin – col. 20 line 64 – col. 21 line 10: If desired, the control circuitry may also identify the occurrence, beginning, or future occurrence of saccades of eyes 156 based on the sensor data (e.g., by comparing the sensor data to predetermined sensor data associated with the user's eyes performing a saccade, beginning a saccade, or a physiology associated with the eyes beginning a saccade in the future). Saccades are physiological occurrences where eyes 156 rapidly change orientation (e.g., jump) from a first orientation to a second orientation rather than smoothly and gradually changing angle from the first orientation to the second orientation. The control circuitry may also identify a duration of the saccade or a typical duration with which the user's eyes perform saccades; Edwin - col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye; Edwin - col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times).
Regarding claim 7, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected point in time as a point in time during which the user moves their head (Juenger – [0032]: as the viewer moves their head, or moves around the room, the alignment of the left and right digital images of digital content 106 may become misaligned; Juenger – [0033]: Thus, in accordance with various implementations, alignment module 126 is configured to re-align the left and right images of the digital content 106 automatically, which is often referred to as a “binocular adjustment”. In order to perform the realignment, alignment module 126 causes alignment patterns 128 to be generated within the digital content 106. In some cases, the alignment module 126 may generate the alignment patterns 128 at periodic time intervals, such as when the computing device 102 is powered on, and every 30 seconds thereafter. Alternately, alignment module 126 may be configured to detect conditions which may be indicative of misalignment, such as a sudden movement by the viewer 108, and thus perform the alignment in response to this detection).
Regarding claim 8, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the point in time as a boot-up period (period when powering on the computer device 102) of the head-wearable device (Juenger - [0033]: alignment module 126 causes alignment patterns 128 to be generated within the digital content 106. In some cases, the alignment module 126 may generate the alignment patterns 128 at periodic time intervals, such as when the computing device 102 is powered on, and every 30 seconds thereafter).
Regarding claim 9, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented are executed in accordance with a determination that the image, as presented to the user's first and second eyes, satisfies misalignment criteria (Edwin – col. 8 lines 59-63: If care is not taken, these changes in optical alignment can undesirably affect the images provided to eye boxes 20A and 20B (e.g., can produce visible misalignment at one or both eye boxes 20A and 20B); Edwin – col. 9 lines 22-40: when first projector 22B and/or first waveguide 24B become misaligned with respect to second projector 22A and/or second waveguide 24B, image data 40A may be received at second eye box 20A at a location other than nominal location 42, as shown in FIG. 4. This misalignment may present itself as a left-right binocular misalignment, causing virtual objects in image data 40A and/or 40B to appear blurry or misaligned between the eye boxes, or otherwise causing user discomfort when viewing both eye boxes 20A and 20B simultaneously. This left-right binocular misalignment may sometimes also be referred to herein as in-field drift (e.g., where virtual objects in one of the eye boxes drifts within the field of view from a nominal location due to misalignment between the left and right displays). In-field drift or other optical distortions may also be produced by misalignment or changes in alignment between first waveguide 24B and first projector 22B and misalignment between second waveguide 24A and second projector 22A; Edwin – col. 10 lines 11-22: When one or more of the OFCs becomes misaligned with respect to one or more of first projector 22B, first waveguide 24B, second projector 22A, and/or second waveguide 24A (e.g., with respect to the first and/or second display), this may cause the virtual objects in the image data of one or both eye boxes to become misaligned with the real-world objects that the virtual objects are registered to. For example, virtual object 46 in first eye box 20B may become misaligned with respect to real-world object 44, such as at location 48, and/or virtual object 52 in second eye box 20A may become misaligned with respect to real-world object 50, such as at location 54; Edwin - col. 10 lines 39-47: Performing in-field calibration operations in this way may allow device 10 to continue to exhibit proper optical alignment and thereby optimal display performance regardless of how the amount and type of optical misalignment present changes over time (e.g., due to mechanical stress effects and thermal effects on the system, how different users handle and operate the system, etc.); Edwin – col. 19 lines 16-33: In performing calibration operations using optical bridge sensor 112, projectors 22 (FIG. 2) may transmit a sequence (series) of one or more predetermined calibration patterns of image data in image light 38 (e.g., while processing operations 132-134 of FIG. 8). These calibration patterns may be used specifically for the purpose of calibrating optical alignment (e.g., without including other virtual objects intended for the user's view). Optical bridge sensor 112 may capture images of the calibration patterns as frames of optical bridge sensor image data (sometimes referred to herein simply as sensor data). Optical bridge sensor 112 (e.g., one or more processors) may compare the captured images of the calibration patterns to predetermined (reference) images of the calibration patterns (e.g., under predetermined ideal optical alignment conditions) and/or may compare the captured images of the calibration patterns transmitted by projectors 22A and 22B to identify and correct for optical misalignments in the system; Juenger – [0016]: Various environmental factors may cause the left and right images of the stereoscopic digital content to become misaligned while the viewer is using the computing device. Thus, an alignment pattern that is configured to enable automatic realignment of the stereoscopic digital content may be generated within the stereoscopic digital content such that the alignment pattern is within the field of view of the viewer. Generating the alignment pattern within the field of view of the viewer reduces the need to utilize additional display pixels (e.g., which are outside the field-of-view of the viewer) for the sole purpose of displaying the alignment pattern; Juenger – [0028]: The user experience manager module 104 is also illustrated as including an alignment module 126 that is configured to perform a “binocular alignment” of the digital content 106. When display device 116 is implemented as stereoscopic display 103, the alignment module 126 is configured to detect that the left and right images of the digital content 106 are misaligned. In response, the alignment module adjusts at least one of the left or right images of the digital content 106 in order to realign the digital content 106. Alternately, to realign the digital content 106, the alignment module 126 can guide a physical realignment of display device 116; Juenger – [0033]: Alternately, alignment module 126 may be configured to detect conditions which may be indicative of misalignment, such as a sudden movement by the viewer 108, and thus perform the alignment in response to this detection).
Regarding claim 10, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented are executed at predetermined periods of time (Edwin – col. 15 lines 6-9: Devices 10 can also use internal clocks in their control circuitry to measure the current time (e.g., to determine whether a predetermined time for making position sensor measurements has been reached); Juenger - every 30 seconds after the computing device is powered on is functionally analogous to predetermined periods of time; Juenger - [0033]: alignment module 126 causes alignment patterns 128 to be generated within the digital content 106. In some cases, the alignment module 126 may generate the alignment patterns 128 at periodic time intervals, such as when the computing device 102 is powered on, and every 30 seconds thereafter).
Regarding claim 11, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein: the head-wearable device further comprises an eye-tracking camera (Edwin – gaze tracking sensor on device 10, col. 19 lines 46-47; Edwin - fig. 10 shows device 10 comprising a gaze tracking sensor; Edwin – col. 19 lines 49-57: As shown in FIG. 10, device 10 may include an optical sensor such as eye tracking sensor 150. Eye tracking sensor 150 may sometimes also be referred to herein as gaze tracking sensor 150, gaze monitoring sensor 150, eye monitoring sensor 150, gaze sensor 150, or eye sensor 150. Eye tracking sensor 150 may include one or more optical emitters such as infrared (IR) emitter(s) 152 and one or more optical receivers (sensors) such as IR sensor(s) 158 (sometimes referred to herein as optical sensor(s) 158)), and the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented are based on data collected by the eye-tracking camera of the head-wearable device (Edwin – col. 19 lines 44-48: Device 10 may implement one or more gaze-based hiding/obfuscation techniques to help hide the calibration pattern from view based on sensor data captured by a gaze tracking sensor on device 10. FIG. 10 is a diagram showing how device 10 may include a gaze tracking sensor; Edwin – col. 21 lines 34-40: At operation 174, projector 22A and/or projector 22B may transmit a calibration pattern in image light 38 based on the sensor data generated at operation 172. The projector(s) may transmit the calibration pattern(s) at locations within the field of view (FOV) of image light 38 that are selected based on the sensor data and/or at times that are selected based on the sensor data gathered by eye tracking sensor 150; Edwin – col. 21 lines 48-56: As one example, projector 22 may transmit a calibration pattern in image light 38 (for receipt by optical bridge sensor 22) within a region of the FOV of image light 38 that is different from the user's current gaze direction as detected by eye tracking sensor 150. FIG. 12 is a front view (e.g., as viewed by a user at a corresponding eye box 20) showing how projector 22 may hide the calibration pattern within a region of the FOV of image light 38 that is different from the user's current gaze direction; Edwin – col. 23 lines 11-13: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150; Edwin – col. 23 lines 59-61: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150; Edwin – col. 24 lines 17-20: any desired adjustments to the timing and/or positioning of some or all of calibration pattern 184 may be performed in response to any desired features of eyes 156 as captured (imaged) by eye tracking sensor 150; Juenger - alignment pattern is selected to be displayed when a sudden movement by the viewer is detected; sudden movement by the viewer inherently includes head movements or eye movements such as blinking and saccades; Juenger – [0032]: as the viewer moves their head, or moves around the room, the alignment of the left and right digital images of digital content 106 may become misaligned; Juenger – [0033]: Thus, in accordance with various implementations, alignment module 126 is configured to re-align the left and right images of the digital content 106 automatically, which is often referred to as a “binocular adjustment”. In order to perform the realignment, alignment module 126 causes alignment patterns 128 to be generated within the digital content 106. In some cases, the alignment module 126 may generate the alignment patterns 128 at periodic time intervals, such as when the computing device 102 is powered on, and every 30 seconds thereafter. Alternately, alignment module 126 may be configured to detect conditions which may be indicative of misalignment, such as a sudden movement by the viewer 108, and thus perform the alignment in response to this detection).
Regarding claim 12, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein: the head-wearable device further comprises a disparity sensor (cameras 120, [0023] and [0034] - Juenger), and the instructions for modifying the presentation characteristics are based on an image of the realignment pattern (Juenger - [0018]: the digital content is realigned by adjusting at least one of the left and right images of the stereoscopic digital content based on the detected alignment patterns; Juenger - [0028]: the alignment module adjusts at least one of the left or right images of the digital content 106 in order to realign the digital content 106. Alternately, to realign the digital content 106, the alignment module 126 can guide a physical realignment of display device 116; Juenger - [0034]: After the alignment patterns 128 are generated, the cameras 120 are controlled to detect the alignment patterns 128 in each of the left and right images of the digital content 106. The location and position of the alignment patterns 128 may then be used to automatically realign the left and right images of the digital content 106; Juenger - [0046]: Alignment module 126 knows the position of the alignment patterns 128 within the digital content 106, and thus alignment module 126 can use the detected alignment patterns to calculate an amount of offset of the alignment pattern 128 in both of the left and right images of the digital content 106. Then, alignment module 126 uses the offset to adjust at least one of the left or right images of the digital content to perform the realignment; Juenger - [0053]: At 608, the stereoscopic digital content is realigned by adjusting at least one of the left and right images of the stereoscopic digital content based on the detected alignment patterns. For example, alignment module 126 adjusts at least one of the left and right images of the digital content 106 in order to align the images), wherein the image of the realignment pattern is captured by the disparity sensor of the head-wearable device (Juenger – [0029]: the alignment patterns 128 are configured such that they are detectable by the one or more cameras 120 of computing device 102; Juenger – [0034]: After the alignment patterns 128 are generated, the cameras 120 are controlled to detect the alignment patterns 128 in each of the left and right images of the digital content 106. The location and position of the alignment patterns 128 may then be used to automatically realign the left and right images of the digital content 106; Juenger – [0045]: after the alignment patterns 128 are generated, alignment module 126 controls cameras 120 to detect alignment patterns 128 in each of the left and right images of the digital content. For example, a first camera mounted on a left side of the housing 112 of computing device 102 can detect alignment patterns 128 in the left image of the digital content 106 rendered for the viewer's left eye, and a second camera mounted on a right side of the housing 112 can detect alignment patterns 128 in the right image of the digital content 106 rendered for the viewer's right eye. Alternately, rather than utilizing two cameras to detect the alignment patterns 128, a single camera 120 may utilize a prism in order to view the left and right images at the same time).
Regarding claim 13, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein the head-wearable device is a pair of artificial-reality glasses (Edwin - head-mounted device 10, fig. 2; Edwin - col. 5 lines 30-32: devices 10 include a head-mounted device such as a pair of glasses (sometimes referred to as augmented reality glasses); Edwin - col. 8 lines 26-43: In an augmented reality configuration, first waveguide 24B may also transmit (pass) real-world light from the scene/environment in front of (facing) device 10. The real-world light (sometimes referred to herein as world light, scene light, or environmental light) may include light emitted and/or reflected by objects in the scene/environment in front of device 10 ... This may allow images in image light 38B to be overlaid with world light 36 of real-world objects 34 (e.g., to overlay virtual objects from image data in image light 38B as displayed by first projector 22B with real-world objects 34 in front of the user when viewed at first eye box 20A); Juenger – [0019]: environment 100 includes a computing device 102 configured for use in augmented reality and/or virtual reality scenarios; Juenger – [0021]: the housing 112 is configured to be worn on the head of a viewer 108 (e.g., as a head-mounted display device 118), such as through configuration as goggles, glasses, contact lens, and so forth; Juenger – [0030]: FIG. 2 illustrates an example 200 of digital content that can be rendered for viewing in an augmented reality or virtual reality environment).
Regarding claim 14, the combination of Edwin and Juenger teaches the head-wearable device of claim 1, wherein the first and second image-projection systems each include at least one respective waveguide (Edwin – col. 5 lines 41-64: Images may be displayed in eye boxes 20 using displays 22 and waveguides 24. Displays 22 may sometimes be referred to herein as projectors 22, projector displays 22, display projectors 22, light projectors 22, image projectors 22, light engines 22, or display modules 22. Projector 22 may include a first projector 22B (sometimes referred to herein as left projector 22B) and a second projector 22A (sometimes referred to herein as right projector 22A). Projectors 22A and 22B may be mounted at opposing right and left edges of main portion 18M of housing 18, for example. Eye boxes 20 may include a first eye box 20B (sometimes referred to herein as left eye box 20B) and may include a second eye box 20A (sometimes referred to herein as right eye box 20A). Waveguides 24 may include a first waveguide 24B (sometimes referred to herein as left waveguide 24B) and a second waveguide 24A (sometimes referred to herein as right waveguide 24A). Main portion 18M of housing 18 may, for example, have a first portion that includes first projector 22B and first waveguide 24B and a second portion that includes second projector 22A and second waveguide 24A (e.g., where nose bridge NB separates the first and second portions such that the first portion is at a first side of the nose bridge and the second portion is at a second side of the nose bridge); Edwin – col. 6 lines 53-56: first projector 22B may emit (e.g., produce, generate, project, or display) image light that is coupled into first waveguide 24B (e.g., by a first input coupler on first waveguide 24B); Edwin – col. 6 lines 61-65: second projector 22A may emit (e.g., produce, generate, project, or display) image light that is coupled into second waveguide 24A (e.g., by a second input coupler on second waveguide 24A); Edwin – col. 9 lines 5-10: As shown in FIG. 4, image data 40B (e.g., a left image) may be produced by first projector 22B and may be directed to first eye box 20B by first waveguide 24. Image data 40A (e.g., a right image) may be produced by second projector 22A and may be directed to second eye box 20A by second waveguide 24A).
Claims 15 and 17 are similar in scope to claims 1 and 7 respectively, and therefore the examiner provides similar rationale to reject these claims. Moreover, the combination of Edwin and Juenger teaches a non-transitory computer-readable storage medium (Juenger - [0055] and [0057]).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Edwin, and further in view of Vlaskamp (US 2021/0181840).
Regarding claim 18, Edwin teaches a head-wearable device (head-mounted device 10, fig. 2; col. 5 lines 30-32: devices 10 include a head-mounted device such as a pair of glasses (sometimes referred to as augmented reality glasses)), configured to be worn by a user (fig. 2 shows a pair of augmented reality glasses that could be worn by a user):
while an image is being presented to a user's first eye using a first image- projection system of the head-wearable device (col. 6 lines 53-56: first projector 22B may emit (e.g., produce, generate, project, or display) image light that is coupled into first waveguide 24B (e.g., by a first input coupler on first waveguide 24B); col. 9 lines 5-10: As shown in FIG. 4, image data 40B (e.g., a left image) may be produced by first projector 22B and may be directed to first eye box 20B by first waveguide 24B) and the image is being presented to a user's second eye using a second image-projection system of the head-wearable device (col. 6 lines 61-65: second projector 22A may emit (e.g., produce, generate, project, or display) image light that is coupled into second waveguide 24A (e.g., by a second input coupler on second waveguide 24A); col. 6 lines 41-49: Waveguides 24 may have input couplers that receive light from projectors 22. This image light is then guided laterally (along the X axis) within waveguides 24 in accordance with the principal of total internal reflection. Each waveguide 24 may have an output coupler in front of a respective eye box 20. The output coupler couples the image light out of the waveguide 24 and directs an image towards the associated eye box 20 for viewing by a user (e.g., a user whose eyes are located in eye boxes 20); col. 9 lines 5-10: Image data 40A (e.g., a right image) may be produced by second projector 22A and may be directed to second eye box 20A by second waveguide 24A):
making a determination, by the head-wearable device, that presenting a realignment pattern (col. 21 lines 58-61: Projector 22 may include a calibration pattern 184 in image light 38 that is displayed for one or more frames (e.g., in one or more frames of the image light 38 generated by projector 22)) at one or both of (i) a candidate point in time (time when the blink or saccade is about to begin or is currently in progress; col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye) and (ii) a candidate location within the image (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye; col. 23 lines 4-10: At operation 192, projector 22 may transmit calibration pattern 184 within a region 180 of FOV 176 that is different from (e.g., separated from, non-overlapping with, away from, etc.) the identified location 178 of the user's gaze within FOV 176. This may help to obfuscate the presence of calibration pattern 184 within image light 38 from being observed by the user) would result in the user not perceiving the realignment pattern due to one or both of the candidate point in time and the candidate location (while displaying the images, if the system determines a misalignment, then it is determined to display a hidden calibration pattern in the image light at a particular time or at a particular location in the image; col. 10 lines 23-30: Device 10 may perform in-field calibration operations using a set of sensors. In performing in-field calibration operations, the set of sensors may gather (e.g., measure, sense, or generate) sensor data that identifies the amount of optical misalignment in device 10. Control circuitry in device 10 may then perform adjustments to device 10 based on the identified amount of optical misalignment (e.g., to mitigate the identified amount of optical misalignment); col. 19 lines 16-23: In performing calibration operations using optical bridge sensor 112, projectors 22 (FIG. 2) may transmit a sequence (series) of one or more predetermined calibration patterns of image data in image light 38 (e.g., while processing operations 132-134 of FIG. 8). These calibration patterns may be used specifically for the purpose of calibrating optical alignment (e.g., without including other virtual objects intended for the user's view); col. 19 lines 34-43: If care is not taken, the calibration pattern can undesirably obscure the user's view of real-world objects in world light transmitted to eye box 20 through the waveguide and/or can undesirably distract the user from other virtual objects in image light 38 that are intended for the user to see (e.g., virtual objects associated with an application running on the device). It may therefore be desirable to be able to hide or obfuscate the calibration pattern within image light 38 such that the calibration pattern is unnoticeable or invisible to the user when the user's eyes are at eye boxes 20A and 20B; col. 19 lines 44-47: Device 10 may implement one or more gaze-based hiding/obfuscation techniques to help hide the calibration pattern from view based on sensor data captured by a gaze tracking sensor on device 10; col. 23 lines 11-15: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by transmitting calibration pattern 184 within the blind spot of eye 156; col. 23 lines 34-45: At operation 200, eye tracking sensor 150 may perform a pre-calibration operation to identify the location of blind spot 196 for the current user of device 10. This may involve, for example, gathering IR sensor data at one or more times (e.g., while the user looks in one or more predetermined directions) to identify the location of the user's blind spot 196. The control circuitry on device 10 may identify a relationship or mapping between the user's gaze direction (e.g., gaze vector 166) and the location of their blind spot 196. In this way, subsequent measurements of the user's gaze direction may be used to identify the location of the user's blind spot (e.g., in angle space) within the eye box; col. 23 lines 46-50: At operation 202, the control circuitry may identify the location of the user's blind spot 196 based on the sensor data generated by eye tracking sensor 150 (e.g., using the calibrated relationship between the user's gaze direction and the location of their blind spot 196); col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times);
selecting, based on the determination, one or both of (i) the candidate point in time as a selected point in time (time when the blink or saccade is about to begin or is currently in progress) at which to present the realignment pattern via the head-wearable device (col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye; col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times) and (ii) the candidate location as a selected location within the image (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye) at which the realignment pattern should be presented (col. 23 lines 11-15: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by transmitting calibration pattern 184 within the blind spot of eye 156; col. 23 lines 34-45: At operation 200, eye tracking sensor 150 may perform a pre-calibration operation to identify the location of blind spot 196 for the current user of device 10. This may involve, for example, gathering IR sensor data at one or more times (e.g., while the user looks in one or more predetermined directions) to identify the location of the user's blind spot 196. The control circuitry on device 10 may identify a relationship or mapping between the user's gaze direction (e.g., gaze vector 166) and the location of their blind spot 196. In this way, subsequent measurements of the user's gaze direction may be used to identify the location of the user's blind spot (e.g., in angle space) within the eye box; col. 23 lines 46-50: At operation 202, the control circuitry may identify the location of the user's blind spot 196 based on the sensor data generated by eye tracking sensor 150 (e.g., using the calibrated relationship between the user's gaze direction and the location of their blind spot 196); col. 23 lines 51-58: At operation 204, projector 22 may transmit some or all of the calibration dots 186 of calibration pattern 184 within a region of FOV 176 (FIG. 12) that is located at, within, or overlapping (e.g., partially or completely overlapping) the identified location of blind spot 196. As the user's eye has no sensitivity within this region, hiding the calibration pattern within blind spot 196 may prevent the user from being able to perceive or see calibration pattern 184; col. 24 lines 17-25: any desired adjustments to the timing and/or positioning of some or all of calibration pattern 184 may be performed in response to any desired features of eyes 156 as captured (imaged) by eye tracking sensor 150. If desired, image data provided to projectors 22 for generating image light 38 may be adjusted based on any desired features of eyes 156 as captured by eye tracking sensor 150. If desired, two or more of the implementations of FIGS. 12-16 may be combined);
presenting, via the head-wearable device, the realignment pattern (calibration pattern) at one or both of the selected point in time (time when the blink or saccade is about to begin or is currently in progress) and the selected location (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye) in accordance with the determining, such that the user does not perceive the realignment pattern (col. 23 lines 51-58: At operation 204, projector 22 may transmit some or all of the calibration dots 186 of calibration pattern 184 within a region of FOV 176 (FIG. 12) that is located at, within, or overlapping (e.g., partially or completely overlapping) the identified location of blind spot 196. As the user's eye has no sensitivity within this region, hiding the calibration pattern within blind spot 196 may prevent the user from being able to perceive or see calibration pattern 184; At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade)); and
modifying presentation characteristics (calibrating optical alignment) for the first image-projection system or the second image-projection system based on the presenting of the realignment pattern (col. 10 lines 48-58: The in-field calibration operations may serve to mitigate (e.g., calibrate, compensate for, or correct) optical misalignment that may be present in device 10, as shown by arrow 56. Such calibration may, for example, compensate for left-right binocular misalignment between the left and right displays (e.g., aligning image data 40A in second eye box 20A with nominal location 42) and/or may allow for proper registration of virtual objects with real-world objects (e.g., by properly registering virtual object 44 to real-world object 46, by properly registering virtual object 52 to real-world object 50, etc.); col. 15 lines 39-52: At operation 134, device 10 may adjust (e.g., correct, calibrate, alter, etc.) optical alignment between first projector 22B, second projector 22A, first waveguide 24B, and/or second waveguide 24A based on the position measurements and/or the optical bridge sensor image data. The adjustments may include adjustments to the image data displayed at first eye box 20B using the image light 38B produced by first projector 22B and/or adjustments to the image data displayed at second eye box 20A using the image light 38A produced by second projector 22A (e.g., image warping, geometric transforms, image distortion, image translations, etc.) and/or may include mechanical adjustments to one or more of first projector 22B, second projector 22A, first waveguide 24B, and/or second waveguide 24A; col. 19 lines 16-23: In performing calibration operations using optical bridge sensor 112, projectors 22 (FIG. 2) may transmit a sequence (series) of one or more predetermined calibration patterns of image data in image light 38 (e.g., while processing operations 132-134 of FIG. 8). These calibration patterns may be used specifically for the purpose of calibrating optical alignment (e.g., without including other virtual objects intended for the user's view)).
Edwin does not explicitly teach a handheld intermediary processing device configured to process data for a head-wearable device, wherein the handheld intermediary processing device includes one or more programs including instructions for presenting an artificial reality environment at the head-wearable device, the instructions including providing realignment patterns at the selected location.
Vlaskamp teaches a handheld intermediary processing device (controller 460 or controller 1701 is functionally analogous to the handheld intermediary processing device; fig. 4 input device 466; fig. 17 shows user’s hand 1700 holding the controller 1701, [0307]: FIG. 17 is a perspective view of a user's hand 1700 and a controller 1701 including various input devices 1702, 1704, 1706. Any of the input devices 1702, 1704, 1706 may be configured to provide inputs regarding the vertical shift desired to align the alignment markers 1502 and 1504) configured to process data for a head- wearable device ([0194]: A controller 460 controls the operation of the stacked waveguide assembly 480 and the image injection devices 420, 422, 424, 426, 428. The controller 460 includes programming (e.g., instructions in a non-transitory computer-readable medium) that regulates the timing and provision of image information to the waveguides 440b, 438b, 436b, 434b, 432b. In some embodiments, the controller 460 may be a single integral device, or a distributed system connected by wired or wireless communication channels. The controller 460 may be part of the processing modules 260 or 270 (illustrated in FIG. 2); [0207]: The wearable system 400 may include a user input device 466 by which the user may input commands to the controller 460 to interact with the wearable system 400. For example, the user input device 466 may include a trackpad, a touchscreen, a joystick, a multiple degree-of-freedom (DOF) controller, a capacitive sensing device, a game controller, a keyboard, a mouse, a directional pad (D-pad), a wand, a haptic device, a totem (e.g., functioning as a virtual user input device), and so forth. A multi-DOF controller may sense user input in some or all possible translations (e.g., left/right, forward/backward, or up/down) or rotations (e.g., yaw, pitch, or roll) of the controller. A multi-DOF controller which supports the translation movements may be referred to as a 3DOF while a multi-DOF controller which supports the translations and rotations may be referred to as 6DOF. In some cases, the user may use a finger (e.g., a thumb) to press or swipe on a touch-sensitive input device to provide input to the wearable system 400 (e.g., to provide user input to a user interface provided by the wearable system 400). The user input device 466 may be held by the user's hand during the use of the wearable system 400. The user input device 466 may be in wired or wireless communication with the wearable system 400; [0209]: The user input device 466 (shown in FIG. 4) may be an embodiment of a totem, which may include a trackpad, a touchpad, a trigger, a joystick, a trackball, a rocker or virtual switch, a mouse, a keyboard, a multi-degree-of-freedom controller, or another physical input device. A user may use the totem, alone or in combination with poses, to interact with the wearable system or other users), wherein the handheld intermediary processing device includes one or more programs including instructions for presenting an artificial reality environment at the head-wearable device ([0194]: A controller 460 controls the operation of the stacked waveguide assembly 480 and the image injection devices 420, 422, 424, 426, 428. The controller 460 includes programming (e.g., instructions in a non-transitory computer-readable medium) that regulates the timing and provision of image information to the waveguides 440b, 438b, 436b, 434b, 432b; [0305]: The system may take the selected vertical positions of the markers 1502 and 1504 and/or the selected offset between the markers 1502 and 1504 into account when the user provides input to vertically align the markers 1502 and 1504. After the user provides input to vertically align the marks 1502 and 1504, the system may be able to determine the magnitude and direction of the left-right vertical misalignment based on the magnitude and direction of the user input. Preferably, adjustments to the vertical alignment is performed on only one display at a time; [0311]: At block 1810, the wearable system may perform a display alignment flow, which may include blocks such as block 1812 and 1814. At block 1812, the HMD may provide unfused left-eye and right-eye alignment markers. As an example, the HMD may display alignment marker 1502 of FIGS. 15 and 16 on a left-eye display and may display alignment marker 1504 of FIGS. 15 and 16 on a right-eye display, as discussed herein. As described above, in some examples, the HMD may display one or both of alignment markers 1502 and 1504 at randomly-, pseudorandomly-, or quasi-randomly-selected vertical positions. At block 1814, the wearable system may receive user feedback on left-eye or right-eye display vertical alignment adjustments. In particular and as discussed in connection with FIGS. 15-17, the wearable system may receive user inputs that shift at least one of the alignment markers 1502 and 1504 until the markers are vertically aligned with each other from the perspective of the user. Blocks 1812 and 1814 may continue until the user exits the alignment process or accepts any vertical adjustments they have made to the alignment markers. As described above, in some examples, the wearable system may conduct one display alignment process per waveguide included in the HMD. In these examples, the wearable system may perform the operations associated with one or more of blocks 1810, 1812, and 1814 for each waveguide included in the HMD), the instructions including providing realignment patterns at the selected location ([0082]: wherein to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, the at least one processor is configured to; [0083]: provide, with the left-eye display, the left-eye alignment marker at the first vertical position; [0084]: provide, with the right-eye display, the right-eye alignment marker at the second vertical position; [0085]: wherein to select the first vertical position at which to present the left-eye alignment marker and select the second vertical position at which to present the right-eye alignment marker, the at least one processor is configured to; [0086]: pseudo- or quasi-randomly select the first vertical position at which to present the left-eye alignment marker; [0303]: When a user wearing the HMD views the screens 1600a and 1600b simultaneously (e.g., with their left and right eyes respectively), the vertical alignment markers 1506a and 1506b may appear to the user as being fused together (e.g., as mark 1506). However, at least because the left-eye and right-eye horizontal alignment markers 1502 and 1504 are not spatially aligned to one another, the user does not perceive the marks 1502 and 1504 as being fused together; [0304]: alignment markers 1502 and 1504 are initially presented at the same positions of the left and right eye displays, though deformation may prompt perception of the positions as non-aligned. In some embodiments, the system may intentionally introduce a vertical offset between alignment markers 1502 and 1504. In such embodiments, the alignment of markers 1502 and 1504 may not necessarily be representative of the alignment of the left and right eye displays. For instance, although the left and right eye displays may exhibit relatively little or no misalignment, in these embodiments, the markers 1502 and 1504 presented by the system may exhibit a relatively high amount of misalignment. Doing so may serve to promote user engagement in the display alignment process). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Vlaskamp’s knowledge of using a handheld intermediary processing device and providing the left-eye and right-eye alignment markers on the selected positions on displayed imagery as taught and modify the system of Edwin because such a system uses user feedback to adjust content displayed through the system by compensating for any vertical alignment differences identified by the user, thus improving the user's comfort when viewing the HMD ([0160]).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Edwin, in view of Vlaskamp, and further in view of Juenger.
Regarding claim 20, the combination of Edwin and Vlaskamp does not explicitly teach the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected point in time as a point in time during which the user moves their head.
Juenger teaches the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected point in time as a point in time during which the user moves their head ([0032]: as the viewer moves their head, or moves around the room, the alignment of the left and right digital images of digital content 106 may become misaligned; [0033]: Thus, in accordance with various implementations, alignment module 126 is configured to re-align the left and right images of the digital content 106 automatically, which is often referred to as a “binocular adjustment”. In order to perform the realignment, alignment module 126 causes alignment patterns 128 to be generated within the digital content 106. In some cases, the alignment module 126 may generate the alignment patterns 128 at periodic time intervals, such as when the computing device 102 is powered on, and every 30 seconds thereafter. Alternately, alignment module 126 may be configured to detect conditions which may be indicative of misalignment, such as a sudden movement by the viewer 108, and thus perform the alignment in response to this detection). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Juenger’s knowledge of determining the selected point in time when to present the realignment pattern based the time when the user moves his/her head as taught and modify the system of Edwin and Vlaskamp because such a system enhances a user’s experience by increasing visual comfort ([0002]).
Claim(s) 5 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Edwin, in view of Juenger, and further in view of Hong (US 2013/0278779).
Regarding claim 5, the combination of Edwin and Juenger does not explicitly teach the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected location as at least one location within the image at which the realignment pattern would blend in with other image content.
Hong teaches the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected location as at least one location (locations where the calibration patterns becomes blended with the background of the image) within the image at which the realignment pattern would blend in with other image content ([0023]: While the calibration pattern 22 is clearly visible in the schematic representation of image 20', it is to be understood that the calibration pattern may become blended with the background. For example, the contrast of the calibration pattern 22 may be much lower than the contrast of the background due to the semi-transparent property of the screen 12). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Hong’s knowledge of lowering the contrast of the calibration pattern so blend it with the background of the image and modify the system of Edwin and Juenger because such a system temporally varies a calibration pattern during the capturing of an image sequence so that temporal correlation may be applied on the sequence of captured images, and thereby enabling the calibration pattern to be separated from the image of the background and features to be extracted from the calibration pattern. Such a robust feature detection enables automatic calibration of the projector-camera system including the semi-transparent screen ([0012]).
Claim 16 is similar in scope to claim 5, and therefore the examiner provides similar rationale to reject claim 16.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Edwin, in view of Vlaskamp, and further in view of Hong (US 2013/0278779).
Regarding claim 5, the combination of Edwin and Vlaskamp does not explicitly teach the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected location as at least one location within the image at which the realignment pattern would blend in with other image content.
Hong teaches the head-wearable device of claim 1, wherein the instructions for selecting one or both of (i) the selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the selected location within the image at which the realignment pattern should be presented include: determining the selected location as at least one location (locations where the calibration patterns becomes blended with the background of the image) within the image at which the realignment pattern would blend in with other image content ([0023]: While the calibration pattern 22 is clearly visible in the schematic representation of image 20', it is to be understood that the calibration pattern may become blended with the background. For example, the contrast of the calibration pattern 22 may be much lower than the contrast of the background due to the semi-transparent property of the screen 12). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Hong’s knowledge of lowering the contrast of the calibration pattern so blend it with the background of the image and modify the system of Edwin and Vlaskamp because such a system temporally varies a calibration pattern during the capturing of an image sequence so that temporal correlation may be applied on the sequence of captured images, and thereby enabling the calibration pattern to be separated from the image of the background and features to be extracted from the calibration pattern. Such a robust feature detection enables automatic calibration of the projector-camera system including the semi-transparent screen ([0012]).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-29 have been considered but are moot because the new ground of rejection does not rely on the same combination of references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Response to the argument that the cited references do not teach making a determination, by the head-wearable device, that presenting a realignment pattern at one or both of (i) a candidate point in time and (ii) a candidate location within the image would result in the user not perceiving the realignment pattern due to one or both of the candidate point in time and candidate location. See page 11 of Applicant’s Remarks filed on 6/3/2026.
Edwin (US 12,717,404) teaches making a determination, by the head-wearable device, that presenting a realignment pattern (col. 21 lines 58-61: Projector 22 may include a calibration pattern 184 in image light 38 that is displayed for one or more frames (e.g., in one or more frames of the image light 38 generated by projector 22)) at one or both of (i) a candidate point in time (time when the blink or saccade is about to begin or is currently in progress; col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye) and (ii) a candidate location within the image (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye; col. 23 lines 4-10: At operation 192, projector 22 may transmit calibration pattern 184 within a region 180 of FOV 176 that is different from (e.g., separated from, non-overlapping with, away from, etc.) the identified location 178 of the user's gaze within FOV 176. This may help to obfuscate the presence of calibration pattern 184 within image light 38 from being observed by the user) would result in the user not perceiving the realignment pattern due to one or both of the candidate point in time and the candidate location (while displaying the images, if the system determines a misalignment, then it is determined to display a hidden calibration pattern in the image light at a particular time or at a particular location in the image; col. 10 lines 23-30: Device 10 may perform in-field calibration operations using a set of sensors. In performing in-field calibration operations, the set of sensors may gather (e.g., measure, sense, or generate) sensor data that identifies the amount of optical misalignment in device 10. Control circuitry in device 10 may then perform adjustments to device 10 based on the identified amount of optical misalignment (e.g., to mitigate the identified amount of optical misalignment); col. 19 lines 16-23: In performing calibration operations using optical bridge sensor 112, projectors 22 (FIG. 2) may transmit a sequence (series) of one or more predetermined calibration patterns of image data in image light 38 (e.g., while processing operations 132-134 of FIG. 8). These calibration patterns may be used specifically for the purpose of calibrating optical alignment (e.g., without including other virtual objects intended for the user's view); col. 19 lines 34-43: If care is not taken, the calibration pattern can undesirably obscure the user's view of real-world objects in world light transmitted to eye box 20 through the waveguide and/or can undesirably distract the user from other virtual objects in image light 38 that are intended for the user to see (e.g., virtual objects associated with an application running on the device). It may therefore be desirable to be able to hide or obfuscate the calibration pattern within image light 38 such that the calibration pattern is unnoticeable or invisible to the user when the user's eyes are at eye boxes 20A and 20B; col. 19 lines 44-47: Device 10 may implement one or more gaze-based hiding/obfuscation techniques to help hide the calibration pattern from view based on sensor data captured by a gaze tracking sensor on device 10; col. 23 lines 11-15: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by transmitting calibration pattern 184 within the blind spot of eye 156; col. 23 lines 34-45: At operation 200, eye tracking sensor 150 may perform a pre-calibration operation to identify the location of blind spot 196 for the current user of device 10. This may involve, for example, gathering IR sensor data at one or more times (e.g., while the user looks in one or more predetermined directions) to identify the location of the user's blind spot 196. The control circuitry on device 10 may identify a relationship or mapping between the user's gaze direction (e.g., gaze vector 166) and the location of their blind spot 196. In this way, subsequent measurements of the user's gaze direction may be used to identify the location of the user's blind spot (e.g., in angle space) within the eye box; col. 23 lines 46-50: At operation 202, the control circuitry may identify the location of the user's blind spot 196 based on the sensor data generated by eye tracking sensor 150 (e.g., using the calibrated relationship between the user's gaze direction and the location of their blind spot 196); col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times).
Response to the argument that the cited references do not teach selecting, based on the determination, one or both of (i) the candidate point in time as a selected point in time at which to present the realignment pattern via the head-wearable device and (ii) the candidate location as a selected location within the image at which the realignment pattern should be presented, and presenting, via the head-wearable device, the realignment pattern at one or both of the selected point in time and the selected location in accordance with the determining, such that the user does not perceive the realignment pattern. See page 11 of Applicant’s Remarks.
Edwin (US 12,717,404) teaches selecting, based on the determination, one or both of (i) the candidate point in time as a selected point in time (time when the blink or saccade is about to begin or is currently in progress) at which to present the realignment pattern via the head-wearable device (col. 23 lines 59-64: projector 22 may transmit calibration pattern 184 based on a blink or saccade of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by synchronizing the transmission of calibration pattern 184 with a detected current or future blink or saccade of the user's eye; col 24 lines 3-15: At operation 210, the control circuitry may identify (detect) the beginning of a blink or saccade from the sensor data gathered by eye tracking sensor 150. The blink or saccade may be currently in progress or about to begin (e.g., at a future or approaching time). At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade). As the user's eye is rapidly moving during a saccade or is covered by the user's eyelids during a blink, the user will be unable to see or perceive calibration pattern 184 in image light 38 at these times) and (ii) the candidate location as a selected location within the image (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye) at which the realignment pattern should be presented (col. 23 lines 11-15: projector 22 may transmit calibration pattern 184 based on a blind spot of eye 156 as identified by eye tracking sensor 150. For example, projector 22 may hide calibration pattern 184 by transmitting calibration pattern 184 within the blind spot of eye 156; col. 23 lines 34-45: At operation 200, eye tracking sensor 150 may perform a pre-calibration operation to identify the location of blind spot 196 for the current user of device 10. This may involve, for example, gathering IR sensor data at one or more times (e.g., while the user looks in one or more predetermined directions) to identify the location of the user's blind spot 196. The control circuitry on device 10 may identify a relationship or mapping between the user's gaze direction (e.g., gaze vector 166) and the location of their blind spot 196. In this way, subsequent measurements of the user's gaze direction may be used to identify the location of the user's blind spot (e.g., in angle space) within the eye box; col. 23 lines 46-50: At operation 202, the control circuitry may identify the location of the user's blind spot 196 based on the sensor data generated by eye tracking sensor 150 (e.g., using the calibrated relationship between the user's gaze direction and the location of their blind spot 196); col. 23 lines 51-58: At operation 204, projector 22 may transmit some or all of the calibration dots 186 of calibration pattern 184 within a region of FOV 176 (FIG. 12) that is located at, within, or overlapping (e.g., partially or completely overlapping) the identified location of blind spot 196. As the user's eye has no sensitivity within this region, hiding the calibration pattern within blind spot 196 may prevent the user from being able to perceive or see calibration pattern 184; col. 24 lines 17-25: any desired adjustments to the timing and/or positioning of some or all of calibration pattern 184 may be performed in response to any desired features of eyes 156 as captured (imaged) by eye tracking sensor 150. If desired, image data provided to projectors 22 for generating image light 38 may be adjusted based on any desired features of eyes 156 as captured by eye tracking sensor 150. If desired, two or more of the implementations of FIGS. 12-16 may be combined); presenting, via the head-wearable device, the realignment pattern (calibration pattern) at one or both of the selected point in time (time when the blink or saccade is about to begin or is currently in progress) and the selected location (location within a region of the FOV that is different from the identified location of the user’s gaze within the FOV or location based on blind spot of eye) in accordance with the determining, such that the user does not perceive the realignment pattern (col. 23 lines 51-58: At operation 204, projector 22 may transmit some or all of the calibration dots 186 of calibration pattern 184 within a region of FOV 176 (FIG. 12) that is located at, within, or overlapping (e.g., partially or completely overlapping) the identified location of blind spot 196. As the user's eye has no sensitivity within this region, hiding the calibration pattern within blind spot 196 may prevent the user from being able to perceive or see calibration pattern 184; At operation 212, projector 22 may transmit calibration pattern 184 during the detected blink or saccade (e.g., at a time after the detected beginning of the blink or saccade and prior to completion of the expected duration of the blink or saccade)).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JWALANT B AMIN whose telephone number is (571)272-2455. The examiner can normally be reached Monday-Friday 10am - 630pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JWALANT AMIN/Primary Examiner, Art Unit 2612
Prosecution Insights