Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the application filed March 6, 2026, claims 1-19 are presented for examination. Claim 1 is an independent claim.
Priority
Examiner acknowledges the claims for domestic priority under 35 U.S. C. 119 (e) to provisional patent application 62686586, which was filed June 18, 2018.
Information Disclosure Statement
The Applicant’s Information Disclosure Statement filed (May 19, 2026) has been received, entered into the record, and considered.
Oath/Declaration
The Office acknowledges receipt of a properly signed Oath/Declaration submitted on March 6, 2026.
Drawings
The drawings filed March 6, 2026 are accepted by the examiner.
Abstract
The abstract filed March 6, 2026 is accepted by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the "right to exclude" granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428,46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046,29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Omum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CPR 3.73(b).
Claims 1-19 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-16 of application No. 18952446 Patent 12591135. Although the conflicting claims are not identical, they are not patentably distinct from each other because the claims recite A head-mounted display system configured to project light to an eye of a user to display augmented reality image content in a vision field of said user, said head-mounted display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display augmented reality image content to the user's vision field, at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears said frame such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user and said head-mounted display; one or more sensors; and processing electronics in communication with the display and the one or more sensors, the processing electronics configured to: receive from the one or more sensors data relating to movement of the frame; applying a machine learning technique to the received data to determine whether the frame of the head-mounted display is in a position for use on the head of the user with the head-mounted display positioned in front of the user's eye; and cause the head-mounted display system to reduce power of one or more components of the head-mounted display system in response to at least in part on a determination that the frame is not in position for use on the head of the user with the head-mounted display positioned in front of the user's eye, therefore the same limitations as claimed in application No. 18952446 Patent 12591135.
This is an obviousness-type double patenting rejection.
Application 19559828
Application 18952446 Patent 12591135
A head-mounted display system configured to project light to an eye of a user to display augmented reality image content in a vision field of said user, said head-mounted display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display augmented reality image content to the user's vision field, at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears said frame such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user and said head-mounted display; one or more sensors; and processing electronics in communication with the display and the one or more sensors, the processing electronics configured to: receive from the one or more sensors data relating to movement of the frame; applying a machine learning technique to the received data to determine whether the frame of the head-mounted display is in a position for use on the head of the user with the head-mounted display positioned in front of the user's eye; and cause the head-mounted display system to reduce power of one or more components of the head-mounted display system in response to at least in part on a determination that the frame is not in position for use on the head of the user with the head-mounted display positioned in front of the user's eye.
A head-mounted display system configured to project light to an eye of a user to display augmented reality image content in a vision field of said user, said head-mounted display system comprising: a frame configured to be supported on a head of the user in an upright position; a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display augmented reality image content to the user's vision field, at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears the frame in the upright position such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user and said head-mounted display; one or more sensors that comprise one or more inertial measurement units (IMUs) configured to measure change in the orientation of the frame; and processing electronics in communication with the display and the one or more sensors, the processing electronics configured to: receive a signal from the one or more sensors; determine whether the frame is in an upside-down position based at least in part on the signal received from the one or more sensors by: receiving measurements from the one or more IMUs; and analyzing the received measurements to determine whether the frame has been flipped over with respect to the upright position based at least in part on the measurements from the one or more IMUs, wherein the frame is in the upside-down position if the frame has been flipped over with respect to the upright position, and wherein the frame has been flipped over when the frame has been rotated from 160 degrees to 200 degrees with respect to the upright position; and cause the head-mounted display system to reduce power of one or more components of the head-mounted display system in response to at least in part on a determination that the frame is in the upside-down position.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9, 12-15, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over IDS submitted prior art Ishikawa et al US 20180239413 Al in view of IDS submitted prior art Latta et al. US 20140145914 Al in further view of IDS submitted prior art Nakada et al. US 20040104864 Al. and Aleem et al US 20190324532 A1.
As to Claim 1:
Ishikawa et al. discloses a head-mounted display system (Ishikawa, see figure 1) configured to project light to an eye of a user to display augmented reality image content in a vision field of said user (Ishikawa, see Abstract, where Ishikawa discloses that to effectively reduce power consumption of a wearable optical device by determining whether the wearable optical device is usable. Provided is an electronic apparatus including a wearable optical device, a state detection unit configured to detect a state relating to the wearable optical device, a state determination unit configured to determine that the detected state is at least one of a first state in which the wearable optical device is worn by a user in usable state or a second state in which the wearable optical device is worn or carried by the user in unusable state, and a power supply controller configured to control a power supply state of the electronic apparatus based on a result obtained by the determination), said head-mounted display system (Ishikawa, see 110 in figure 1) comprising: a device configured to be supported on a head of the user (Ishikawa, see paragraph [0003], where Ishikawa discloses a wearable optical device, and in particular, a device that is worn by the user on the head is known as a head-mounted display (HMD)); a head-mounted display (Ishikawa, see 110 in figure 10) disposed on the device (Ishikawa, see 114 in figure 10), said display configured to project light into said user's eye to display augmented reality image content to the user's vision field (Ishikawa, see paragraph [0003], where Ishikawa discloses an HMD capable of guiding the image display light in the lateral direction with respect to the viewer's pupils to be incident on the viewer's pupils); one or more sensors; and processing electronics in communication with the display and the one or more sensors, the processing electronics configured to: receive from the one or more sensors data relating to movement of the device (Ishikawa, see paragraph [0054], where Ishikawa discloses that when the three-axis acceleration sensor detects acceleration of the display unit 110, the detected acceleration contains a gravitational acceleration component (g = 9.8 m/s2). The gravitational acceleration component has a fixed direction (vertically downward), and thus the attitude of the display unit 110 can be specified on the basis of the relative direction of the gravitational acceleration component with respect to the detection axes (x-axis, y-axis, and z-axis) of the three-axis acceleration sensor); and cause the head-mounted display system to reduce power of one or more components of the head-mounted display system in response to at least in part on a determination that the device is not in position for use on the head of the user with the head-mounted display positioned in front of the user's eye (Ishikawa, see paragraph [0059], where Ishikawa discloses that when the direction of the gravitational acceleration component G contained in the acceleration detection value differs from the predetermined direction (negative direction of z-axis ), the state determination unit 510 is capable of detecting a second state in which the display unit 110 is worn or carried by the user in unusable state (the state shown in FIGS. 5(b) and 5(c)). Examples of the second state may include a state in which the display unit 110 is placed on a table or the like without being worn or carried by the user. Such a state is also treated in a way similar to the state in which the display unit 110 is worn or carried by the user in unusable state from a point of view that it is preferable to be set as a power saving state by power supply control).
Ishikawa differs from the claimed subject matter in that Ishikawa does not explicitly disclose at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears said frame such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user and said head-mounted display and applying a machine learning technique to the received data to determine whether the frame of the head-mounted display is in a position for use on the head of the user with the head-mounted display positioned in front of the user's eye.
However in analogous art, Latta discloses at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears said frame such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user and said head-mounted display (Latta, see figure 3 and paragraph [0017], where Latta discloses that with reference now also to FIG. 2, one example of an HMD device 200 in the form of a pair of wearable glasses with a transparent display 50 is provided. It will be appreciated that in other examples, the HMD device 200 may take other suitable forms in which a transparent, semi-transparent or non-transparent display is supported in front of a viewer's eye or eyes. It will also be appreciated that the HMD device 34 shown in FIG.1 may take the form of the HMD device 200, as described in more detail below, or any other suitable HMD device. Additionally, many other types and configurations of display devices having various form factors may also be used. For example, a hand-held display device that provides an augmented reality experience may also be used).
It would have been obvious to one of ordinary skill in the art to modify the invention of Ishikawa with Latta. One would be motivated to modify Ishikawa by disclosing at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears said frame such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user as taught by Latta, and thereby providing an acceptable battery life while allowing users to have a consistent high quality augmented reality experience (Latta, see paragraph [0002]). Latta does not explicitly disclose a frame and applying a machine learning technique to the received data to determine whether the frame of the head-mounted display is in a position for use on the head of the user with the head-mounted display positioned in front of the user's eye.
However in an analogous art, Nakada discloses a frame (Nakada, see paragraph [0015], where Nakada discloses that the glasses type display provides an open and closed state detecting means that detects the open and closed state of side frames of the glasses type display, a wearing state detecting means that detects the wearing state of the glasses type display by a user, a displaying means that is disposed in a main frame of the glasses type display and displays images, a voice and sound outputting means that outputs voice and sound, a voice and sound inputting means that inputs voice and sound, and a timer that measures a designated period).
It would have been obvious to one of ordinary skill in the art to modify the invention of Ishikawa and Latta with Nakada. One would be motivated to modify Ishikawa and Latta by disclosing a frame as taught by Nakada, and thereby the operation by the user is made simple by preventing error operation and the power consumption is decreased by not consuming the unnecessary power (Nakada, see paragraph [0014]). Nakada does not explicitly disclose applying a machine learning technique to the received data to determine whether the frame of the head-mounted display is in a position for use on the head of the user with the head-mounted display positioned in front of the user's eye.
However in an analogous art, Aleem discloses applying a machine learning technique to the received data to determine whether the frame of the head-mounted display is in a position for use on the head of the user with the head-mounted display positioned in front of the user's eye (Aleem, see paragraphs [0079] and [0092], where Aleem discloses that At 250, the WHUD is subjected to various types of motion over a given learning period. For example, learning data would be recorded while one or more training subjects move their head up and down and in circles, jog or run, and/or jump up and down. While the WHUD is subjected to various types of motion, the training marker is displayed at a select position in the display space (this select position should correspond to the position in 246 if the glint center position obtained at 248 is to be used as a reference glint center position), and the training subject focuses on the training marker. At 252, for each detected motion sensor output value MSi at least one glint is obtained from the edge detection process started at 244. For each glint, a glint center position GPi is determined. At 256, a first range of motion parameter values may be determined from MS, and oE, data for oE, that does not exceed an error threshold. Also, a second range of motion parameter values may be determined from MS, and oE, data for oE, that exceeds the error threshold. The error threshold may be based on the allowable eye tracking error for the WHUD. Thresholds or machine learning may be used to determine the first range of motion parameter values and second range of motion parameter values. The first range of motion parameters corresponds to when glint is stable enough that eye tracking in the first mode may be used. The second range of motion parameter values corresponds to when glint may be too unstable such that eye tracking in the second mode should be triggered). At 334, the eye tracking process sends the gaze position to the controller of the WHUD. The controller upon receiving the gaze position may adjust display content in a field of view of the user based on gaze position. Adjustment of display content may include any of modifying the display content, selecting a portion or all of the display content, moving a portion or all of the display content to where the user is looking, and removing a portion or all of the display content, e.g., if the user is not looking at a target area or anywhere within the display space. The eye tracking process returns to 314 to repeat determining if the WHUD is experiencing unstable movements, setting the eye tracking mode to the first mode or second mode, and performing eye tracking in the first mode or second mode. The eye tracking process may run until interrupted, e.g., by the controller of the WHUD or by powering down of the WHUD or by another external stimulus. Each of 314, 316, 318, 320, 322, 324, 326, 328, 330, 332, and 334 may be performed by a processor).
It would have been obvious to one of ordinary skill in the art to modify the invention of Ishikawa, Latta and Nakada with Aleem. One would be motivated to modify Ishikawa, Latta and Nakada by disclosing applying a machine learning technique to the received data to determine whether the frame of the head-mounted display is in a position for use on the head of the user with the head-mounted display positioned in front of the user's eye as taught by Aleem and thereby improving the utility of eye tracking in WHUDs by influencing where content is displayed in the field of view of the user, conserving power by not displaying content that is outside of the field of view of the user, influencing what content is displayed to the user, determining where the user is looking or gazing, determining whether the user is looking at displayed content on the display or at scenes in the external environment, and providing an interface through which the user may control or interact with displayed content (Aleem, see paragraph [0005]).
As to Claim 2:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 1, wherein the one or more sensors comprise one or more motion sensors configured to measure change in position of the frame (Ishikawa, see paragraph [0054], where Ishikawa discloses that when the three-axis acceleration sensor detects acceleration of the display unit 110, the detected acceleration contains a gravitational acceleration component (g = 9.8 m/s2). The gravitational acceleration component has a fixed direction (vertically downward), and thus the attitude of the display unit 110 can be specified on the basis of the relative direction of the gravitational acceleration component with respect to the detection axes (x-axis, y-axis, and z-axis) of the three-axis acceleration sensor).
As to Claim 3:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 2, wherein the one or more motion sensors comprise one or more inertial measurement units (IMUs) (Ishikawa, see paragraph [0054], where Ishikawa discloses that when the three-axis acceleration sensor detects acceleration of the display unit 110, the detected acceleration contains a gravitational acceleration component (g = 9.8 m/s2). The gravitational acceleration component has a fixed direction (vertically downward), and thus the attitude of the display unit 110 can be specified on the basis of the relative direction of the gravitational acceleration component with respect to the detection axes (x-axis, y-axis, and z-axis) of the three-axis acceleration sensor).
As to Claim 4:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 2, wherein the processing electronics is configured to determine whether the frame of the head-mounted display is in the position for use on the head of the user with the head-mounted display positioned in front of the user's eye by: receiving measurements from the one or more motion sensors; and analyzing the received measurements by applying the machine learning technique to the received measurements to determine whether the frame of the head-mounted display is in position for use on the head of the user with the head-mounted display positioned in front of the user's eye (Aleem, see paragraphs [0079] and [0092], where Aleem discloses that At 250, the WHUD is subjected to various types of motion over a given learning period. For example, learning data would be recorded while one or more training subjects move their head up and down and in circles, jog or run, and/or jump up and down. While the WHUD is subjected to various types of motion, the training marker is displayed at a select position in the display space (this select position should correspond to the position in 246 if the glint center position obtained at 248 is to be used as a reference glint center position), and the training subject focuses on the training marker. At 252, for each detected motion sensor output value MSi at least one glint is obtained from the edge detection process started at 244. For each glint, a glint center position GPi is determined. At 256, a first range of motion parameter values may be determined from MS, and oE, data for oE, that does not exceed an error threshold. Also, a second range of motion parameter values may be determined from MS, and oE, data for oE, that exceeds the error threshold. The error threshold may be based on the allowable eye tracking error for the WHUD. Thresholds or machine learning may be used to determine the first range of motion parameter values and second range of motion parameter values. The first range of motion parameters corresponds to when glint is stable enough that eye tracking in the first mode may be used. The second range of motion parameter values corresponds to when glint may be too unstable such that eye tracking in the second mode should be triggered). At 334, the eye tracking process sends the gaze position to the controller of the WHUD. The controller upon receiving the gaze position may adjust display content in a field of view of the user based on gaze position. Adjustment of display content may include any of modifying the display content, selecting a portion or all of the display content, moving a portion or all of the display content to where the user is looking, and removing a portion or all of the display content, e.g., if the user is not looking at a target area or anywhere within the display space. The eye tracking process returns to 314 to repeat determining if the WHUD is experiencing unstable movements, setting the eye tracking mode to the first mode or second mode, and performing eye tracking in the first mode or second mode. The eye tracking process may run until interrupted, e.g., by the controller of the WHUD or by powering down of the WHUD or by another external stimulus. Each of 314, 316, 318, 320, 322, 324, 326, 328, 330, 332, and 334 may be performed by a processor).
As to Claim 5:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 1, wherein the one or more sensors comprise an inward-facing imaging system and/or an outward facing imaging system (Ishikawa, see paragraph [0037], where Ishikawa discloses that the display unit 110 is provided with a motion sensor 116, and a camera 118 as shown in FIG. 2. The motion sensor 116 includes, for example, a triaxial acceleration sensor, a triaxial gyro sensor, and a triaxial geomagnetic sensor. Based on acceleration, an angular velocity, and a direction of the display unit 110 detected by the sensors, an attitude and a motion ( displacement and rotation) of the display unit 110 can be specified. The camera 118 photographs images of the real space. The images photographed by the camera 118 are treated as, for example, images corresponding to the visual field of the user in the real space)..
As to Claim 6:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 5, wherein the processing electronics is configured to determine whether the frame of the head-mounted display is in the position for use on the head of the user with the head-mounted display positioned in front of the user's eye by: receiving images in sequence from the inward-facing imaging system and/or the outward-facing imaging system; and analyzing the received images by applying the machine learning technique to the received measurements to determine whether the frame of the head-mounted display is in position for use on the head of the user with the head-mounted display positioned in front of the user's eye (Aleem, see paragraphs [0079] and [0092], where Aleem discloses that At 250, the WHUD is subjected to various types of motion over a given learning period. For example, learning data would be recorded while one or more training subjects move their head up and down and in circles, jog or run, and/or jump up and down. While the WHUD is subjected to various types of motion, the training marker is displayed at a select position in the display space (this select position should correspond to the position in 246 if the glint center position obtained at 248 is to be used as a reference glint center position), and the training subject focuses on the training marker. At 252, for each detected motion sensor output value MSi at least one glint is obtained from the edge detection process started at 244. For each glint, a glint center position GPi is determined. At 256, a first range of motion parameter values may be determined from MS, and oE, data for oE, that does not exceed an error threshold. Also, a second range of motion parameter values may be determined from MS, and oE, data for oE, that exceeds the error threshold. The error threshold may be based on the allowable eye tracking error for the WHUD. Thresholds or machine learning may be used to determine the first range of motion parameter values and second range of motion parameter values. The first range of motion parameters corresponds to when glint is stable enough that eye tracking in the first mode may be used. The second range of motion parameter values corresponds to when glint may be too unstable such that eye tracking in the second mode should be triggered). At 334, the eye tracking process sends the gaze position to the controller of the WHUD. The controller upon receiving the gaze position may adjust display content in a field of view of the user based on gaze position. Adjustment of display content may include any of modifying the display content, selecting a portion or all of the display content, moving a portion or all of the display content to where the user is looking, and removing a portion or all of the display content, e.g., if the user is not looking at a target area or anywhere within the display space. The eye tracking process returns to 314 to repeat determining if the WHUD is experiencing unstable movements, setting the eye tracking mode to the first mode or second mode, and performing eye tracking in the first mode or second mode. The eye tracking process may run until interrupted, e.g., by the controller of the WHUD or by powering down of the WHUD or by another external stimulus. Each of 314, 316, 318, 320, 322, 324, 326, 328, 330, 332, and 334 may be performed by a processor).
As to Claim 7:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 1, wherein the processing electronics is further configured to: determine whether the frame of the head-mounted display has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye; and cause the head-mounted display system to reduce the power of the one or more components of the head-mounted display system in response to at least in part on a determination that the frame of the head-mounted display has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye (Aleem, see paragraphs [0079] and [0092], where Aleem discloses that At 250, the WHUD is subjected to various types of motion over a given learning period. For example, learning data would be recorded while one or more training subjects move their head up and down and in circles, jog or run, and/or jump up and down. While the WHUD is subjected to various types of motion, the training marker is displayed at a select position in the display space (this select position should correspond to the position in 246 if the glint center position obtained at 248 is to be used as a reference glint center position), and the training subject focuses on the training marker. At 252, for each detected motion sensor output value MSi at least one glint is obtained from the edge detection process started at 244. For each glint, a glint center position GPi is determined. At 256, a first range of motion parameter values may be determined from MS, and oE, data for oE, that does not exceed an error threshold. Also, a second range of motion parameter values may be determined from MS, and oE, data for oE, that exceeds the error threshold. The error threshold may be based on the allowable eye tracking error for the WHUD. Thresholds or machine learning may be used to determine the first range of motion parameter values and second range of motion parameter values. The first range of motion parameters corresponds to when glint is stable enough that eye tracking in the first mode may be used. The second range of motion parameter values corresponds to when glint may be too unstable such that eye tracking in the second mode should be triggered). At 334, the eye tracking process sends the gaze position to the controller of the WHUD. The controller upon receiving the gaze position may adjust display content in a field of view of the user based on gaze position. Adjustment of display content may include any of modifying the display content, selecting a portion or all of the display content, moving a portion or all of the display content to where the user is looking, and removing a portion or all of the display content, e.g., if the user is not looking at a target area or anywhere within the display space. The eye tracking process returns to 314 to repeat determining if the WHUD is experiencing unstable movements, setting the eye tracking mode to the first mode or second mode, and performing eye tracking in the first mode or second mode. The eye tracking process may run until interrupted, e.g., by the controller of the WHUD or by powering down of the WHUD or by another external stimulus. Each of 314, 316, 318, 320, 322, 324, 326, 328, 330, 332, and 334 may be performed by a processor).
As to Claim 8:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 7, wherein the one or more sensors comprise an inward-facing imaging system configured to sense one or more facial features of the user (Ishikawa, see paragraph [0037], where Ishikawa discloses that the display unit 110 is provided with a motion sensor 116, and a camera 118 as shown in FIG. 2. The motion sensor 116 includes, for example, a triaxial acceleration sensor, a triaxial gyro sensor, and a triaxial geomagnetic sensor. Based on acceleration, an angular velocity, and a direction of the display unit 110 detected by the sensors, an attitude and a motion (displacement and rotation) of the display unit 110 can be specified. The camera 118 photographs images of the real space. The images photographed by the camera 118 are treated as, for example, images corresponding to the visual field of the user in the real space).
As to Claim 9:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 8, wherein the facial features include periocular features (Aleem, see paragraph [0004], where Aleem discloses that Eye tracking is a process by which one or more of position, orientation, and motion of an eye may be measured or monitored. In many applications, this is done with a view towards determining the gaze direction of a user. There are various techniques for measuring the position, orientation, and/or motion of the eye, the least invasive of which employs one or more optical sensors, e.g., cameras, to optically track the eye. Common techniques involve illuminating or flooding the eye with infrared light and measuring reflections from the eye with at least one optical sensor that is tuned to be sensitive to the infrared light. Information about how the infrared light is reflected from the eye is analyzed to determine the position, orientation, and/or motion of one or more eye features such as the cornea, pupil, iris, and/or retinal blood vessels, the office notes that periocular features has been interpreted as referring to the anatomical structures and skin characteristics located in the immediate area surrounding the human eye).
As to Claim 12:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 7, wherein the one or more sensors comprise one or more pressure sensors configured to sense pressure exerted by a portion of the user's head on the frame (Nakada, see paragraph [0010], where Nakada discloses that glasses type display provides a pressure sensor, which detects the wearing state of the display by the user, on the position where the nose of the user touches. When the user puts on the display, the pressure sensor detects the pressure and makes images display on the display, and when the user takes off the display, the power is turned off. With this, the power consumption is decreased by not consuming the unnecessary power).
As to Claim 13:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 12, wherein the one or more pressure sensors are configured to sense pressure on a temple, band, or nose pad of the frame (Aleem, see paragraph [0128], where Aleem discloses that the system 500 may include a motion sensor 596 ( corresponding to motion sensor 496 in FIG. 4A), such as an
inertial motion unit (IMU), accelerometer, gyroscope, and the like. Motion sensor 596 may be used to detect movements of front frame 508. In the example shown in FIG. 5, motion sensor 596 is coupled to an arm (or temple) 509a of the support frame 506, allowing motion sensor 596 to give
an indication of motion of the support frame 506. In general, motion sensor 596 may be mounted anywhere on arms 509a, 509b and front frame 508. Information from motion sensor 596 may be used to determine when to switch the eye tracking system from the first mode, where eye gaze is
tracked by glint center position, to the second mode, where eye gaze is tracked by glint-pupil vector).
As to Claim 14:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 12, wherein the one or more pressure sensors comprise at least one strain gauge (Ishikawa, see paragraph [0097], where Ishikawa discloses that the state detection unit may be configured to include a strain gauge used to detect the state in which the attachment member is elastically deformed and to determine that the state in which the attachment member is elastically deformed is the second state).
As to Claim 15:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 12, wherein the processing electronics is configured to determine whether the frame of the head-mounted display has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye by: receiving sensed pressures from the one or more pressure sensors; and analyzing the sensed pressures by applying the machine learning technique (Aleem, see paragraphs [0079] and [0092], where Aleem discloses that At 250, the WHUD is subjected to various types of motion over a given learning period. For example, learning data would be recorded while one or more training subjects move their head up and down and in circles, jog or run, and/or jump up and down. While the WHUD is subjected to various types of motion, the training marker is displayed at a select position in the display space (this select position should correspond to the position in 246 if the glint center position obtained at 248 is to be used as a reference glint center position), and the training subject focuses on the training marker. At 252, for each detected motion sensor output value MSi at least one glint is obtained from the edge detection process started at 244. For each glint, a glint center position GPi is determined. At 256, a first range of motion parameter values may be determined from MS, and oE, data for oE, that does not exceed an error threshold. Also, a second range of motion parameter values may be determined from MS, and oE, data for oE, that exceeds the error threshold. The error threshold may be based on the allowable eye tracking error for the WHUD. Thresholds or machine learning may be used to determine the first range of motion parameter values and second range of motion parameter values. The first range of motion parameters corresponds to when glint is stable enough that eye tracking in the first mode may be used. The second range of motion parameter values corresponds to when glint may be too unstable such that eye tracking in the second mode should be triggered) to the sensed pressures to indicate whether the frame is not in physical contact with the user's head, wherein the frame has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye if the sensed pressures indicate non-contact with the user's head (Ishikawa, see paragraph [0095] through [0097], where Ishikawa discloses that the switch 120 is provided at a portion of the buckle 128 and outputs a signal when the buckle 128 is unfastened as shown in FIGS. 12(a) and 12(b). The state determination unit 610, when acquiring this signal, determines that the display unit 110 is in the second state (the state in which the display unit 110 is worn or carried by the user in unusable state). Alternatively, when the buckle 128 is fastened, the switch outputs a signal. The state determination unit 610, when acquiring this signal, may determine that the display unit 110 is in the first state (the state in which the display unit 110 is worn by the user in usable state). As described above, in the present embodiment, the state of the display unit 110 is determined on the basis of the information indicating the state of the attachment member of the display unit 110 (a physical state of the wearable optical device). The power supply state of the display unit 110 or the HMD 100 including the display unit 110 is controlled on the basis of a determination result obtained by the determination. The control of the power supply state based on the state of the display unit 110 makes it possible to reduce power consumption without compromising the user's convenience by causing the function of the HMD 100 to be stopped while the user does not wear the display unit 110 in usable state. Although the example in which the attachment member has the connection part and the state detection unit detects the state of the connection part has been described in the present embodiment, the attachment member may not necessarily include the connection part in another embodiment. As one example, the wearable optical device may be mounted on the user's head by inserting a body part on the user's head or face into the attachment member that is elastically deformable. In this case, the state detection unit may be configured to include a strain gauge used to detect the state in which the attachment member is elastically deformed and to determine that the state in which the attachment member is elastically deformed is the second state.
As to Claim 17:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of claim 1, wherein the processing electronics is configured to cause the display system to reduce the power of the one or more components by dimming or turning off a light source of the display (Aleem, see paragraphs [0079] and [0092], where Aleem discloses that At 250, the WHUD is subjected to various types of motion over a given learning period. For example, learning data would be recorded while one or more training subjects move their head up and down and in circles, jog or run, and/or jump up and down. While the WHUD is subjected to various types of motion, the training marker is displayed at a select position in the display space (this select position should correspond to the position in 246 if the glint center position obtained at 248 is to be used as a reference glint center position), and the training subject focuses on the training marker. At 252, for each detected motion sensor output value MSi at least one glint is obtained from the edge detection process started at 244. For each glint, a glint center position GPi is determined. At 256, a first range of motion parameter values may be determined from MS, and oE, data for oE, that does not exceed an error threshold. Also, a second range of motion parameter values may be determined from MS, and oE, data for oE, that exceeds the error threshold. The error threshold may be based on the allowable eye tracking error for the WHUD. Thresholds or machine learning may be used to determine the first range of motion parameter values and second range of motion parameter values. The first range of motion parameters corresponds to when glint is stable enough that eye tracking in the first mode may be used. The second range of motion parameter values corresponds to when glint may be too unstable such that eye tracking in the second mode should be triggered). At 334, the eye tracking process sends the gaze position to the controller of the WHUD. The controller upon receiving the gaze position may adjust display content in a field of view of the user based on gaze position. Adjustment of display content may include any of modifying the display content, selecting a portion or all of the display content, moving a portion or all of the display content to where the user is looking, and removing a portion or all of the display content, e.g., if the user is not looking at a target area or anywhere within the display space. The eye tracking process returns to 314 to repeat determining if the WHUD is experiencing unstable movements, setting the eye tracking mode to the first mode or second mode, and performing eye tracking in the first mode or second mode. The eye tracking process may run until interrupted, e.g., by the controller of the WHUD or by powering down of the WHUD or by another external stimulus. Each of 314, 316, 318, 320, 322, 324, 326, 328, 330, 332, and 334 may be performed by a processor).
As to Claim 19:
Ishikawa in view of Latta in further view of Nakada and Aleem discloses that the display system of any of claim 1, wherein the processing electronics is configured to cause the display system to reduce the power of the one or more components by reducing the power of the one or more sensors (Aleem, see paragraphs [0079] and [0092], where Aleem discloses that At 250, the WHUD is subjected to various types of motion over a given learning period. For example, learning data would be recorded while one or more training subjects move their head up and down and in circles, jog or run, and/or jump up and down. While the WHUD is subjected to various types of motion, the training marker is displayed at a select position in the display space (this select position should correspond to the position in 246 if the glint center position obtained at 248 is to be used as a reference glint center position), and the training subject focuses on the training marker. At 252, for each detected motion sensor output value MSi at least one glint is obtained from the edge detection process started at 244. For each glint, a glint center position GPi is determined. At 256, a first range of motion parameter values may be determined from MS, and oE, data for oE, that does not exceed an error threshold. Also, a second range of motion parameter values may be determined from MS, and oE, data for oE, that exceeds the error threshold. The error threshold may be based on the allowable eye tracking error for the WHUD. Thresholds or machine learning may be used to determine the first range of motion parameter values and second range of motion parameter values. The first range of motion parameters corresponds to when glint is stable enough that eye tracking in the first mode may be used. The second range of motion parameter values corresponds to when glint may be too unstable such that eye tracking in the second mode should be triggered). At 334, the eye tracking process sends the gaze position to the controller of the WHUD. The controller upon receiving the gaze position may adjust display content in a field of view of the user based on gaze position. Adjustment of display content may include any of modifying the display content, selecting a portion or all of the display content, moving a portion or all of the display content to where the user is looking, and removing a portion or all of the display content, e.g., if the user is not looking at a target area or anywhere within the display space. The eye tracking process returns to 314 to repeat determining if the WHUD is experiencing unstable movements, setting the eye tracking mode to the first mode or second mode, and performing eye tracking in the first mode or second mode. The eye tracking process may run until interrupted, e.g., by the controller of the WHUD or by powering down of the WHUD or by another external stimulus. Each of 314, 316, 318, 320, 322, 324, 326, 328, 330, 332, and 334 may be performed by a processor).
Claim 18 are rejected under 35 U.S.C. 103 as being unpatentable over IDS submitted prior art Ishikawa et al US 20180239413 Al in view of IDS submitted prior art Latta et al. US 20140145914 Al in further view of IDS submitted prior art Nakada et al. US 20040104864 Al., Aleem et al US 20190324532 A1 and Kozuma et al. US 20180082118 A1
As to Claim 18:
Ishikawa in view of Latta in further view of Nakada and Aleem differs from the claimed subject matter in that Ishikawa in view of Latta in further view of Nakada and Aleem does not explicitly disclose that the display system of claim 1, wherein the processing electronics is configured to cause the display system to reduce the power of the one or more components by reducing a refresh rate associated with the display.
However in an analogous art, Kozuma discloses wherein the processing electronics is configured to cause the display system to reduce the power of the one or more components by reducing a refresh rate associated with the display (Kozuma, see paragraphs [0089] and [0311], where Kozuma discloses that the refresh rate of the image data based on the image data Vl can be lowered. Furthermore, power that is consumed to display a moving image captured by the display device can be reduced. Lowering the frame rate of a moving image displayed by the first pixel further reduces power consumption, which is preferable. The display system of one embodiment of the present invention can be used include display apparatuses, personal computers, image storage devices or image reproducing devices provided with recording media, mobile phones, game machines (including portable game consoles), portable data terminals, e-book readers, cameras such as video cameras and digital still cameras, goggle-type displays (head mounted displays)).
It would have been obvious to one of ordinary skill in the art to modify the invention of Ishikawa, Latta, Nakada and Aleem with Kozuma. One would be motivated to modify Ishikawa, Latta, Nakada and Aleem by disclosing applying wherein the processing electronics is configured to cause the display system to reduce the power of the one or more components by reducing a refresh rate associated with the display as taught by Kozuma and thereby providing an improved moving image display system that can stably display an image of a virtual object (Kozuma, see paragraph [0022]).
Allowable Subject Matter
Claims 10, 11 and 16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Referring to claim 10, the following is a statement of reasons for the indication of allowable subject matter: the prior art fail to suggest limitations “wherein the processing electronics is configured to determine whether the frame of the head-mounted display has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye by: receiving images from the inward-facing imaging system; and analyzing the images by applying the machine learning technique to the images to identify one or more periocular features, wherein the frame of the head-mounted display has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye if the one or more periocular features are not identified in the images”.
Referring to claim 11, the following is a statement of reasons for the indication of allowable subject matter: the prior art fail to suggest limitations “wherein the processing electronics is configured to determine whether the frame of the head-mounted display has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye by: receiving from the inward-facing imaging system a first image; receiving from the inward-facing imaging system a second image after the first image; and analyzing the first image and the second image by applying the machine learning technique to the first and second images to identify one or more periocular features, wherein the frame has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye if the one or more periocular features are identified in the first image and not identified in the second image”.
Referring to claim 16, the following is a statement of reasons for the indication of allowable subject matter: the prior art fail to suggest limitations “wherein the processing electronics is configured to determine whether the frame of the head-mounted display has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye by: receiving from the one or more pressure sensors a first sensed pressure; receiving from the one or more pressure sensors a second sensed pressure after the first sensed pressure; and analyzing the first sensed pressure and the second sensed pressure by applying the machine learning technique to the first and second sensed pressures to indicate whether the frame is in physical contact with the user's head, wherein the frame has been removed from the position for use in which the head-mounted display is positioned in front of the user's eye if the first sensed pressure indicates contact with the user's head and the second sensed pressure does not indicate contact with the user's head”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to
applicant's disclosure. Crow (US 20160025971 A1) discloses eyelid tracking on a computing device are disclosed. For example, one disclosed embodiment provides a head-mounted computing device comprising an image sensor positioned to acquire an image of an eyelid when worn on a head, a logic system, and a storage system. The storage system comprises instructions stored thereon that are executable by the logic system to capture image data of an eyelid via the image sensor, track a movement of the eyelid via the image data, track an eyelid state based upon the captured image data of the eyelid, and take an action on the computing device based upon the eyelid state.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NELSON ROSARIO whose telephone number is (571)270-1866. The examiner can normally be reached on Monday through Friday, 7:30am- 5:00pm EST. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Eason can be reached on (571) 270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NELSON M ROSARIO/ Primary Examiner, Art Unit 2624
