DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending regarding this application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/22/2025 are considered and attached.
Claim Objections
Claims 8 and 14 are objected to because of the following informalities:
Claim 8 recites “an image capturing device” in both line 4 and line 7, please change the recitation of “an image capturing device” as recited in line 7 to recite “the image capturing device”.
Claim 14 recites “a mixed reality headset g,”. This is believed to be a typo, please amend to recite “a mixed reality headset,”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, claim 1 recites “wherein at least one of the light source of the image capturing device”. However, the light source is introduced in the claim as “a light source” in line 2. As such, it is unclear how there can exist “at least one of” the light source, as the “at least one of” phrase directly implies a plurality of light source and the claim only introduces a single light source. Applicant discusses that there exists a plurality of light sources in the specification. See also claim 5 (dependent upon claim 1) which claims a plurality of light sources. However, since only one light source is claimed in claim 1, it remains unclear how there can be claimed “at least one of the light source” as claimed in line 4 of claim 1.
Claim 1 recites the limitation "the sclera" in line 8. There is insufficient antecedent basis for this limitation in the claim.
Corresponding independent claims 8 and 14 are similarly rejected. Claims 2-7, 9-13, and 15-20 are rejected due to their dependency upon rejected claims 1, 8, and 14.
Claim 3 recites “a change in the spatial coordinate”. However, claim 2, upon which claim 3 depends, also recites “a change in the spatial coordinate”. As such, it is unclear whether the change as introduced in claim 2 is equivalent to or distinct from the change as referenced in claim 3. Applicant discusses the above subject matter in para. [0023] and [0043] of the specification. However, nowhere in these sections does the applicant clarify whether the change in the spatial coordinate recited in claim 2 is equivalent to or distinct from the change in the spatial coordinate recited in claim 3. Therefore, claim 3 fails to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Corresponding claim 16 is similarly rejected.
Regarding claim 6, claim 6 recites “wherein generating a representation of the eye comprises training a machine learning model to estimate a location of a user’s pupil”. However, claim 1, upon which claim 6 depends, already recites “an eye of a user” in line 2 and “generating a representation of the eye” in lines 8-9. As such, it is unclear if the recitation of “a user’s pupil” and “a representation of the eye” in claim 6 is distinct from or equivalent to the recitation of “a user” and “generating a representation of the eye” as recited in claim 1. Applicant’s specification discusses this subject matter in para. [0023], [0025], [0030]-[0032] and [0044]. However, none of the aforementioned sections clarify whether the user or the representation of the eye as recited in claim 6 is distinct from or equivalent to the user and/or the representation of the eye as recited in claim 1. Therefore, claim 6 fails to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Corresponding claim 19 is similarly rejected.
Claim 7 recites “the light source comprising polarized light” in line 1. However, claim 1, upon which claim 7 depends, recites “the light source of the image capturing devices comprises a polarization element”. As such, it is unclear whether the light source comprising a polarization element as claimed in claim 1 is equivalent to the light source comprising polarized light as claimed in claim 7. Said differently, the phrase “polarized light” appears to be more narrow than the phrase “polarization element”, so it is unclear if applicant means to recite a second light source comprising polarized light, distinct from the light source comprising a polarization element, or if the light source comprising a polarization element is equivalent to the light source comprising polarized light. In regards to prior art searching, the claim will be interpreted as though the light source comprising a polarization element is equivalent to the light source comprising polarized light. If applicant intends for these two light sources to be equivalent, it is recommended to amend the aforementioned limitation in claim 7 to recite “wherein the polarization element of the light source is polarized light, and wherein the light source comprising the polarized light is […]”.
Corresponding claims 12 and 20 are similarly rejected.
Regarding claim 11, claim 11 recites “wherein the light source can comprise” in line 1. Here, it is unclear what the word “can” entails. In other words, does the light source have to comprise a polarization alternation feature, or is the light source simply able to comprise a polarization alternation feature. Applicant discusses the light source in the above context in para. [0027]. However, nowhere in this section does the applicant clarify the above lack of clarity regarding the scope of the word “can”. As such, claim 11 is rejected for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 3, 5, 8, 10, 14, 15, 16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Cavin et al. (U.S. Patent No. 10417784 B1), hereinafter Cavin, in view of Price et al. (U.S. Publication No. 2018/0329489 A1), hereinafter Price.
Regarding claim 1, Cavin teaches a method for eye tracking in a mixed reality headset (Cavin teaches an “artificial reality system that provides the artificial reality content may be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system” in col. 2, lines 34-40, wherein the HMD contains “an eye tracking system 215” as shown in col. 3, lines 15-18 and FIG. 2), the method comprising:
emitting a light source at an eye of a user (Cavin teaches that “the source assembly 225 illuminates a portion of the eye 220 with a light pattern 240 in accordance with tracking instructions (e.g., from the controller 235)” in col. 4, lines 30-32);
capturing light propagation of the light source with an image capturing device oriented in proximity to the eye of the user (Cavin teaches “the camera assembly 320 captures images of the light pattern reflected from the target area [(e.g., cornea, iris, and/or sclera)]”, in col. 6, lines 1-2. See also FIG. 2 wherein the camera 230 is oriented in proximity to an eye of a user), wherein at least one of the light source of the image capturing device comprises a polarization element (Cavin teaches “the camera assembly 230 includes one or more polarization sensitive cameras that are able to capture polarization information” in col. 6, lines 12-18);
identifying at least one (Cavin teaches that “the target area may also include some of the sclera of the eye 220” in col. 4, lines 55-65, wherein “the camera 230 captures images of the light pattern 240 reflected from the target area” in col. 5, lines 4-7. Here, since Cavin teaches capturing images of the reflected light pattern 240 from the target area (which may include some of the sclera of the eye), it is determined that Cavin teaches identifying at least one region of a user’s sclera);
in response to identifying the at least one (Cavin teaches “the position estimation module 360 generates a 3D approximation of a surface of the eye corresponding to the area of the eye illuminated by illumination assembly 310” in col. 7, lines 53-58, wherein the area of the eye illuminated by the illumination assembly includes the target area which includes the region of the user’s sclera); and
determining a gaze direction of the eye based on the representation of the eye (Cavin teaches “using the depth information the position estimation module 360 updates a 3D approximation of a portion of the eye. The position estimation module 360 may use the depth information to determine eye tracking information. Eye tracking information can include, e.g., position of an eye, gaze angle, inter-pupillary distance, etc.” in col. 7, line 66 through col. 8, line 12, wherein the gaze angle is interpreted as equivalent to the claimed gaze direction).
Cavin fails to teach identifying a textured region of the sclera.
However, Price teaches identifying a textured region of the sclera (Price teaches “eye image may be used to identify rough regions of the eye, such as sclera 314, iris 316, and pupil 318” in para. [0025]).
Cavin and Price are both considered to be analogous to the claimed invention because they are in the same field of tracking a user’s gaze through imaging techniques. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin to incorporate the teachings of Price and “identifying a textured region of the sclera”. The motivation for doing so would have been that “ocular features [] are located within the acquired image, and a gaze direction may be determined based on the locations of such features. Gaze direction computed in this manner may be used to navigate a graphical user-interface, to launch a program, make a selection of an on-screen object, move a character in a game, optimize graphical rendering for a limited angular resolution and so on”, as suggested by Price in para. [0010]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin with Price to obtain the invention specified in claim 1.
Regarding claim 2, Cavin and Price teach the method of claim 1,
further comprising tracking a spatial coordinate associated with the at least one textured region and determining a change in the spatial coordinate (Cavin teaches that, “in a time of flight operation, the position estimation module 360 uses captured pulses of light to disambiguate the locations of the user's eyes”, wherein the position estimation can include a gaze angle, in col. 8, lines 1-12. Here, Cavin teaches a “target area [] of the eye 220 that the eye tracking system 215 uses to track the eye 220” in col. 4, lines 55-56. Since the target area includes regions of the sclera (as shown in claim 1) it is inherent that changes in coordinates associated with the sclera/target region are utilized in the tracking process. See additionally that Price specifically teaches the textured region as shown in claim 1. Furthermore, since Cavin teaches updating the model M of the eye based on the position estimation and calibration parameters (see claims 1 and 3), it is clear that the coordinates of the target area of the eye are being tracked).
Similar motivations as applied to claim 1 can be applied here to claim 2.
Regarding claim 3, Cavin and Price teach the method of claim 2,
further comprising, in response to determining a change in the spatial coordinate, adjusting the representation of the eye (Cavin teaches that “the calibration module 370 continues to update M during tracking” in col. 8, lines 31-39, wherein M is interpreted as equivalent to the claimed representation, and the update is based on the updated locations of the target area as shown in col. 8, lines 1-12).
Regarding claim 5, Cavin and Price teach the method of claim 1,
further comprising emitting light toward the eye of the user from a plurality of light sources (Cavin teaches “the source assembly 225 illuminates a portion of the eye 220 with a light pattern 240 in accordance with tracking instructions (e.g., from the controller 235). In some embodiments, the source assembly 225 includes a plurality of emitters (e.g., may act as individual point sources), which emit light in the infrared (IR) band (˜750 nm to 1700 nm)” in col. 4, lines 30-35, wherein “the illumination assembly 310 illuminates some or all of one or both eyes of a user with a light pattern in accordance with tracking instructions. The illumination assembly 310 includes one or more source assemblies (e.g., the source assembly 225)” as shown in col. 5, lines 45-50).
Regarding claim 8, Cavin teaches a system for eye tracking in a mixed reality headset (Cavin teaches an “artificial reality system that provides the artificial reality content may be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system” in col. 2, lines 34-40, wherein the HMD contains “an eye tracking system 215” as shown in col. 3, lines 15-18 and FIG. 2):
a headset comprising (Cavin teaches “a head-mounted display (HMD) connected to a host computer system” or a “standalone HMD” in col. 2, lines 34-40. See FIG. 1):
a light source (Cavin teaches that “the source assembly 225 illuminates a portion of the eye 220 with a light pattern 240” in col. 4, lines 30-32);
an image capturing device comprising a polarized filter (Cavin teaches “the camera assembly 230 includes one or more polarization sensitive cameras that are able to capture polarization information” in col. 6, lines 12-18); and
one or more hardware processors configured by machine-readable instructions (Cavin teaches that the eye tracking system 300 resides in the HMD (see FIG. 7), wherein the eye tracking system 330 comprises “a software module implemented on one or more processors, a dedicated hardware unit, or some combination thereof” as further shown in FIG. 3 and col. 7, lines 49-51) to:
emit a light source at an eye of a user (Cavin teaches that “the source assembly 225 illuminates a portion of the eye 220 with a light pattern 240 in accordance with tracking instructions (e.g., from the controller 235)” in col. 4, lines 30-32);
capture light propagation of the light source with an image capturing device oriented in proximity to the eye of the user (Cavin teaches “the camera assembly 320 captures images of the light pattern reflected from the target area [(e.g., cornea, iris, and/or sclera)]”, in col. 6, lines 1-2. See also FIG. 2 wherein the camera 230 is oriented in proximity to an eye of a user), wherein at least one of the light source of the image capturing device comprises a polarization element (Cavin teaches “the camera assembly 230 includes one or more polarization sensitive cameras that are able to capture polarization information” in col. 6, lines 12-18);
identify at least one (Cavin teaches that “the target area may also include some of the sclera of the eye 220” in col. 4, lines 55-65, wherein “the camera 230 captures images of the light pattern 240 reflected from the target area” in col. 5, lines 4-7. Here, since Cavin teaches capturing images of the reflected light pattern 240 from the target area (which may include some of the sclera of the eye), it is determined that Cavin teaches identifying at least one region of a user’s sclera);
in response to identifying the at least one (Cavin teaches “the position estimation module 360 generates a 3D approximation of a surface of the eye corresponding to the area of the eye illuminated by illumination assembly 310” in col. 7, lines 53-58, wherein the area of the eye illuminated by the illumination assembly includes the target area which includes the region of the user’s sclera); and
determine a gaze direction of the eye based on the representation of the eye (Cavin teaches “using the depth information the position estimation module 360 updates a 3D approximation of a portion of the eye. The position estimation module 360 may use the depth information to determine eye tracking information. Eye tracking information can include, e.g., position of an eye, gaze angle, inter-pupillary distance, etc.” in col. 7, line 66 through col. 8, line 12, wherein the gaze angle is interpreted as equivalent to the claimed gaze direction).
Cavin fails to teach identifying a textured region of the sclera.
However, Price teaches identifying a textured region of the sclera (Price teaches “eye image may be used to identify rough regions of the eye, such as sclera 314, iris 316, and pupil 318” in para. [0025]).
Cavin and Price are both considered to be analogous to the claimed invention because they are in the same field of tracking a user’s gaze through imaging techniques. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin to incorporate the teachings of Price and “identifying a textured region of the sclera”. The motivation for doing so would have been that “ocular features [] are located within the acquired image, and a gaze direction may be determined based on the locations of such features. Gaze direction computed in this manner may be used to navigate a graphical user-interface, to launch a program, make a selection of an on-screen object, move a character in a game, optimize graphical rendering for a limited angular resolution and so on”, as suggested by Price in para. [0010]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin with Price to obtain the invention specified in claim 8.
Regarding claim 10, Cavin and Price teach the system of claim 8,
further comprising a point scanning sensor (Cavin teaches “the light pattern emitted by the illumination assembly 310 includes a plurality of features that are uniquely identifiable. In one embodiment, a feature is light emitted by a point source light emitter” in col. 7, lines 56-60, wherein “the source assembly 225 includes a plurality of emitters (e.g., may act as individual point sources)” in col. 4, lines 33-34) and
at least one of
a micro-electromechanical system (MEMs) or
liquid crystal device-based scanner (Cavin teaches “the electronic display element 205 emits image light toward the optics block 210. Examples of the electronic display element 205 include: a liquid crystal display (LCD)” in col. 3, lines 29-35).
Regarding claim 14, Cavin teaches a non-transient computer-readable storage medium having instructions embodied thereon, the instructions being executable by one or more processors to perform a method for eye tracking in a mixed reality headset g (Cavin teaches “this apparatus [] may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability” in col. 15, lines 17-23, wherein Cavin teaches an “artificial reality system that provides the artificial reality content may be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system” or a standalone HMD in col. 2, lines 34-40, wherein the HMD contains “an eye tracking system 215” as shown in col. 3, lines 15-18 and FIG. 2), the method comprising:
emitting a light source at an eye of a user (Cavin teaches that “the source assembly 225 illuminates a portion of the eye 220 with a light pattern 240 in accordance with tracking instructions (e.g., from the controller 235)” in col. 4, lines 30-32);
capturing light propagation of the light source with an image capturing device oriented in proximity to the eye of the user (Cavin teaches “the camera assembly 320 captures images of the light pattern reflected from the target area [(e.g., cornea, iris, and/or sclera)]”, in col. 6, lines 1-2. See also FIG. 2 wherein the camera 230 is oriented in proximity to an eye of a user), wherein at least one of the light source of the image capturing device comprises a polarization element (Cavin teaches “the camera assembly 230 includes one or more polarization sensitive cameras that are able to capture polarization information” in col. 6, lines 12-18);
identifying at least one (Cavin teaches that “the target area may also include some of the sclera of the eye 220” in col. 4, lines 55-65, wherein “the camera 230 captures images of the light pattern 240 reflected from the target area” in col. 5, lines 4-7. Here, since Cavin teaches capturing images of the reflected light pattern 240 from the target area (which may include some of the sclera of the eye), it is determined that Cavin teaches identifying at least one region of a user’s sclera);
in response to identifying the at least one (Cavin teaches “the position estimation module 360 generates a 3D approximation of a surface of the eye corresponding to the area of the eye illuminated by illumination assembly 310” in col. 7, lines 53-58, wherein the area of the eye illuminated by the illumination assembly includes the target area which includes the region of the user’s sclera); and
determining a gaze direction of the eye based on the representation of the eye (Cavin teaches “using the depth information the position estimation module 360 updates a 3D approximation of a portion of the eye. The position estimation module 360 may use the depth information to determine eye tracking information. Eye tracking information can include, e.g., position of an eye, gaze angle, inter-pupillary distance, etc.” in col. 7, line 66 through col. 8, line 12, wherein the gaze angle is interpreted as equivalent to the claimed gaze direction).
Cavin fails to teach identifying a textured region of the sclera.
However, Price teaches identifying a textured region of the sclera (Price teaches “eye image may be used to identify rough regions of the eye, such as sclera 314, iris 316, and pupil 318” in para. [0025]).
Cavin and Price are both considered to be analogous to the claimed invention because they are in the same field of tracking a user’s gaze through imaging techniques. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin to incorporate the teachings of Price and “identifying a textured region of the sclera”. The motivation for doing so would have been that “ocular features [] are located within the acquired image, and a gaze direction may be determined based on the locations of such features. Gaze direction computed in this manner may be used to navigate a graphical user-interface, to launch a program, make a selection of an on-screen object, move a character in a game, optimize graphical rendering for a limited angular resolution and so on”, as suggested by Price in para. [0010]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin with Price to obtain the invention specified in claim 14.
Regarding claim 15, Cavin and Price teach the non-transient computer-readable storage medium of claim 14,
further comprising tracking a spatial coordinate associated with the at least one textured region and determining a change in the spatial coordinate (Cavin teaches that, “in a time of flight operation, the position estimation module 360 uses captured pulses of light to disambiguate the locations of the user's eyes”, wherein the position can include gaze angle, in col. 8, lines 1-12. Here, Cavin teaches a “target area [] of the eye 220 that the eye tracking system 215 uses to track the eye 220” in col. 4, lines 55-56. Since the target area includes regions of the sclera (as shown in claim 1) it is inherent that changes in coordinates associated with the sclera/target region are utilized in the tracking process. See additionally that Price specifically teaches the textured region as shown in claim 1).
Similar motivations as applied to claim 14 can be applied here to claim 15.
Regarding claim 16, Cavin and Price teach the non-transient computer-readable storage medium of claim 15,
further comprising, in response to determining a change in the spatial coordinate, adjusting the representation of the eye (Cavin teaches that “the calibration module 370 continues to update M during tracking” in col. 8, lines31-39, wherein M is interpreted as equivalent to the claimed representation, and the update is based on the updated locations of the target area as shown in col. 8, lines 1-12).
Regarding claim 18, Cavin and Price teach the non-transient computer-readable storage medium of claim 14,
further comprising emitting light toward the eye of the user from a plurality of light sources (Cavin teaches “the source assembly 225 illuminates a portion of the eye 220 with a light pattern 240 in accordance with tracking instructions (e.g., from the controller 235). In some embodiments, the source assembly 225 includes a plurality of emitters (e.g., may act as individual point sources), which emit light in the infrared (IR) band (˜750 nm to 1700 nm)” in col. 4, lines 30-35, wherein “the illumination assembly 310 illuminates some or all of one or both eyes of a user with a light pattern in accordance with tracking instructions. The illumination assembly 310 includes one or more source assemblies (e.g., the source assembly 225)” as shown in col. 5, lines 45-50).
Claims 4, 7, 9, 11, 12, 13, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Cavin et al. (U.S. Patent No. 10417784 B1), hereinafter Cavin, in view of Price et al. (U.S. Publication No. 2018/0329489 A1), hereinafter Price and Lu et al. (U.S. Publication No. 2020/0018962 A1), hereinafter Lu.
Regarding claim 4, Cavin and Price teach the method of claim 1,
further comprising receiving image data from a plurality of image capturing devices (Cavin teaches that “the camera assembly 320 includes one or more cameras” in col. 6, lines 5-8).
Cavin and Price fail to teach wherein at least one image capturing device of the plurality of image capturing devices is configured with a polarization angle that differs from another image capturing device of the plurality of image capturing devices.
However, Lu teaches wherein at least one image capturing device of the plurality of image capturing devices is configured with a polarization angle that differs from another image capturing device of the plurality of image capturing devices (Lu teaches “a lens assembly including two or more polarization-dependent liquid crystal (LC) lenses sensitive to either linear or circular polarization and having same or different optical powers can be used to project a displayed image on one of multiple image planes that are at different distances from the user's eyes. In some embodiments, the lens assembly may also include a polarizer, such as a linear polarizer or circular polarizer, and a polarization converter which may rotate linearly polarized light or change the handedness of circularly polarized light” in para. [0043], wherein “the alignment direction of the first LC lens may be θ, while the alignment direction of the second LC lens may be θ+90°” as shown in para. [0044]).
Cavin, Price, and Lu are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Lu and include “wherein at least one image capturing device of the plurality of image capturing devices is configured with a polarization angle that differs from another image capturing device of the plurality of image capturing devices”. The motivation for doing so would have been that “eye-tracking unit 130 may be arranged to increase contrast in images of an eye captured by eye-tracking unit 130 while reducing the overall power consumed by eye-tracking unit 130 (e.g., reducing power consumed by a light emitter and an imaging system included in eye-tracking unit 130 )” and “virtual images may be formed on different image planes by turning on or off the switchable polarization converter”, as suggested by Lu in para. [0065] and [0155], respectively. See also para. [0131]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Lu to obtain the invention specified in claim 4.
Regarding claim 7, Cavin and Price teach the method of claim 2.
While Cavin teaches “a polarization sensitive camera comprises an array of polarization sensitive pixels (e.g., 0°, 45°, 90°, and 135°)” in col. 6, lines 16-18, Cavin and Price fail to teach wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees.
However, Lu teaches wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees (Lu teaches utilizing a “linear polarization rotator” for determining gaze direction of a user as shown in para. [0066] and [0122]-[0123]. Lu further teaches that “the linear polarization rotator is configured to rotate the polarization direction of linearly polarized light” wherein the linear polarization rotator may have an angle of 45 degrees as shown in para. [0123]).
Cavin, Price, and Lu are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Lu and include “wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees”. The motivation for doing so would have been that “eye-tracking unit 130 may be arranged to increase contrast in images of an eye captured by eye-tracking unit 130 while reducing the overall power consumed by eye-tracking unit 130 (e.g., reducing power consumed by a light emitter and an imaging system included in eye-tracking unit 130 )” and “virtual images may be formed on different image planes by turning on or off the switchable polarization converter”, as suggested by Lu in para. [0065] and [0155], respectively. See also para. [0131]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Lu to obtain the invention specified in claim 7.
Regarding claim 9, Cavin and Price teach the system of claim 8.
While Cavin teaches “a polarization sensitive camera comprises an array of polarization sensitive pixels (e.g., 0°, 45°, 90°, and 135°)” in col. 6, lines 16-18, Cavin and Price fail to teach where the light source comprises linearly polarized light, wherein the light source comprises a polarized filter oriented at a grid angle.
However, Lu teaches where the light source comprises linearly polarized light, wherein the light source comprises a polarized filter oriented at a grid angle (Lu teaches “the lens assembly may also include a polarizer, such as a linear polarizer or circular polarizer, and a polarization converter which may rotate linearly polarized light or change the handedness of circularly polarized light” in para. [0043], wherein the linear polarization rotator may have an angle of 45 degrees as shown in para. [0123]. Here, the 45 degree angle of the linear polarization rotator is interpreted as equivalent to the claimed grid angle).
Cavin, Price, and Lu are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Lu and include “where the light source comprises linearly polarized light, wherein the light source comprises a polarized filter oriented at a grid angle”. The motivation for doing so would have been that “eye-tracking unit 130 may be arranged to increase contrast in images of an eye captured by eye-tracking unit 130 while reducing the overall power consumed by eye-tracking unit 130 (e.g., reducing power consumed by a light emitter and an imaging system included in eye-tracking unit 130 )” and “virtual images may be formed on different image planes by turning on or off the switchable polarization converter”, as suggested by Lu in para. [0065] and [0155], respectively. See also para. [0131]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Lu to obtain the invention specified in claim 9.
Regarding claim 11, Cavin, Price, and Lu teach the system of claim 9,
wherein the light source can comprise a polarization alternation feature, wherein the light source switches the grid angle (Lu teaches that “when the switchable polarization rotator is turned on, the polarized light in the first linear polarization state may be changed to polarized light in the orthogonal second linear polarization state” in para. [0045], wherein “a half-wave plate with its axes oriented at 45° with respect to the polarization direction of the incident light may be used to rotate the polarization direction by 90°” as shown in para. [0122]. See also para. [0123], [0126]-[0131], and FIGs. 12-14C).
Similar motivations as applied to claim 9 can be applied here to claim 11.
Regarding claim 12, Cavin and Price teach the system of claim 8.
While Cavin teaches “a polarization sensitive camera comprises an array of polarization sensitive pixels (e.g., 0°, 45°, 90°, and 135°)” in col. 6, lines 16-18, Cavin and Price fail to teach wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees.
However, Lu teaches wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees (Lu teaches utilizing a “linear polarization rotator” for determining gaze direction of a user as shown in para. [0066] and [0122]-[0123]. Lu further teaches that “the linear polarization rotator is configured to rotate the polarization direction of linearly polarized light” wherein the linear polarization rotator may have an angle of 45 degrees as shown in para. [0123]).
Cavin, Price, and Lu are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Lu and include “wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees”. The motivation for doing so would have been that “eye-tracking unit 130 may be arranged to increase contrast in images of an eye captured by eye-tracking unit 130 while reducing the overall power consumed by eye-tracking unit 130 (e.g., reducing power consumed by a light emitter and an imaging system included in eye-tracking unit 130 )” and “virtual images may be formed on different image planes by turning on or off the switchable polarization converter”, as suggested by Lu in para. [0065] and [0155], respectively. See also para. [0131]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Lu to obtain the invention specified in claim 12.
Regarding claim 13, Cavin and Price teach the system of claim 8,
further comprising a plurality of light sources (Cavin teaches that “the camera assembly 320 includes one or more cameras” in col. 6, lines 5-8).
Cavin and Price fail to teach wherein each light source of the plurality of light sources is configured to switch an angle of polarization associated with the light source.
However, Lu teaches wherein each light source of the plurality of light sources is configured to switch an angle of polarization associated with the light source (Lu teaches that “when the switchable polarization rotator is turned on, the polarized light in the first linear polarization state may be changed to polarized light in the orthogonal second linear polarization state” in para. [0045], wherein “a half-wave plate with its axes oriented at 45° with respect to the polarization direction of the incident light may be used to rotate the polarization direction by 90°” as shown in para. [0122]. See also para. [0123], [0126]-[0131], and FIGs. 12-14C. See also that there can exist “two or more polarization-dependent liquid crystal (LC) lenses sensitive to either linear or circular polarization and having same or different optical powers can be used to project a displayed image on one of multiple image planes that are at different distances from the user's eyes” in para. [0043]. As such, it is inherent that there can exist multiple linear polarization rotators).
Cavin, Price, and Lu are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Lu and include “wherein each light source of the plurality of light sources is configured to switch an angle of polarization associated with the light source”. The motivation for doing so would have been that “eye-tracking unit 130 may be arranged to increase contrast in images of an eye captured by eye-tracking unit 130 while reducing the overall power consumed by eye-tracking unit 130 (e.g., reducing power consumed by a light emitter and an imaging system included in eye-tracking unit 130 )” and “virtual images may be formed on different image planes by turning on or off the switchable polarization converter”, as suggested by Lu in para. [0065] and [0155], respectively. See also para. [0131]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Lu to obtain the invention specified in claim 13.
Regarding claim 17, Cavin and Price teach the non-transient computer-readable storage medium of claim 14, further comprising receiving image data from a plurality of image capturing devices (Cavin teaches that “the camera assembly 320 includes one or more cameras” in col. 6, lines 5-8).
Cavin and Price fail to teach wherein at least one image capturing device of the plurality of image capturing devices is configured with a polarization angle that differs from another image capturing device of the plurality of image capturing devices.
However, Lu teaches wherein at least one image capturing device of the plurality of image capturing devices is configured with a polarization angle that differs from another image capturing device of the plurality of image capturing devices (Lu teaches “a lens assembly including two or more polarization-dependent liquid crystal (LC) lenses sensitive to either linear or circular polarization and having same or different optical powers can be used to project a displayed image on one of multiple image planes that are at different distances from the user's eyes. In some embodiments, the lens assembly may also include a polarizer, such as a linear polarizer or circular polarizer, and a polarization converter which may rotate linearly polarized light or change the handedness of circularly polarized light” in para. [0043], wherein “the alignment direction of the first LC lens may be θ, while the alignment direction of the second LC lens may be θ+90°” as shown in para. [0044]).
Cavin, Price, and Lu are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Lu and include “wherein at least one image capturing device of the plurality of image capturing devices is configured with a polarization angle that differs from another image capturing device of the plurality of image capturing devices”. The motivation for doing so would have been that “eye-tracking unit 130 may be arranged to increase contrast in images of an eye captured by eye-tracking unit 130 while reducing the overall power consumed by eye-tracking unit 130 (e.g., reducing power consumed by a light emitter and an imaging system included in eye-tracking unit 130 )” and “virtual images may be formed on different image planes by turning on or off the switchable polarization converter”, as suggested by Lu in para. [0065] and [0155], respectively. See also para. [0131]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Lu to obtain the invention specified in claim 17.
Regarding claim 20, Cavin and Price teach the non-transient computer-readable storage medium of claim 15.
While Cavin teaches “a polarization sensitive camera comprises an array of polarization sensitive pixels (e.g., 0°, 45°, 90°, and 135°)” in col. 6, lines 16-18, Cavin and Price fail to teach wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees.
However, Lu teaches wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees (Lu teaches utilizing a “linear polarization rotator” for determining gaze direction of a user as shown in para. [0066] and [0122]-[0123]. Lu further teaches that “the linear polarization rotator is configured to rotate the polarization direction of linearly polarized light” wherein the linear polarization rotator may have an angle of 45 degrees as shown in para. [0123]).
Cavin, Price, and Lu are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Lu and include “wherein the light source comprising polarized light is generated by a polarized grid such that the polarized grid is oriented at approximately one of: 0 degrees, 45 degrees, 90 degrees or 135 degrees”. The motivation for doing so would have been that “eye-tracking unit 130 may be arranged to increase contrast in images of an eye captured by eye-tracking unit 130 while reducing the overall power consumed by eye-tracking unit 130 (e.g., reducing power consumed by a light emitter and an imaging system included in eye-tracking unit 130 )” and “virtual images may be formed on different image planes by turning on or off the switchable polarization converter”, as suggested by Lu in para. [0065] and [0155], respectively. See also para. [0131]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Lu to obtain the invention specified in claim 20.
Claims 6 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Cavin et al. (U.S. Patent No. 10417784 B1), hereinafter Cavin, in view of Price et al. (U.S. Publication No. 2018/0329489 A1), hereinafter Price and Held et al. (U.S. Publication No. 2022/0413603 A1), hereinafter Held.
Regarding claim 6, Cavin and Price teach the method of claim 1.
Cavin and Price fail to teach wherein generating a representation of the eye comprises training a machine learning model to estimate a location of a user’s pupil.
However, Held teaches wherein generating a representation of the eye comprises training a machine learning model to estimate a location of a user’s pupil (Held teaches “a machine learning function may facilitate the processing of image data capturing overlapping images of an eyebox from different perspectives, and may output information such as a probable identification of each imaged glint (e.g. a light source identification and perspective identification for each imaged glint), a probable identification of an imaged retinal reflection, or even a likely gaze direction. […] Machine learning function 220 may be trained using a training set of image data and associated ground truth eye position data” in para. [0031], wherein “a trained machine learning function may be employed to process image data from the image sensor and determine a location of a pupil of eye 506, wherein the trained machine learning function may be trained using labeled image data comprising overlapping images corresponding to an eye imaged” as shown in para. [0040]).
Cavin, Price, and Held are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Held and include “wherein generating a representation of the eye comprises training a machine learning model to estimate a location of a user’s pupil”. The motivation for doing so would have been that “image data from each eye tracking camera is analyzed to determine the location of retinal reflections, the location of glint from each illumination source, and a location of the pupil, which may be used to determine a gaze direction”, as suggested by Held in para. [0017]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Held to obtain the invention specified in claim 6.
Regarding claim 19, Cavin and Price teach the non-transient computer-readable storage medium of claim 14.
Cavin and Price fail to teach wherein generating a representation of the eye comprises training a machine learning model to estimate a location of a user’s pupil.
However, Held teaches wherein generating a representation of the eye comprises training a machine learning model to estimate a location of a user’s pupil (Held teaches “a machine learning function may facilitate the processing of image data capturing overlapping images of an eyebox from different perspectives, and may output information such as a probable identification of each imaged glint (e.g. a light source identification and perspective identification for each imaged glint), a probable identification of an imaged retinal reflection, or even a likely gaze direction. […] Machine learning function 220 may be trained using a training set of image data and associated ground truth eye position data” in para. [0031], wherein “a trained machine learning function may be employed to process image data from the image sensor and determine a location of a pupil of eye 506, wherein the trained machine learning function may be trained using labeled image data comprising overlapping images corresponding to an eye imaged” as shown in para. [0040]).
Cavin, Price, and Held are all considered to be analogous to the claimed invention because they are in the same field of analyzing eye gaze of a user using image analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cavin (as modified by Price) to incorporate the teachings of Held and include “wherein generating a representation of the eye comprises training a machine learning model to estimate a location of a user’s pupil”. The motivation for doing so would have been that “image data from each eye tracking camera is analyzed to determine the location of retinal reflections, the location of glint from each illumination source, and a location of the pupil, which may be used to determine a gaze direction”, as suggested by Held in para. [0017]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Cavin and Price with Held to obtain the invention specified in claim 19.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Geng et al. (U.S. Publication No. 2019/0313087 A1) teaches an eye tracking system comprising a head mounted display for determining gaze direction by utilizing a linear polarizer.
Smyth (U.S. Patent No. 8824779 B1) teaches an eye gaze monitoring method comprising a head mounted display that utilizes a linear polarizer with the plane of polarization rotated 90 degrees.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can
normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/
Examiner, Art Unit 2661
/COURTNEY JOAN WINDSOR/Primary Examiner, Art Unit 2661
Prosecution Insights