DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Response to Amendments
Applicant’s remarks, see page 6, filed 06/24/2026, with respect to claim objections directed to claims 9 and 18, submitted in the non-final office action dated 03/24/2026, have been fully considered and are persuasive due to amendments in accordance with Examiner’s suggested corrections. Thus, the objections directed to claims 9 and 18 have been withdrawn.
Applicant’s remarks, see pages 7-8, filed 06/24/2026, with respect to claim limitations interpreted under 35 U.S.C. 112(f) in claims 1 and 10, submitted in the non-final office action dated 03/24/2026, have been fully considered and are persuasive due to amendments in accordance with Examiner’s interpretation of “analyzing unit” being a processor. Thus, the interpretation of “analyzing unit” under 35 U.S.C. 112(f) in claims 1 and 10 has been withdrawn.
Response to Arguments
Applicant’s arguments, see remarks, filed 06/24/2026, with respect to claims 1-20, have been fully considered, but are moot because the arguments do not apply to the current references and current combinations of references being used in the current rejection.
The applicant argues on page 11, “the Office asserts that Imamura’s mirror 121L and scanning mirror 122L (Fig. 9, paragraph [0101]) disclose this feature. However, Applicant respectfully disagrees with this assertion. Imamura’s mirror 121L and scanning mirror 122L are structural elements used to emit video display light- specifically, “laser light emitted by the laser source 120L is reflected off the mirror 121L, and then reaches the scanning mirror 122L. The scanning mirror 122L two-dimensionally scans the laser light” (paragraph [0101]). These are components of a video display projection system , not a light modulating unit configured to prevent adjacent light-emitting elements’ light from overlapping in an eye tracking context. The function, structure, and purpose of Imamura’s mirror/scanning mirror arrangement are fundamentally different from the claimed light modulating unit.”
In response to applicant's argument that IMAMURA’s components are that of “a video display projection system, not a light modulating unit configured to prevent adjacent light-emitting elements’ light from overlapping in an eye tracking context”, a recitation of the intended use of the claimed invention must result in a structural difference between the claimed invention and the prior art in order to patentably distinguish the claimed invention from the prior art. If the prior art structure is capable of performing the intended use, then it meets the claim.
Based on the breadth of the claim limitation, IMAMURA et al. (US 20230213766 A1), hereinafter referenced as IMAMURA, explicitly teaches a light modulating unit (Fig. 9, 109L and 109R called temple, Paragraph [0105] – IMAMURA discloses the light source section 101L and the projective optical system 102L are arranged in the temple 109L.)
disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements (Fig. 9, Paragraph [0101] – IMAMURA discloses the light source section 101L may include, for example, a laser source 120L, a mirror 121L, and a scanning mirror 122L that are structural elements used to emit the video display light. Laser light emitted by the laser source 120L is reflected off the mirror 121L, and then reaches the scanning mirror 122L. The scanning mirror 122L two-dimensionally scans the laser light. The scanning mirror 122L may be, for example, a MEMS mirror.)
so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements (Fig. 9, Paragraph [0102] – IMAMURA further discloses the projective optical system 102L adjusts the direction of the video display light such that the video display light reaches a desired region and/or position in the HOE 103L.).
The applicant argues on page 12, “the Office asserts that it would have been obvious to combine Yao and Imamura because “both YAO and IMAMURA relate to eye tracking methods and systems employed in display devices.” However, the mere fact that two references relate to a similar general field is insufficient to establish a proper motivation to combine under 35 U.S.C. 103. A person of ordinary skill in the art would not have a clear reason to incorporate Imamura’s video display laser projection components into Yao’s infrared eye tracking system in the manner required to arrive at the claimed light modulating unit.”
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the Office has clearly described that it would have been obvious to one of ordinary skill in the art at the time the invention was made to modify the teachings of YAO in view of IMAMURA, which are both related to eye tracking methods and systems employed in display devices that provide an enhanced method of tracking an eyeball that effectively improves tracking accuracy and overall system efficiency, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and IMAMURA relates to a display apparatus and a display method that makes it possible to further improve a performance in controlling video presentation according to characteristics of an eyeball of a user. Please see YAO (US 20240341593 A1), Paragraph [0161], and IMAMURA (US 20230213766 A1), Paragraph [0066].
The applicant argues on pages 12-13, “Sharma does not disclose that the groove comprises “at least two adjacent sub-grooves, and side walls of the sub-grooves are configured to dispose a light modulating unit” as recited in claim 20, where the light modulating unit functions to prevent adjacent light-emitting elements’ light from overlapping. Sharma’s reflectors serve a different purpose (camera field of view management) in a different structural context (single light source with reflectors, rather than sub-grooves containing multiple light-emitting elements).”.
The office respectfully disagrees. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “the light modulating unit functions to prevent adjacent light-emitting elements’ light from overlapping” and “sub-grooves containing multiple light-emitting elements”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
In this case, based on the breadth of the provided claim limitation of claim 20, SHARMA et al. (US 20190361523 A1), hereinafter referenced as SHARMA explicitly teaches wherein the groove (Fig. 4, Paragraph [0066] – SHARMA discloses near-eye display 400 may include a frame 405 and a display system that includes display electronics 410 and/or display optics 420 coupled to or embedded in [wherein embedded in is the groove] frame 405.)
comprises at least two adjacent sub-grooves Fig. 4, Paragraph [0069] – SHARMA discloses a single light source 430 and two reflectors 424-1 and 424-2 [wherein the two reflectors are adjacent sub-grooves] are shown in FIG. 4. See also Fig. 5, Paragraph [0071] – SHARMA discloses a plurality of reflectors may be immersed in a transparent substrate [wherein immersed in is a sub-groove].),
and side walls of the sub-grooves are configured to dispose a light modulating unit (Fig. 4, Paragraph [0069] – SHARMA discloses a single light source 430 and two reflectors 424-1 and 424-2 [wherein the two reflectors are a light modulating unit] are shown in FIG. 4.).
The applicant argues on page 13, “the Office asserts that it would have been obvious to combine Yao and Sharma because both relate to eye tracking in display devices. However, Yao’s system is an eye tracking apparatus using infrared LED light sources in a VR head-mounted display, while Sharma’s system is a near-eye display with laser-based illumination and reflector-based camera occlusion. A person of ordinary skill in the art would not have a clear reason to incorporate Sharma’s reflector arrangement into Yao’s eye tracking system to arrive at the claimed sub-groove structure with light modulating unit.”
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the Office has clearly described that it would have been obvious to one of ordinary skill in the art at the time the invention was made to modify the teachings of YAO in view of SHARMA, which are both related to eye tracking methods and systems employed in display devices that provide an enhanced method of tracking an eyeball that effectively improves tracking accuracy and overall system efficiency, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and SHARMA relates to relates to eye tracking in near-eye display devices to provide improved eye tracking accuracy and responsiveness. Please see YAO (US 20240341593 A1), Paragraph [0161], and SHARMA (US 20190361523 A1), Paragraph [0045].
The applicant further argues on page 14, “In view of the foregoing, Applicant submits that the pending claims are in condition for allowance, and respectfully requests reconsideration and timely allowance of the pending claims.”
In response, the office does not find this argument persuasive based on the same reasons set forth above and the rejection below. The office respectfully encourages the applicant to amend the claims to overcome the prior arts of record.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claims 1, 6, 10, and 15 recite limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claims 1 and 10; recite the limitation, “luminescent light sources configured to form…” [Lines 2 and 5, respectively].
Claims 1 and 10; recite the limitation, “an image acquiring unit configured to acquire…” [Lines 3 and 6, respectively].
Claims 6 and 15; recite the limitation, “a light modulating unit… configured to modulate…” [Lines 2-3 and 3-4, respectively].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claims 1, 6, 10, and 15:
“luminescent light sources” (Fig. 3A, #3011 and 3012, Paragraph [0038] – “The luminescent light sources 3011 and 3012 are configured to form light spots on an eye of a user. It can be understood that the luminescent light sources may be infrared LEDs, and the infrared LEDs may emit infrared light that is not perceived by human eyes, thereby not interfering with the user's perception of ambient light.” Thus, “luminescent light sources” has sufficient structure wherein the luminescent light sources are understood to be infrared LEDs.
“image acquiring unit” (Fig. 3A, #302, Paragraph [0046] – “It can be understood that those skilled in the art may select an appropriate image acquiring unit based on the features of the luminescent light sources. For example, if the luminescent light sources are infrared LEDs, the image acquiring unit 302 may acquire an image of infrared light. Optionally, the image acquiring unit 302 includes an infrared camera. The number of infrared cameras may be 1 (as shown in Figure 3A), 2, or other numbers, which is not limited in this disclosure.” Thus, the “image acquiring unit” has sufficient structure wherein it is understood to be an infrared camera.
“a light modulating unit” (Fig. 3, Paragraph [0064] – “Specifically, a light modulating unit is added to the module for tracking an eyeball, and the light modulating unit is disposed in at least part of periphery areas of respective light-emitting elements in the first light source 3011, and is configured to modulate light of the light-emitting elements so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements.” Paragraph [0065] – “Optionally, the light modulating unit includes first reflective surfaces 3041 and second reflective surfaces 3042 disposed oppositely; wherein the first reflective surfaces 3041 and the second reflective surfaces 3042 are arranged along arrangement directions of respective light-emitting elements.” Paragraph [0067] – “Optionally, the light modulating unit may also be a light-absorbing layer.”) Thus, the “light modulating unit” has sufficient structure wherein it is understood to be reflective surfaces.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed
invention is not identically disclosed as set forth in section 102 of this title, if the
differences between the claimed invention and the prior art are such that the claimed
invention as a whole would have been obvious before the effective filing date of the
claimed invention to a person having ordinary skill in the art to which the claimed
invention pertains. Patentability shall not be negated by the manner in which the
invention was made.
Claims 1-3, 5, 8-12, 14, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over YAO (US 20240341593 A1), herein after referenced as YAO in view of SHOUSHTARI (US 20200326777 A1), hereinafter referenced as SHOUSHTARI.
Regarding claim 1, YAO teaches a module for tracking an eyeball (Fig. 4, #412 called eye tracking module, Paragraph [0091] - YAO discloses eye tracking module 412 is configured to track movement of a human eye), comprising:
luminescent light sources (Fig. 7, #4121 called light sources, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121)
configured to form light spots on an eye of a user (Fig. 9, Paragraph [0133] - YAO discloses FIG. 9 is a schematic diagram of a scenario in which the light source 4121 irradiates an eye to obtain a human eye light spot image.);
an image acquiring unit (Fig. 7, #4122 called camera module, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121 and one or more camera modules 4122.)
configured to acquire a tracking image of a target area (Fig. 7, Paragraph [0122] - YAO discloses near-infrared camera 4122 may obtain reflected light and a human eye image, to obtain a human eye light spot image);
a processor (Fig. 4, #401 called a processor, Paragraph [0070] - YAO discloses the processor 401 may obtain a human eye light spot image)
configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image (Fig. 4, Paragraph [0070] - YAO discloses the processor 401 may obtain a human eye light spot image sent by a camera module in the eye tracking module 412, and may further learn a position of an eye of the user, to calculate a fixation point of the user.);
wherein the luminescent light sources comprise a first light source (Fig. 7, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121)
and form a first light spot on the eye (Fig. 9, Paragraph [0133] - YAO discloses FIG. 9 is a schematic diagram of a scenario in which the light source 4121 irradiates an eye to obtain a human eye light spot image.),
Although YAO further teaches so as to determine a position of a luminescent light source corresponding to at least part of light spots in the light spot image based on the first light spot (Fig. 10, Paragraph [0149] – YAO discloses there is a correspondence between a spatial position of a light spot and a spatial position of a light source, and a relative position between light spots is generally consistent with a relative position between light sources, so that a light source corresponding to each of the light spot 1 to the light spot 4 can be conveniently and accurately learned based on the human eye light spot image. Paragraph [0153] - YAO discloses the light spot in the human eye light spot image is matched with the light source in the eye tracking module 412 with reference to the types of light emitted by the light sources),
YAO fails to explicitly teach wherein the first light source is a position mark light source, and the first light spot formed by the first light source is a position mark light spot, and the processor is configured to locate, based on the position mark light spot, positions of luminescent light sources corresponding to at least part of the light spots in the light spot image.
However, SHOUSHTARI explicitly teaches wherein the first light source is a position mark light source (Fig. 1, Paragraph [0031] – SHOUSHTARI discloses the images may include glints, which are reflections of the IR or NIR light sources (e.g., arrays of LEDs) on the surface of the cornea.),
and the first light spot formed by the first light source is a position mark light spot (Fig. 8A, Paragraph [0055] – SHOUSHTARI discloses in this example, there is an array of six LEDs that emit light towards the user's eyes, resulting in glints (reflections of the LEDs off the cornea surface) as shown in FIG. 8A. As the user's eye moves, the location of the pupil with respect to the glints change. The detected glints may be used by the pupil detection and tracking process to assist in detecting and tracking the pupil location and contour.),
and the processor (Fig. 3A-B, Paragraph [0043] – SHOUSHTARI discloses the controller 260 may include one or more of various types of processors, image signal processors (ISPs), graphics processing units (GPUs), coder/decoders (codecs), and/or other components for processing and rendering video and/or images.) is configured to locate,
based on the position mark light spot (Fig. 7A-B, Paragraph [0056] – SHOUSHTARI discloses at 700, glints may be detected in an input image of the user's eye captured by a gaze tracking camera. Paragraph [0128] – SHOUSHTARI discloses at 706, glint-LED matching is performed in image space. See also Figs. 9A-C.),
positions of luminescent light sources corresponding to at least part of the light spots in the light spot image (Figs. 7A-B, 9A-C, Paragraph [0058] – SHOUSHTARI discloses FIGS. 9A through 9C graphically illustrate glint-LED matching in image (2D) space, according to some embodiments. In this example, there is an array of six LEDs that emit light towards the user's eyes, resulting in glints as shown in FIG. 8A. Paragraph [0059] – SHOUSHTARI discloses at 708, the candidate glint-LED matches that were selected and matched to LEDs at 706 may be verified geometrically in 3D space.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO of having a module for tracking an eyeball, comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and a processor configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, so as to determine a position of a luminescent light source corresponding to at least part of light spots in the light spot image based on the first light spot, with the teachings of SHOUSHTARI of having wherein the first light source is a position mark light source, and the first light spot formed by the first light source is a position mark light spot, and the processor is configured to locate, based on the position mark light spot, positions of luminescent light sources corresponding to at least part of the light spots in the light spot image.
Wherein having YAO’s module for tracking an eyeball wherein the first light source is a position mark light source, and the first light spot formed by the first light source is a position mark light spot, and the processor is configured to locate, based on the position mark light spot, positions of luminescent light sources corresponding to at least part of the light spots in the light spot image.
The motivation behind the modification would have been to obtain an enhanced module for tracking an eyeball that effectively improves eye tracking accuracy and improves robustness of pupil localization, since both YAO and SHOUSHTARI relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and SHOUSHTARI discloses methods and apparatus for glint-assisted gaze tracking in VR/AR head-mounted displays wherein a glint geometric matching method is applied in 3D space to detect and correct potential mismatches using the glint matching in image space method. Please see YAO (US 20240341593 A1), Paragraph [0161], and SHOUSHTARI (US 20200326777 A1), Paragraph [0063].
Regarding claim 2, YAO in view of SHOUSHTARI teach the module for tracking an eyeball of claim 1,
YAO further teaches wherein the luminescent light sources (Fig. 7, #4121 called light sources, Paragraph [0121]) further comprise a second light source (Fig. 7, Paragraph [0159] – YAO discloses the eye tracking apparatus includes M light sources 4121 and the camera module 4122, where M is a positive integer greater than or equal to 2),
and the first light source and the second light source form different light spots (Fig. 11, Paragraph [0145] – YAO discloses the human eye light spot image includes the human eye contour 901, and the iris range 902 includes four light spots: a light spot 1, a light spot 2, a light spot 3, and a light spot 4. See also Paragraph [0152].).
Regarding claim 3, YAO in view of SHOUSHTARI teach the module for tracking an eyeball of claim 2,
YAO further teaches wherein the different light spots comprise at least one of different shapes and different flicker frequencies (Fig. 10, Paragraph [0207] – YAO discloses shapes of the sub-regions that filter light with different wavelengths in the light filtering unit may be the same or may be different.).
Regarding claim 5, YAO teaches the module for tracking an eyeball of claim 1,
wherein the first light source (Fig. 7, #4121 called light source, Paragraph [0121]) comprises at least two light-emitting elements disposed side by side (Fig. 2A, Paragraph [0051] – YAO discloses for example, eight light sources 2501 are disposed on the end surface 210a that is of the lens tube and that faces the eye, the eight light sources 2501 may be evenly distributed in a circular shape, and the camera module 2502 may be disposed on the end surface 210a that is of the lens tube and that faces the eye. The light sources 2501 and the camera module 2502 may be disposed around a side that is of the optical device 211 and that faces the eye.).
Regarding claim 8, YAO in view of SHOUSHTARI teach the module for tracking an eyeball of claim 1,
YAO further teaches wherein the luminescent light source comprises at least two first light sources (Fig. 7, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121);
wherein, a plurality of the first light sources are arranged symmetrically (Fig. 12, Paragraph [0183] – YAO discloses for example, (a) in FIG. 12 shows an example in which the eight light sources are evenly arranged in a rectangle. For example, (b) in FIG. 12 shows an example in which the eight light sources are evenly arranged in a rhombus.);
and/or light spots of the plurality of first light sources are the same or different (Fig. 11, Paragraph [0145] – YAO discloses the human eye light spot image includes the human eye contour 901, and the iris range 902 includes four light spots: a light spot 1, a light spot 2, a light spot 3, and a light spot 4. Paragraph [0139] – YAO further discloses one light spot may correspond to a plurality of light sources.).
Regarding claim 9, YAO in view of SHOUSHTARI teach the module for tracking an eyeball of claim 1,
YAO further teaches wherein the image acquiring unit comprises a camera (Fig. 7, #4122 called camera module, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121 and one or more camera modules 4122.),
and an angle between the camera and the at least one first light source relative to a center of the eye is not greater than 10° (Fig. 7, Paragraph [0167] – YAO discloses camera module 4122 may be placed at a plurality of different positions relative to the light source 4121.).
Regarding claim 10, YAO teaches an extended reality device (Fig. 1, #100 called VR head mounted display device, Paragraph [0050] – YAO discloses a VR head mounted display device 100 may be referred to as an all-in-one VR machine), comprising:
a body (Fig. 2A, Paragraph [0051] – YAO discloses FIG. 2A is a schematic diagram of the VR head mounted display device 100.);
and a module disposed on the body and configured for tracking an eyeball (Fig. 2A, Paragraph [0051] – YAO discloses at least one eye tracking module (including M light sources 2501 and at least one camera module 2502) may be disposed on the VR head mounted display device 100, and is configured to track a movement of a human eye, to determine a fixation point of the human eye. In some embodiments, a light source 2501 may be disposed on an end surface 100a that is of the VR head mounted display device 100 and that faces a face (or an eye of the user). In addition, a camera module 2502 may be further disposed on the end surface 100a that is of the VR head mounted display device 100 and that faces the face (or the eye of the user).),
the module for tracking the eyeball (Fig. 4, #412 called eye tracking module, Paragraph [0091] - YAO discloses eye tracking module 412 is configured to track movement of a human eye) comprising:
luminescent light sources (Fig. 7, #4121 called light sources, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121)
configured to form light spots on an eye of a user (Fig. 9, Paragraph [0133] - YAO discloses FIG. 9 is a schematic diagram of a scenario in which the light source 4121 irradiates an eye to obtain a human eye light spot image.);
an image acquiring unit (Fig. 7, #4122 called camera module, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121 and one or more camera modules 4122.)
configured to acquire a tracking image of a target area (Fig. 7, Paragraph [0122] - YAO discloses near-infrared camera 4122 may obtain reflected light and a human eye image, to obtain a human eye light spot image);
and a processor (Fig. 4, #401 called a processor, Paragraph [0070] - YAO discloses the processor 401 may obtain a human eye light spot image)
configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image (Fig. 4, Paragraph [0070] - YAO discloses the processor 401 may obtain a human eye light spot image sent by a camera module in the eye tracking module 412, and may further learn a position of an eye of the user, to calculate a fixation point of the user.);
wherein the luminescent light sources comprise a first light source (Fig. 7, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121)
and form a first light spot on the eye (Fig. 9, Paragraph [0133] - YAO discloses FIG. 9 is a schematic diagram of a scenario in which the light source 4121 irradiates an eye to obtain a human eye light spot image.),
Although YAO further teaches so as to determine a position of a luminescent light source corresponding to at least part of light spots in the light spot image based on the first light spot (Fig. 10, Paragraph [0149] – YAO discloses there is a correspondence between a spatial position of a light spot and a spatial position of a light source, and a relative position between light spots is generally consistent with a relative position between light sources, so that a light source corresponding to each of the light spot 1 to the light spot 4 can be conveniently and accurately learned based on the human eye light spot image. Paragraph [0153] - YAO discloses the light spot in the human eye light spot image is matched with the light source in the eye tracking module 412 with reference to the types of light emitted by the light sources),
YAO fails to explicitly teach wherein the first light source is a position mark light source, and the first light spot formed by the first light source is a position mark light spot, and the processor is configured to locate, based on the position mark light spot, positions of luminescent light sources corresponding to at least part of the light spots in the light spot image.
However, SHOUSHTARI explicitly teaches wherein the first light source is a position mark light source (Fig. 1, Paragraph [0031] – SHOUSHTARI discloses the images may include glints, which are reflections of the IR or NIR light sources (e.g., arrays of LEDs) on the surface of the cornea.),
and the first light spot formed by the first light source is a position mark light spot (Fig. 8A, Paragraph [0055] – SHOUSHTARI discloses in this example, there is an array of six LEDs that emit light towards the user's eyes, resulting in glints (reflections of the LEDs off the cornea surface) as shown in FIG. 8A. As the user's eye moves, the location of the pupil with respect to the glints change. The detected glints may be used by the pupil detection and tracking process to assist in detecting and tracking the pupil location and contour.),
and the processor (Fig. 3A-B, Paragraph [0043] – SHOUSHTARI discloses the controller 260 may include one or more of various types of processors, image signal processors (ISPs), graphics processing units (GPUs), coder/decoders (codecs), and/or other components for processing and rendering video and/or images.) is configured to locate,
based on the position mark light spot (Fig. 7A-B, Paragraph [0056] – SHOUSHTARI discloses at 700, glints may be detected in an input image of the user's eye captured by a gaze tracking camera. Paragraph [0128] – SHOUSHTARI discloses at 706, glint-LED matching is performed in image space. See also Figs. 9A-C.),
positions of luminescent light sources corresponding to at least part of the light spots in the light spot image (Figs. 7A-B, 9A-C, Paragraph [0058] – SHOUSHTARI discloses FIGS. 9A through 9C graphically illustrate glint-LED matching in image (2D) space, according to some embodiments. In this example, there is an array of six LEDs that emit light towards the user's eyes, resulting in glints as shown in FIG. 8A. Paragraph [0059] – SHOUSHTARI discloses at 708, the candidate glint-LED matches that were selected and matched to LEDs at 706 may be verified geometrically in 3D space.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO of having an extended reality device, comprising: a body; and a module disposed on the body and configured for tracking an eyeball, the module for tracking the eyeball comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and a processor configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, so as to determine a position of a luminescent light source corresponding to at least part of light spots in the light spot image based on the first light spot, with the teachings of SHOUSHTARI of having wherein the first light source is a position mark light source, and the first light spot formed by the first light source is a position mark light spot, and the processor is configured to locate, based on the position mark light spot, positions of luminescent light sources corresponding to at least part of the light spots in the light spot image.
Wherein having YAO’s extended reality device wherein the first light source is a position mark light source, and the first light spot formed by the first light source is a position mark light spot, and the processor is configured to locate, based on the position mark light spot, positions of luminescent light sources corresponding to at least part of the light spots in the light spot image.
The motivation behind the modification would have been to obtain an enhanced device for tracking an eyeball that effectively improves eye tracking accuracy and improves robustness of pupil localization, since both YAO and SHOUSHTARI relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and SHOUSHTARI discloses methods and apparatus for glint-assisted gaze tracking in VR/AR head-mounted displays wherein a glint geometric matching method is applied in 3D space to detect and correct potential mismatches using the glint matching in image space method. Please see YAO (US 20240341593 A1), Paragraph [0161], and SHOUSHTARI (US 20200326777 A1), Paragraph [0063].
Regarding claim 11, YAO in view of SHOUSHTARI teach the extended reality device of claim 10,
YAO further teaches wherein the luminescent light sources (Fig. 7, #4121 called light sources, Paragraph [0121]) further comprise a second light source (Fig. 7, Paragraph [0159] – YAO discloses the eye tracking apparatus includes M light sources 4121 and the camera module 4122, where M is a positive integer greater than or equal to 2),
and the first light source and the second light source form different light spots (Fig. 11, Paragraph [0145] – YAO discloses the human eye light spot image includes the human eye contour 901, and the iris range 902 includes four light spots: a light spot 1, a light spot 2, a light spot 3, and a light spot 4. See also Paragraph [0152].).
Regarding claim 12, YAO in view of SHOUSHTARI teach the extended reality device for tracking an eyeball of claim 11,
YAO further teaches wherein the different light spots comprise at least one of different shapes and different flicker frequencies (Fig. 10, Paragraph [0207] – YAO discloses shapes of the sub-regions that filter light with different wavelengths in the light filtering unit may be the same or may be different.).
Regarding claim 14, YAO in view of SHOUSHTARI teach the extended reality device of claim 10,
wherein the first light source (Fig. 7, #4121 called light source, Paragraph [0121]) comprises at least two light-emitting elements disposed side by side (Fig. 2A, Paragraph [0051] – YAO discloses for example, eight light sources 2501 are disposed on the end surface 210a that is of the lens tube and that faces the eye, the eight light sources 2501 may be evenly distributed in a circular shape, and the camera module 2502 may be disposed on the end surface 210a that is of the lens tube and that faces the eye. The light sources 2501 and the camera module 2502 may be disposed around a side that is of the optical device 211 and that faces the eye.).
Regarding claim 17, YAO in view of SHOUSHTARI teach the extended reality device of claim 10,
YAO further teaches wherein the luminescent light source comprises at least two first light sources (Fig. 7, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121);
wherein, a plurality of the first light sources are arranged symmetrically (Fig. 12, Paragraph [0183] – YAO discloses for example, (a) in FIG. 12 shows an example in which the eight light sources are evenly arranged in a rectangle. For example, (b) in FIG. 12 shows an example in which the eight light sources are evenly arranged in a rhombus.);
and/or light spots of the plurality of first light sources are the same or different (Fig. 11, Paragraph [0145] – YAO discloses the human eye light spot image includes the human eye contour 901, and the iris range 902 includes four light spots: a light spot 1, a light spot 2, a light spot 3, and a light spot 4. Paragraph [0139] – YAO further discloses one light spot may correspond to a plurality of light sources.).
Regarding claim 18, YAO in view of SHOUSHTARI teach the extended reality device of claim 10,
YAO further teaches wherein the image acquiring unit comprises a camera (Fig. 7, #4122 called camera module, Paragraph [0121] - YAO discloses eye tracking module 412 may include one or more light sources 4121 and one or more camera modules 4122.),
and an angle between the camera and the at least one first light source relative to a center of the eye is not greater than 10° (Fig. 7, Paragraph [0167] – YAO discloses camera module 4122 may be placed at a plurality of different positions relative to the light source 4121.).
Regarding claim 19, YAO in view of SHOUSHTARI teach the extended reality device of claim 10,
YAO further teaches wherein the body comprises a groove (Fig. 2A, Paragraph [0051] – YAO discloses the optical display module 220 includes a display and an optical device 221. In some embodiments, the optical display module 210 and the optical display module 220 further include an optical device. In some embodiments, the optical display module 210 and the optical display module 220 each may be lens tubes. The lens tube is hollow cylindrical. In other words, the optical device is accommodated in the lens tube, and the optical device and the display device are disposed on the VR head mounted display device 100 by using the lens tube.),
an opening of the groove faces an eye of a user, and the luminescent light sources included in the module for tracking an eye is disposed in the groove (Fig. 2A, Paragraph [0051] – YAO discloses a light source 2501 may be disposed on an end surface 100a that is of the VR head mounted display device 100 and that faces a face (or an eye of the user).).
Claims 4, 6, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over YAO (US 20240341593 A1), herein after referenced as YAO in view of SHOUSHTARI (US 20200326777 A1), hereinafter referenced as SHOUSHTARI in further view of IMAMURA (US 20230213766 A1), hereinafter referenced as IMAMURA.
Regarding claim 4, YAO in view of SHOUSHTARI teach the module for tracking an eyeball of claim 2,
Although YAO explicitly teaches wherein each of the first light source and the second light source comprises at least one light-emitting element (Fig. 10, Paragraph [0159] – YAO discloses M light sources emit N different types of light to an eye of a user, where N is a positive integer greater than or equal to 2.),
YAO in view of SHOUSHTARI fail to explicitly teach wherein the number of the light-emitting elements included in the first light source is different from that in the second light source.
However, IMAMURA explicitly teaches wherein the number of the light-emitting elements included in the first light source is different from that in the second light source (Fig. 9, Paragraph [0110] – IMAMURA discloses the light source section 101L illustrated in FIG. 9 includes one laser source 120L. However, the number of laser sources included in the light source section 101L may be two or more, and may be, for example, from two to five. The laser sources of a plurality of the laser sources may output pieces of laser light of wavelengths different from each other. Likewise, the light source section 101R includes one laser source 120R. However, the number of laser sources included in the light source section 101R may be two or more, and may be, for example, from two to five. The laser sources of a plurality of the laser sources may output pieces of laser light of wavelengths different from each other.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having a module for tracking an eyeball, comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and an analyzing unit configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of IMAMURA of having wherein the number of the light-emitting elements included in the first light source is different from that in the second light source.
Wherein having YAO’s module for tracking an eyeball wherein the number of the light-emitting elements included in the first light source is different from that in the second light source.
The motivation behind the modification would have been to obtain an enhanced module for tracking an eyeball that effectively improves eye tracking accuracy and overall system efficiency, since both YAO and IMAMURA relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and IMAMURA relates to a display apparatus and a display method that makes it possible to further improve a performance in controlling video presentation according to characteristics of an eyeball of a user. Please see YAO (US 20240341593 A1), Paragraph [0161], and IMAMURA (US 20230213766 A1), Paragraph [0044].
Regarding claim 6, YAO in view of SHOUSHTARI teach the module for tracking an eyeball of claim 5,
YAO in view of SHOUSHTARI fail to explicitly teach further comprising: a light modulating unit disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements.
However, IMAMURA explicitly teaches further comprising: a light modulating unit (Fig. 9, 109L and 109R called temple, Paragraph [0105] – IMAMURA discloses the light source section 101L and the projective optical system 102L are arranged in the temple 109L.)
disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements (Fig. 9, Paragraph [0101] – IMAMURA discloses the light source section 101L may include, for example, a laser source 120L, a mirror 121L, and a scanning mirror 122L that are structural elements used to emit the video display light. Laser light emitted by the laser source 120L is reflected off the mirror 121L, and then reaches the scanning mirror 122L. The scanning mirror 122L two-dimensionally scans the laser light. The scanning mirror 122L may be, for example, a MEMS mirror.)
so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements (Fig. 9, Paragraph [0102] – IMAMURA further discloses the projective optical system 102L adjusts the direction of the video display light such that the video display light reaches a desired region and/or position in the HOE 103L.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having a module for tracking an eyeball, comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and an analyzing unit configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of IMAMURA of having further comprising: a light modulating unit disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements.
Wherein having YAO’s module for tracking an eyeball further comprising: a light modulating unit disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements.
The motivation behind the modification would have been to obtain an enhanced module for tracking an eyeball that effectively improves eye tracking accuracy and overall system efficiency, since both YAO and IMAMURA relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and IMAMURA relates to a display apparatus and a display method that makes it possible to further improve a performance in controlling video presentation according to characteristics of an eyeball of a user. Please see YAO (US 20240341593 A1), Paragraph [0161], and IMAMURA (US 20230213766 A1), Paragraph [0044].
Regarding claim 13, YAO in view of SHOUSHTARI teach the extended reality device of claim 11,
Although YAO explicitly teaches wherein each of the first light source and the second light source comprises at least one light-emitting element (Fig. 10, Paragraph [0159] – YAO discloses M light sources emit N different types of light to an eye of a user, where N is a positive integer greater than or equal to 2.),
YAO in view of SHOUSHTARI fail to explicitly teach wherein the number of the light-emitting elements included in the first light source is different from that in the second light source.
However, IMAMURA explicitly teaches wherein the number of the light-emitting elements included in the first light source is different from that in the second light source (Fig. 9, Paragraph [0110] – IMAMURA discloses the light source section 101L illustrated in FIG. 9 includes one laser source 120L. However, the number of laser sources included in the light source section 101L may be two or more, and may be, for example, from two to five. The laser sources of a plurality of the laser sources may output pieces of laser light of wavelengths different from each other. Likewise, the light source section 101R includes one laser source 120R. However, the number of laser sources included in the light source section 101R may be two or more, and may be, for example, from two to five. The laser sources of a plurality of the laser sources may output pieces of laser light of wavelengths different from each other.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having an extended reality device, comprising: a body; and a module disposed on the body and configured for tracking an eyeball, the module for tracking the eyeball comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and a processor configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of IMAMURA of having wherein the number of the light-emitting elements included in the first light source is different from that in the second light source.
Wherein having YAO’s extended reality device for tracking an eyeball wherein the number of the light-emitting elements included in the first light source is different from that in the second light source.
The motivation behind the modification would have been to obtain an enhanced extended reality device with gaze monitoring capabilities that effectively improves eye tracking accuracy and overall system efficiency, since both YAO and IMAMURA relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and IMAMURA relates to a display apparatus and a display method that makes it possible to further improve a performance in controlling video presentation according to characteristics of an eyeball of a user. Please see YAO (US 20240341593 A1), Paragraph [0161], and IMAMURA (US 20230213766 A1), Paragraph [0044].
Regarding claim 15, YAO in view of SHOUSHTARI teach the extended reality device of claim 14,
Although YAO explicitly teaches wherein the module for tracking the eyeball (Fig. 4, #412 called eye tracking module, Paragraph [0091]),
YAO in view of SHOUSHTARI fail to explicitly teach further comprises: a light modulating unit disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements.
However, IMAMURA explicitly teaches further comprises: a light modulating unit (Fig. 9, 109L and 109R called temple, Paragraph [0105] – IMAMURA discloses the light source section 101L and the projective optical system 102L are arranged in the temple 109L.)
disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements (Fig. 9, Paragraph [0101] – IMAMURA discloses the light source section 101L may include, for example, a laser source 120L, a mirror 121L, and a scanning mirror 122L that are structural elements used to emit the video display light. Laser light emitted by the laser source 120L is reflected off the mirror 121L, and then reaches the scanning mirror 122L. The scanning mirror 122L two-dimensionally scans the laser light. The scanning mirror 122L may be, for example, a MEMS mirror.)
so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements (Fig. 9, Paragraph [0102] – IMAMURA further discloses the projective optical system 102L adjusts the direction of the video display light such that the video display light reaches a desired region and/or position in the HOE 103L.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having an extended reality device, comprising: a body; and a module disposed on the body and configured for tracking an eyeball, the module for tracking the eyeball comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and a processor configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of IMAMURA of having further comprises: a light modulating unit disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements.
Wherein having YAO’s extended reality device for tracking an eyeball further comprises: a light modulating unit disposed in at least part of periphery areas of respective light-emitting elements in the first light source and configured to modulate light of the light-emitting elements so that the light of a light-emitting element does not overlap with light of adjacent light-emitting elements.
The motivation behind the modification would have been to obtain an enhanced extended reality device with gaze monitoring capabilities that effectively improves eye tracking accuracy and overall system efficiency, since both YAO and IMAMURA relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and IMAMURA relates to a display apparatus and a display method that makes it possible to further improve a performance in controlling video presentation according to characteristics of an eyeball of a user. Please see YAO (US 20240341593 A1), Paragraph [0161], and IMAMURA (US 20230213766 A1), Paragraph [0044].
Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over YAO (US 20240341593 A1), herein after referenced as YAO in view of SHOUSHTARI (US 20200326777 A1), hereinafter referenced as SHOUSHTARI in further view of IMAMURA (US 20230213766 A1), hereinafter referenced as IMAMURA in further view of YORK (US 20150241029 A1), hereinafter referenced as YORK.
Regarding claim 7, YAO and SHOUSHTARI in view of IMAMURA teach the module for tracking an eyeball of claim 6,
YAO in view of SHOUSHTARI fail to explicitly teach wherein the light modulating unit comprises first reflective surfaces and second reflective surfaces disposed oppositely;
However, IMAMURA teaches wherein the light modulating unit (Fig. 9, 109L and 109R called temple, Paragraph [0105]) comprises first reflective surfaces (Fig. 9, #121L called mirror, Paragraph [0101]) and second reflective surfaces (Fig. 9, #122L called a scanning mirror, Paragraph [0101]) disposed oppositely (Fig. 9, Paragraph [0101] – IMAMURA discloses the light source section 101L may include, for example, a laser source 120L, a mirror 121L, and a scanning mirror 122L that are structural elements used to emit the video display light. Laser light emitted by the laser source 120L is reflected off the mirror 121L, and then reaches the scanning mirror 122L. See also Paragraph [0129].);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having a module for tracking an eyeball, comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and an analyzing unit configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of IMAMURA of having wherein the light modulating unit comprises first reflective surfaces and second reflective surfaces disposed oppositely;
Wherein having YAO’s module for tracking an eyeball wherein the light modulating unit comprises first reflective surfaces and second reflective surfaces disposed oppositely;
The motivation behind the modification would have been to obtain an enhanced module for tracking an eyeball that effectively improves eye tracking accuracy and overall system efficiency, since both YAO and IMAMURA relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and IMAMURA relates to a display apparatus and a display method that makes it possible to further improve a performance in controlling video presentation according to characteristics of an eyeball of a user. Please see YAO (US 20240341593 A1), Paragraph [0161], and IMAMURA (US 20230213766 A1), Paragraph [0044].
YAO and SHOUSHTARI in view of IMAMURA fail to explicitly teach wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements, and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions.
However, YORK explicitly teaches wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements (Fig. 4A, Paragraph [0049] – YORK discloses a light-emitting device 200 with a reflective element 250 placed between light-emitting elements 210 and 212 that extends from the base substrate 245.),
and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions (Fig. 4A, Paragraph [0014] – YORK discloses a light-emitting device includes a base substrate; multiple light-emitting elements (LEEs) disposed on the base substrate. Paragraph [0008] – YORK further discloses in some embodiments, the first and second reflective surfaces are arranged perpendicular to the base substrate.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having a module for tracking an eyeball, comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and an analyzing unit configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of YORK of having wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements, and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions.
Wherein having YAO’s module for tracking an eyeball wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements, and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions.
The motivation behind the modification would have been to obtain an enhanced method for tracking an eyeball that provides improved eye tracking accuracy and spatially controlled lighting, since both YAO and YORK relate to configurations for light-emitting elements, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and YORK relates to light-emitting devices that include a solid-state based optical system with reflective elements to generate various illumination patterns; with the inclusion of advanced controls and sensing, these light-emitting devices can dynamically adapt to users and tasks throughout the day or over the lifetime of the system. Please see YAO (US 20240341593 A1), Paragraph [0161], and YORK (US 20150241029 A1), Paragraph [0028].
Regarding claim 16, YAO and SHOUSHTARI in view of IMAMURA teach the extended reality device of claim 15,
YAO in view of SHOUSHTARI fail to explicitly teach wherein the light modulating unit comprises first reflective surfaces and second reflective surfaces disposed oppositely;
However, IMAMURA teaches wherein the light modulating unit (Fig. 9, 109L and 109R called temple, Paragraph [0105]) comprises first reflective surfaces (Fig. 9, #121L called mirror, Paragraph [0101]) and second reflective surfaces (Fig. 9, #122L called a scanning mirror, Paragraph [0101]) disposed oppositely (Fig. 9, Paragraph [0101] – IMAMURA discloses the light source section 101L may include, for example, a laser source 120L, a mirror 121L, and a scanning mirror 122L that are structural elements used to emit the video display light. Laser light emitted by the laser source 120L is reflected off the mirror 121L, and then reaches the scanning mirror 122L. See also Paragraph [0129].);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having an extended reality device, comprising: a body; and a module disposed on the body and configured for tracking an eyeball, the module for tracking the eyeball comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and a processor configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of IMAMURA of having wherein the light modulating unit comprises first reflective surfaces and second reflective surfaces disposed oppositely;
Wherein having YAO’s extended reality device for tracking an eyeball wherein the light modulating unit comprises first reflective surfaces and second reflective surfaces disposed oppositely;
The motivation behind the modification would have been to obtain an enhanced method for tracking an eyeball that effectively improves eye tracking accuracy and overall system efficiency, since both YAO and IMAMURA relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and IMAMURA relates to a display apparatus and a display method that makes it possible to further improve a performance in controlling video presentation according to characteristics of an eyeball of a user. Please see YAO (US 20240341593 A1), Paragraph [0161], and IMAMURA (US 20230213766 A1), Paragraph [0044].
YAO and SHOUSHTARI in view of IMAMURA fail to explicitly teach wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements, and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions.
However, YORK explicitly teaches wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements (Fig. 4A, Paragraph [0049] – YORK discloses a light-emitting device 200 with a reflective element 250 placed between light-emitting elements 210 and 212 that extends from the base substrate 245.),
and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions (Fig. 4A, Paragraph [0014] – YORK discloses a light-emitting device includes a base substrate; multiple light-emitting elements (LEEs) disposed on the base substrate. Paragraph [0008] – YORK further discloses in some embodiments, the first and second reflective surfaces are arranged perpendicular to the base substrate.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having a module for tracking an eyeball, comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and an analyzing unit configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image; wherein the luminescent light sources comprise a first light source and form a first light spot on the eye, with the teachings of YORK of having wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements, and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions.
Wherein having YAO’s module for tracking an eyeball wherein the first reflective surfaces and the second reflective surfaces are arranged along arrangement directions of respective light-emitting elements, and the first reflective surfaces and the second reflective surfaces are respectively perpendicular to the arrangement directions.
The motivation behind the modification would have been to obtain an enhanced method for tracking an eyeball that provides improved eye tracking accuracy and spatially controlled lighting, since both YAO and YORK relate to configurations for light-emitting elements, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and YORK relates to light-emitting devices that include a solid-state based optical system with reflective elements to generate various illumination patterns; with the inclusion of advanced controls and sensing, these light-emitting devices can dynamically adapt to users and tasks throughout the day or over the lifetime of the system. Please see YAO (US 20240341593 A1), Paragraph [0161], and YORK (US 20150241029 A1), Paragraph [0028].
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over YAO (US 20240341593 A1), herein after referenced as YAO in view of SHOUSHTARI (US 20200326777 A1), hereinafter referenced as SHOUSHTARI in view of SHARMA (US 20190361523 A1), hereinafter referenced as SHARMA.
Regarding claim 20, YAO in view of SHOUSHTARI teach the extended reality device of claim 19,
YAO in view of SHOUSHTARI fail to explicitly teach wherein the groove comprises at least two adjacent sub-grooves, and side walls of the sub-grooves are configured to dispose a light modulating unit.
However, SHARMA explicitly teaches wherein the groove (Fig. 4, Paragraph [0066] – SHARMA discloses near-eye display 400 may include a frame 405 and a display system that includes display electronics 410 and/or display optics 420 coupled to or embedded in [wherein embedded in is the groove] frame 405.)
comprises at least two adjacent sub-grooves Fig. 4, Paragraph [0069] – SHARMA discloses a single light source 430 and two reflectors 424-1 and 424-2 [wherein the two reflectors are adjacent sub-grooves] are shown in FIG. 4. See also Fig. 5, Paragraph [0071] – SHARMA discloses a plurality of reflectors may be immersed in a transparent substrate [wherein immersed in is a sub-groove].),
and side walls of the sub-grooves are configured to dispose a light modulating unit (Fig. 4, Paragraph [0069] – SHARMA discloses a single light source 430 and two reflectors 424-1 and 424-2 [wherein the two reflectors are a light modulating unit] are shown in FIG. 4.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of YAO in view of SHOUSHTARI of having an extended reality device, comprising: a body; and a module disposed on the body and configured for tracking an eyeball, the module for tracking the eyeball comprising: luminescent light sources configured to form light spots on an eye of a user; an image acquiring unit configured to acquire a tracking image of a target area; and a processor configured to determine a pose of the eyeball based on an eye image and a light spot image in the tracking image, with the teachings of SHARMA of having wherein the groove comprises at least two adjacent sub-grooves, and side walls of the sub-grooves are configured to dispose a light modulating unit.
Wherein having YAO’s extended reality device for tracking an eyeball wherein the groove comprises at least two adjacent sub-grooves, and side walls of the sub-grooves are configured to dispose a light modulating unit.
The motivation behind the modification would have been to obtain an enhanced extended reality device with gaze monitoring capabilities that effectively improves eye tracking accuracy and overall system efficiency, since both YAO and SHARMA relate to eye tracking methods and systems employed in display devices, wherein YAO provides an eye tracking apparatus and an eye tracking method, which helps improve accuracy and efficiency of matching a light spot with a light source, and improves accuracy of a human eye fixation point determined by eye tracking, and SHARMA relates to relates to eye tracking in near-eye display devices to provide improved eye tracking accuracy and responsiveness. Please see YAO (US 20240341593 A1), Paragraph [0161], and SHARMA (US 20190361523 A1), Paragraph [0045].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant’s disclosure.
Caspi et al. (US 20250216671 A1) - An arrangement for tracking a position of an individual's eye comprises: (a) a frame mountable on a head of an individual; (b) at least one visually transparent element mounted within the frame in a the field of view of an eye of the individual; (c) at least one infrared light source configured for emitting an infrared light beam; (d) an image sensor configured for capturing images of the eye; (e) a processing unit configured for controlling the infrared light source and the image sensor and detecting a pupil position and a corneal reflection within images captured the at least image sensor. The transparent element comprises a spectrally-selective layer configured for reflecting the infrared beam. The visually transparent element is configured for selectively directing the infrared beam onto the eye and directing an infrared radiation reflected from a pupil and cornea of the eye to the image sensor… Fig. 2, 4A-B, Abstract.
Ikeda et al. (US 20220350150 A1) - A downsized electronic device with an eye tracking function is provided. The electronic device with an eye tracking function includes a display device, an infrared light source, and an optical system. The display device includes a display element and a light-receiving element; the infrared light source has a function of emitting infrared light; the light-receiving element has a function of detecting the infrared light reflected by an eyeball; and the optical system includes a first optical element positioned on an optical path through which an image from the display element enters the eyeball and a second optical element positioned on an optical path through which the reflected infrared light enters the light-receiving element. The light-receiving element is integrated with the display device and thus, the electronic device can have a reduced size.… Figs. 12A-C, Abstract.
Zhang et al. (US 20220321866 A1) - A head-mounted viewable device which includes a display configured to display and project a virtual scene image to a user's eye, a VR optical lens configured to construct an optical path between the display and the user's eye for allowing a virtual scene image being observed by the user's eye, and, an eye-tracking system configured to detect a sight direction of the user's eye and adjust a display position of the virtual scene image based on the detected sight direction. The eye-tracking system includes at least one light source configured to project the detection light and a receiving module configured to receive the reflected detection light reflected to determine the sight direction of the user's eye… Fig. 10, Abstract.
Rana et al. (US 20190369720 A1) - A method of identifying scleral reflections in an eye tracking system and a corresponding eye tracking system are disclosed. An image of an eye of a user from an image sensor is received, the image resulting from the image sensor detecting light from one or more illuminators reflected from the eye of the user. A glint is identified in the image, wherein the glint is a representation in the image of a reflection of light from a cornea of the eye of the user or from a sclera of the eye of the user. A first pixel intensity of the glint is determined, a second pixel intensity of neighbor pixels of the glint is determined, and an absolute value of the difference between the first pixel intensity of the glint and the second pixel intensity of the neighbor pixels of the glint is determined. … Fig. 1, 3A, Abstract.
Sangu et al. (US 20190286228 A1) - An eyeball-tilt detecting device for detecting a tilt position of an eyeball, including: a light source array including a plurality of light emitting sections that emit light having directivity; an optical spot position detection element configured to detect an optical spot position of light reflected from an eyeball to output a detection signal; and processing circuitry configured to calculate a tilt position of the eyeball based on the detection signal output from the optical spot position detection element.… Fig. 1, 7, Abstract.
Miller et al. (US 20190243448 A1) - A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system images the user's eye and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye. The display system may render virtual image content with a render camera positioned at the determined position of the center of rotation of said eye.… Fig. 7A, Abstract.
Trail et al. (US 20180246590 A1) - Disclosed is a system and method for tracking a user's eye using structured light. The structured light system is calibrated by training a model of surface of the user's eye. A structured light emitter projects a structured light pattern (e.g., infrared structured light) onto a portion of the surface of the eye. From the viewpoint of a camera, the illumination pattern appears distorted. Based on the distortion of the illumination pattern in the captured image, the eye tracking system can determine the shape of the portion of the user's eye that the structured light is incident upon. By comparing the determined shape of the portion of the user's eye to the model, the orientation of the eye may be determined.… Figs. 2, 3, Abstract.
Ollila et al. (US 20180157910 A1) - A gaze-tracking system for use in a head-mounted display apparatus. The gaze-tracking system includes: at least one first optical element comprising particles of a phosphorescent or fluorescent material dispersed therein, the particles of the phosphorescent or fluorescent material being dispersed in a manner that when excited by electromagnetic radiation incident thereupon, the particles produce structured light of a given wavelength, wherein the produced structured light illuminates a user's eye; at least one camera for capturing an image of reflections of the structured light from the user's eye, wherein the image is representative of a form of the reflections and a position of the reflections on an image plane of the at least one camera… Fig. 2A-C, 3, Abstract.
Lewis et al. (US 20130300653 A1) - The technology provides various embodiments for gaze determination within a see-through, near-eye, mixed reality display device. In some embodiments, the boundaries of a gaze detection coordinate system can be determined from a spatial relationship between a user eye and gaze detection elements such as illuminators and at least one light sensor positioned on a support structure such as an eyeglasses frame. The gaze detection coordinate system allows for determination of a gaze vector from each eye based on data representing glints on the user eye, or a combination of image and glint data. A point of gaze may be determined in a three-dimensional user field of view including real and virtual objects… Fig. 1C-1E, 2, Abstract.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEZAWIT N SHIMELES whose telephone number is (571)272-7663. The examiner can normally be reached M-F 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEZAWIT NOLAWI SHIMELES/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights