DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
2. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
3. Claims 16-17, 19, and 29 are objected to because of the following informalities:
Claim 16, line 17, should read “individually controllable;[[ and]]”
Claim 17, line 7, should read “via ”
Claim 17, line 8, should read “projection area of the redirection unit”
Claim 17 should end with a period instead of a comma
Claim 19, line 2, should presumably read “ascertain a respective position”
Claim 29, lines 2-3, should read “wherein the third holographic optical element layer has” to maintain proper antecedent basis
Appropriate correction is required.
Claim Rejections - 35 USC § 102
4. The following is a quotation of the appropriate paragraphs of 35 USC 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
5. Claims 16-24 and 26-33 are rejected under 35 USC 102(a)(1) as being anticipated by Tremblay et al. (US 20170293147 A1).
Regarding claim 16, Tremblay discloses an optical system for a virtual retinal scan display, comprising:
an image source (Fig. 40, camera 3501) which provides image content in the form of image data ([0203], “3501 grabs an image”);
an image processing device for the image data ([0199], “3501 grab an image which is then processed”);
a projector unit (Fig. 40, 3801 for each eye) with a time-modulatable first light source (any light source capable of being turned on/off is inherently time-modulatable) configured to generate at least one first light beam (Fig. 39, λ1 for a first eye) and with a controllable deflection device for the at least one first light beam (Fig. 39, 2703 deflects λ1) for scanning projection of the image content ([0202], “light source 3801…is…deflected by a 2D scanner 2703”), and with a second light source ([0196], each light source “contains one or more light sources”; [0201], “super luminescent diodes and vcsels”) configured to generate second light beams (Fig. 39, λ2 for a first eye) in a first infrared wavelength range ([0196], “using an additional wavelength in the infrared”), wherein the controllable deflection device is configured to deflect second light beams in a scanning manner (Fig. 39, 2703 deflects λ2 for a first eye);
a redirection unit (Figs. 39-40, 204), onto which the image content is projectable ([0196], “light is directed…to an appropriate holographic transflector 204”) and which is configured to direct the projected image content and the second light beams toward an eye of a user (Fig. 40, toward 209);
an optical segmentation element (Fig. 39, 3901 for each eye), which is arranged between the projector unit and the redirection unit (Fig. 39, 3901 is between 3801 and 204) and using which the image content and the second light beams are projectable via different imaging paths ([0202], “3901…produces two distinct diverging beams 3902 and 3903”) onto at least one projection area of the redirection unit (Figs. 39-40, onto 204) so that, at different times ([0188], “a single exit pupil needs to be displayed at any moment in time”), a plurality of spatially-offset first exit pupils (Fig. 6, 7 positions; for a user’s first eye) is produced ([0202], “204…forms…multiple exit pupils”), wherein at least individual imaging paths are individually controllable ([0014]-[0015]); and
an optical replication component (Fig. 40, side of 204 facing a user), which is arranged in the at least one projection area of the redirection unit (Figs. 39-40, in the projection area(s) of 204) and is configured to direct the projected image content in a replicated manner toward the eye of the user (Fig. 40, toward 209) so that, at different times ([0188], “a single exit pupil…at any moment”), a plurality of spatially-offset second, replicated exit pupils is produced with the image content (Fig. 6, 7 positions; for a user’s second eye), wherein the optical replication component is additionally configured to replicate the second light beams toward the eye of the user (Figs. 40, toward 209);
a first sensor (Fig. 40, 2603) configured to detect second light beams ([0204], “detector…2603” is configured to detect λ2 for a first eye) backscattered by an outer eye surface of the user ([0196], “measurement of the backscattered light from a vein”), or a modulation of a laser power of the second light source ([0077], “modulating in intensity each one of the plurality of light beams…and directing the reflected plurality of light beams…to a detector”); and
a computing unit ([0198]) configured to, depending on the backscattered second light beams detected using the first sensor ([0198]) or on the modulation of the power of the second light source ([0077]), to ascertain different positions of the first exit pupils relative to a pupil center ([0198]), and different positions of second exit pupils relative to the pupil center ([0198]), and to differentiate the ascertained positions of the ascertained first exit pupils relative to the pupil center and the ascertained positions of the ascertained second exit pupils relative to the pupil center ([0198]), for image processing (Fig. 34).
Regarding claim 17, Tremblay discloses wherein:
the projector unit includes a third light source configured to generate third light beams (Figs. 39-40, 3801 includes λ2 for a second eye) in a second infrared wavelength range different from the first infrared wavelength range ([0199], “using a binocular projection display…[to] generate augmented reality with 3 dimensional information” causes different light to be projected to each eye), wherein the controllable deflection device is configured to deflect the at least one third light beam in a scanning manner (Figs. 39, 2703 deflects λ2 for a second eye);
the optical segmentation element is configured to project the image content, the second light beams, and the third light beams vis different imaging paths ([0202], “3901…produces two distinct diverging beams 3902 and 3903” for each eye) on the least at least one projection of the redirection unit (Fig. 39-40, onto 204) so that, at the different times ([0188], “a single exit pupil…at any moment), the plurality of spatially-offset first exist pupils is produced (Fig. 6, 7 positions; for a user’s first eye);
the optical replication component is configured to direct the third light beams toward the eye of the user (Fig. 40, side of 204 facing a user is configured to direct λ2 toward a second eye 209);
(i) the first sensor is configured to detect third light beams backscattered by the outer eye surface or a modulating of a laser power of the third light source and/or (ii) the optical system includes a second sensor configured to detect the third light beams (Fig. 40, 2603) backscattered by the outer eye surface ([0196]) or the modulating of the laser power of the third light source ([0077]); and
the computing unit ([0198]) is configured to, depending on the backscattered second light beams detected using the first sensor ([0198]) or on the modulation of the power of the second light source ([0077]) and depending on the detected backscattered third light beams ([0198]) or on the modulation of the third light source ([0077]), to ascertain the different positions of the first exit pupils relative to a pupil center ([0198]), and the different positions of second exit pupils relative to the pupil center ([0198]), and to differentiate the ascertained positions of the ascertained first exit pupils relative to the pupil center ([0198]) and the ascertained positions of the ascertained second exit pupils relative to the pupil center ([0198]), for the image processing (Fig. 34),
Regarding claim 18, Tremblay discloses wherein the computing unit is configured to ascertain the positions ([0198]) of only those of the first and second exit pupils that impinge on a retina of the user (Fig. 34, 3401 of each eye) at a time of detecting the backscattered second light beams ([0196]) or the modulation of the power of the second light source ([0077]) and/or at a time of detecting the backscattered third light beams ([0196]) or the modulation of the power of the third light source ([0077]).
Regarding claim 19, Tremblay discloses wherein the computing unit is additionally configured to ascertain a respective portion ([0198]) of the first and second exit pupils that impinges on the retina of the user (Fig. 34, 3401 of each eye) at the time of detecting the backscattered second light beams ([0196]) or the modulation of the power of the second light source ([0077]) and/or at the time of detecting the backscattered third light beams ([0196]) or the modulation of the power of the third light source ([0077]).
Regarding claim 20, Tremblay discloses wherein the computing unit is configured to represent the ascertained position of a first exit pupil of the first exit pupils differently in intensity in an image than the ascertained position of a second exit pupil of the second exit pupils produced simultaneously with the first exit pupil ([0087]).
Regarding claim 21, Tremblay discloses wherein the redirection unit is configured to direct the second light beams (Figs. 39-40, 204 directs λ2) onto the first exit pupils toward the eye of the user (Abstract, “to create for each light beam an exit pupil at the eye”).
Regarding claim 22, Tremblay discloses wherein the optical replication component is configured to direct the replicated second light beams (Fig. 40, side of 204 facing a user is configured to direct 3904) onto the second exit pupils toward the eye of the user (Fig. 6).
Regarding claim 23, Tremblay discloses wherein the optical replication component is configured to scan the third light beams over an entire area of an eye region including the pupil of the user (Fig. 34).
Regarding claim 24, Tremblay discloses wherein the optical replication component is configured to direct the third light beams onto the second exit pupils (Fig. 40, side of 204 facing user is configured to direct λ2 onto a user’s second eye).
Regarding claim 26, Tremblay discloses wherein the projector unit ([0201], “light source 3801, such as…light emitting diodes”) is configured to combine the first light beam and the second light beams into a common light beam ([0014], “LED light sources are…beam combined…to form the light source”).
Regarding claim 27, Tremblay discloses wherein the projection unit ([0201], “light source 3801, such as…light emitting diodes”) is configured to combine the first light beam, the second light beams, and the third light beams into a common light beam ([0014], “LED light sources are…beam combined…to form the light source”).
Regarding claim 28, Tremblay discloses wherein the redirection unit is a first holographic optical element layer ([0201], “holographic transflector 204”), and the optical replication component is a second holographic optical element (Figs. 39-40, side of 204 facing a user).
Regarding claim 29, Tremblay discloses wherein the redirection unit and the optical replication component are a third holographic optical element layer (Fig. 40, 204 and the side which face a user are a third holographic optical element), wherein the third holographic optical element has a first redirection function (Fig. 40, 204 and its side facing a user redirect light to a first eye), which directs the projected image content and the second light beams toward the eye of the user (Figs. 39-40), wherein the third holographic optical element has a second redirection function (Fig. 40, 204 and its side facing a user redirect light to a second eye), which replicates the projected image content and the second light beams (Figs. 39-40) and/or directs the third light beams (Figs. 39-40) toward the eye of the user (Fig. 40, toward 209).
Regarding claim 30, Tremblay discloses wherein the image processing device is configured to, depending on the ascertained positions of the first exit pupils relative to the pupil center and the differentiated ascertained positions of the second exit pupils relative to the pupil center, produce subimage data from the image data ([0011], “For each position of the scan mirror a pixel may be formed on the retina through raster scanning”) such that only one exit pupil produced on a common imaging path, the same image data, is always imaged on a retina of the user ([0188], “a single exit pupil needs to be displayed at any moment in time”).
Regarding claim 31, Tremblay discloses wherein the optical system is a pair of smart glasses ([0203] & Fig. 40, 2601).
Regarding claim 32, Tremblay discloses a method for projecting image contents onto a retina of a user using an optical system which includes:
an image source (Fig. 40, camera 3501) configured to provide image content in the form of image data ([0203], “3501 grabs an image”),
an image processing device for the image data ([0199], “3501 grab an image which is then processed”),
a projector unit (Fig. 40, 3801 for each eye) with a time-modulatable first light source (any light source capable of being turned on/off is inherently time-modulatable) configured to generate at least one first light beam (Fig. 39, λ1 for a first eye) and with a controllable deflection device for the at least one first light beam (Fig. 39, 2703 deflects λ1) for scanning projection of the image content ([0202], “light source 3801…is…deflected by a 2D scanner 2703”), and with a second light source ([0196], each light source “contains one or more light sources”; [0201], “super luminescent diodes and vcsels”) configured to generate second light beams (Fig. 39, λ2 for a first eye) in a first infrared wavelength range ([0196], “using an additional wavelength in the infrared”), wherein the controllable deflection device is configured to deflect the at least one second light beam in a scanning manner (Fig. 39, 2703 deflects λ2 for a first eye),
a redirection unit (Figs. 39-40, 204 for each eye) onto which the image content is projected ([0196], “light is directed…to an appropriate holographic transflector 204”) and which directs the projected image content and the second light beams toward an eye of a user (Fig. 40, toward 209),
an optical segmentation element (Fig. 39, 3901 for each eye) arranged between the projector unit and the redirection unit (Fig. 39, 3901 is between 3801 and 204),
an optical replication component arranged in a projection area of the redirection unit (Fig. 40, side of 204 facing a user),
a first sensor (Fig. 40, 2603), and
a computing unit ([0198], for calibration),
the method comprising the following steps:
projecting the image content and the second light beams using the optical segmentation element via different imaging paths ([0202], “3901…produces two distinct diverging beams 3902 and 3903”) onto at least one projection area of the redirection unit (Figs. 39-40, onto 204) so that at different times ([0188], “a single exit pupil needs to be displayed at any moment in time”), a plurality of spatially-offset first exit pupils (Fig. 6, 7 positions; for a user’s first eye) is produced ([0202], “204…forms…multiple exit pupils”), wherein at least individual imaging paths the different imaging paths are individually controlled ([0014]-[0015]);
replicating the projected image content using the optical replication component and directing the replicated projected image content (Figs. 39-40, side of 204 facing a user replicates/directs the light), spatially-offset (Fig. 39), toward the eye of the user (Fig. 40, toward 209) so that at different times ([0188], “a single exit pupil…at any moment”), a plurality of spatially-offset second, replicated exit pupils is produced with the image content (Fig. 6, 7 positions; for a user’s second eye), and wherein the second light beams are replicated using the optical replication component and directed toward the eye of the user (Figs. 39-40, side of 204 facing a user replicates/directs λ2 toward a first eye 209);
detecting the second light beams backscattered by an outer eye surface of the user ([0196], “measurement of the backscattered light from a vein”), or a modulation of a laser power of the second light source ([0077], “modulating in intensity each one of the plurality of light beams…and directing the reflected plurality of light beams…to a detector”) using the first sensor ([0204], “detector…2603” is configured to detect λ2 for a first eye);
ascertaining, using the computing unit, depending on the backscattered second light beams detected by the first sensor ([0198]) or on the modulation of the power of the second light source ([0077]) different, positions of the first exit pupils relative to a pupil center ([0198]) and different, positions of second exit pupils relative to the pupil center ([0198]) for image processing (Fig. 34).
Regarding claim 33, Tremblay discloses wherein the projector unit includes a third light source configured to generate third light beams (Figs. 39-40, 3801 includes λ2 for a second eye) in a second infrared wavelength range different from the first infrared wavelength range ([0199], “using a binocular projection display…[to] generate augmented reality with 3 dimensional information” causes different light to be projected to each eye), and the controllable deflection device is configured to deflect the third light beams in a scanning manner (Figs. 39, 2703 deflects λ2 for a second eye), wherein the third light beams are directed toward the eye of the user (Figs. 39-40, λ2 is directed toward a second eye 209), wherein: (i) third light beams backscattered by the outer eye surface or a modulation of laser power of the third light source is detected using the first sensor and/or (ii) the third light beams backscattered by the outer eye surface ([0196], “measurement of the backscattered light from a vein”) or the modulation of the laser power of the third light source ([0077], “modulating in intensity each one of the plurality of light beams…and directing the reflected plurality of light beams…to a detector”) is detected using a second sensor of the optical system (Fig. 40; 2603 is configured to detect λ2 for a second eye), and wherein the computing unit ([0198]) ascertains the positions of the first exit pupils relative to the pupil center ([0198]) and different, the positions of the second exit pupils relative to the pupil center ([0198]) for image processing (Fig. 34), also depending on detected third light beams backscattered by the outer eye surface ([0196]) or the detected modulation of the laser power of the third light source ([0077]).
Claim Rejections - 35 USC § 103
6. The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. The factual inquiries for establishing a background for determining obviousness under 35 USC 103 are summarized as follows:
1) Determining the scope and contents of the prior art.
2) Ascertaining the differences between the prior art and the claims at issue.
3) Resolving the level of ordinary skill in the pertinent art.
4) Considering objective evidence present in the application indicating obviousness or nonobviousness.
8. Claim 25 is rejected under 35 USC 103 as being unpatentable over Tremblay in view of Aleem et al. (US 20200142479 A1).
Regarding claim 25, Tremblay fails to explicitly disclose wherein the first sensor and/or the second sensor are photodiodes.
However, Aleem teaches projecting infrared light onto an eye and then detecting a reflection of said light in order to determine gaze position (Abstract), and discloses wherein an “infrared detector 144 may be a single photodiode sensor” ([0068]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Tremblay and Aleem such that the first sensor and/or the second sensor were photodiodes, motivated by determining positions of features on the eye ([0069], “positions of features on the eye can be determined”).
Conclusion
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniel Jeffery Jordan whose telephone number is 571-270-7641. The examiner can normally be reached 9:30a-6:00p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephone Allen can be reached at 571-272-2434. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D. J. J./Examiner, Art Unit 2872
/TRAVIS S FISSEL/Primary Examiner, Art Unit 2872
Prosecution Insights