CTNF 18/778,045 CTNF 101775 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. DETAILED ACTION The United States Patent & Trademark Office appreciates the application that is submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below. Priority This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of KR-10-2023-0122672 , filed in Korea on 09/14/2023 , and continuation PCT/KR2024/005925 , filed on 05/02/2024 . Specification 07-29 AIA The disclosure is objected to because of the following informalities: In Paragraph [0084], line 7, “second coded image 22 may not include information” should read “second coded image 22 may include information.” In Paragraph [0129], lines 4-5, “the light source 102 may be an infrared light source, and the light source 102 may output light in an ultraviolet band” should read “the light source 102 may be an ultraviolet light source, and the light source 102 may output light in an ultraviolet band.” In paragraph [0150], line 3, “gaze information of the use ” should read “gaze information of the user .” In paragraph [0162], line 6, “may track the gage ” should read “may track the gaze .” In paragraph [0186], line 3, “ focusedat ” should read “ focused at .” Appropriate correction is required. Drawings 06-22 AIA The drawings are objected to because : In Fig. 9, processor is reference character “ 140 ” but paragraphs [0169], [0172], [0176] and [0177] of the specification state “ a processor 250 .” Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance. 12-151 AIA 26-51 12-51 Status of Claims Claims 1-20 are pending. 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. 07-30-06 This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “polarization change element” in claims 1, 4, 10, 13, and 14 . Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20 AIA The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action: (a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made. 07-23 AIA The factual inquiries set forth in Graham v. John Deere Co. , 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21 AIA Claim s 1-9, 11, and 13-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Song et al. (U.S. Patent Pub. No. 2022/0270645, hereafter referred to as Song) in view of Sharma et al. (U.S. Patent No. 11,205,069, hereafter referred to as Sharma) . Regarding Claim 1, Song teaches an electronic device (Paragraph [0054], Song teaches an eye tracking system.) comprising: a polarization filter configured to receive incident light and to change a polarization of the incident light (Paragraphs [0072-73], Song teaches a polarizer (213) which substantially transmits the linearly polarized light (217) having the first polarization, and substantially blocks, via absorption, the linearly polarized light (216) having a second polarization. The polarizer (213) is one component of the polarization separator (210).) ; a polarization change element configured to receive light from the polarization filter (Paragraphs [0071-72], Song teaches a polarization switch (212) which may be an active element controllable to operate between a switching and non-switching state. The polarization switch (212) operating in a switching state may be configured to convert the polarization of a polarized input light from a first polarization to a second polarization different from the first polarization, while transmitting the polarized light input therethrough. The polarization switch (212) is one component of the polarization separator (210). The elements of the polarization switch can be arranged in other configurations.) ; a metalens configured to refract light received from the polarization change element into different paths according to a polarization direction of the light received from the polarization change element (Paragraphs [0076-77], Song teaches the polarization separator (210) may also include a selective steering assembly for providing a spatial multiplexing view eye tracking. The polarization selective steering assembly may be configured to steer input lights of different polarizations to different angles (or directions), such that the steering lights may propagate toward different detecting regions of the optical sensor. The polarization selective steering assembly (210) may include a metasurface steering element.) ; an image sensor configured to receive the refracted light from the metalens (Paragraphs [0054], [0077], Song teaches an optical sensor (110). The polarization selective steering assembly may steer the first signal light as a light (228) in a first steering angle and steer the second signal light as a light (229) in a second steering angle different from the first steering angle. Light (228) and light (229) may be received by different detecting regions of the optical sensor.) ; at least one processor (Paragraph [0056], Fig. 1, Song teaches a processor (121).) ; and at least one memory storing at least one instruction that, when executed by the at least one processor, cause the electronic device to (Paragraph [0056], Fig. 1, Song teaches a storage device (123), which may be a memory configured to store data or information, including computer-executable program instructions or codes, which may be executed by the processor (121) to perform various controls or functions of the methods or processes.) : cause the polarization filter to change the incident light to a first polarized light of a first direction and to transmit the first polarized light (Paragraph [0073], Song teaches the polarizer (213) may be a linear polarizer configured to substantially transmit a linearly polarized light having a first polarization (s-polarization).) , obtain, through the image sensor, a first coded image based on the first polarized light (Paragraph [0080], Song teaches generating a first eye tracking image based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction.) , wherein the first polarized light is refracted into a first path by the metalens (Paragraph [0077], Song teaches the polarization selective steering assembly may steer the first signal light (206) as a light (228) in a first steering angle.) , and based on identifying that gaze information of a user may not be obtained from the first coded image: cause the polarization change element to change the first polarized light to a second polarized light of a second direction and transmit the second polarized light (Paragraph [0071], Song teaches a polarization switch (212) which may be an active element controllable to operate between a switching and non-switching state. The polarization switch (212) operating in switching state may be configured to convert the polarization of a polarized input light from a first polarization to a second polarization different from the first polarization, while transmitting the polarized light input therethrough.) , obtain, through the image sensor, a second coded image based on the second polarized light (Paragraph [0080], Song teaches generating a second eye tracking image based on a signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction.) , wherein the second polarized light is refracted into a second path by the metalens (Paragraphs [0077], [0080], Song teaches the polarization selective steering assembly (210) may steer the second signal light (208) as a light (229) in a second steering angle different from a first steering angle. Light (229) may be received by a different detecting region on the optical sensor. The polarization selective steering assembly may include a metasurface steering element.) , and obtain the gaze information of the user from the second coded image (Paragraphs [0080], [0045], Song teaches separate images may be generated based on the received lights of different polarizations (0-degree, 45-degree, 90-degree, 135-degree). For example, a second eye tracking image may be generated based on a signal provided by the photo diodes received by the second linearly polarized light of the 90-degree polarization direction. Different perspective views of the eye may be generated based on the second signal light, thereby providing a spatial multiplexing, multi-view eye tracking. The image of the eye can be used to extract eye tracking information, such as gaze direction.) . Song does not explicitly disclose (based on) identifying that gaze information of a user may not be obtained from the first coded image. Sharma is in the same field of art of eye-tracking by capturing images of the eye and performing image processing on the images in order to determine the position of the eye/gaze. Further, Sharma teaches identifying that gaze information of a user may not be obtained from the first coded image (Col. 11, lines 6-23, Sharma teaches detecting a blink event of an eye by analyzing eye-tracking images captured by a camera where the eye-tracking images do not include bright cornea glints because the eyelid has closed over the cornea, for example. A subsequent gaze direction of the eye may be determined based on the subsequent eye-tracking image.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Song by first determining whether a gaze direction can be determined from the first eye-tracking image that is taught by Sharma, to make the invention that captures eye-tracking images until the captured image contains all the information required to track eye gaze; thus, one of ordinary skilled in the art would be motivated to combine the references since in the event of a blink, the user’s eyelid is closed over the cornea and therefore, the eye-tracking image cannot be used to identify a cornea glint (feature point). A subsequent eye-tracking image may then be captured that includes the pupil while the eye is illuminated (Sharma, Col. 11, lines 6-23) which can be used to track the user’s eye gaze. Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 2, Song in view of Sharma teaches the electronic device of claim 1, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : based on identifying that the gaze information of the user may be obtained from the first coded image (Paragraph [0043], Song teaches a single optical sensor (camera) arranged off-axis with respect to the eye has been used. The single optical sensor may generate a single tracking signal based on the IR light reflected from the eye and deflected by the light deflecting element. The optical sensor may generate an image of the eye based on the tracking signal obtained.) , obtain the gaze information of the user from the first coded image (Paragraphs [0043], [0080], Song teaches the optical sensor may generate an image of the eye, which may be used to extract eye tracking information related to the eye, such as gaze direction. A first eye tracking image may be generated based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction.) . In regards to Claim 3, Song in view of Sharma teaches the electronic device of claim 1, wherein the first direction is perpendicular to the second direction (Paragraph [0045], Song teaches the polarization selective optical element is configured to deflect (e.g., diffract) the second light reflected from the object into a plurality of signal lights. At least two of the plurality of signal lights may have orthogonal polarizations.) . In regards to Claim 4, Song in view of Sharma teaches the electronic device of claim 1, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : based on identifying that the gaze information of the user may not be obtained from the second coded image (Col. 8, lines 37-61, Col. 11, lines 6-23, Sharma teaches the second eye-tracking image (682) may not have the contrast sufficient to determine the pupil-position of the eye. A blink event of the eye may be detected by analyzing eye-tracking images captured by the camera where the eye-tracking images do not include bright cornea glints because the eyelid has closed over the cornea.) : cause the polarization change element to change the first polarized light to a third polarized light of a third direction and transmit the third polarized light (Paragraph [0062], Fig. 1B, Song teaches diffracting the RHCP and LHCP components of light in opposite diffraction angles. The right-hand circularly polarized light and left-hand circularly polarized light may be diffracted at a third positive angle and a third negative angle, respectively. The Examiner interprets the RHCP and LHCP light diffracted at a third angle to be a third polarized light in a third direction.) , obtain, through the image sensor, a third coded image based on the third polarized light (Col. 8, lines 37-61, Sharma teaches determining cornea-position from cornea glints included in the third eye-tracking image (683). A gaze direction may be determined based on a cornea position where a cornea-center is determined by calculating center of a sphere of a cornea. The cornea glints from the eye-tracking image (683) provides the data to determine the cornea position and corresponding cornea-center.) , wherein the third polarized light is refracted into a third path by the metalens (Paragraph [0080], Song teaches receiving light of a predetermined polarization, such as light of 0-degree, 45-degree, 90-degree, or 135-degree polarization. Separate images may be generated based on the received lights of different polarizations. Therefore, the first eye tracking image may be generated based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction, the second eye-tracking image may be generated based on the signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction, and so on.) , and obtain the gaze information of the user from the third coded image (Col. 9, lines 15-41, Col. 8, lines 37-61, Figs. 5B-5E, Sharma teaches determining a gaze direction based on a cornea-position such as a cornea center (515) determined from calculating center of sphere of the cornea. The cornea glints (551) from eye-tracking images 682, 683, and/ or 684 (683 is the “third” eye-tracking image) provide the data to determine the cornea position and corresponding cornea center.) . In regards to Claim 5, Song in view of Sharma teaches the electronic device of claim 1, wherein the first coded image is formed by the first polarized light reaching a first point within an imaging surface of the image sensor along the first path (Paragraph [0080], Song teaches generating a first eye tracking image based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction.) , and wherein the second coded image is formed by the second polarized light reaching a second point within the imaging surface of the image sensor along the second path (Paragraph [0080], Song teaches generating a second eye tracking image based on a signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction.) . In regards to Claim 6, Song in view of Sharma teaches the electronic device of claim 1, wherein the first path does not intersect with an imaging surface of the image sensor (Paragraph [0075], Song teaches the linearly polarized light converted from the first signal light may not be received by the optical sensor. The combination of the waveplate, polarization switch, and the polarizer may alternately transmit the first signal light and the second signal light toward the optical sensor in a time multiplexing manner.) , and wherein the second coded image is formed by the second polarized light reaching the imaging surface of the image sensor along the second path (Paragraphs [0080], [0075], Song teaches generating a second eye tracking image based on a signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction.) . In regards to Claim 7, Song in view of Sharma teaches the electronic device of claim 1, further comprising a first light source configured to output a first light (Paragraph [0052], Fig. 1A, Song teaches the eye tracking system (100) may include one or more light sources (105) configured to emit light (102) to illuminate one or two eyes (115) of a person, such as a user of the eye tracking system.) , PNG media_image1.png 715 580 media_image1.png Greyscale wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : cause the first polarized light to be transmitted by allowing the first light to pass through the polarization filter (Paragraph [0073], Song teaches a polarizer (213) which substantially transmits the linearly polarized light (217) having the first polarization, and substantially blocks, via absorption, the linearly polarized light (216) having a second polarization.) . In regards to Claim 8, Song in view of Sharma teaches the electronic device of claim 4, further comprising: a first light source configured to output a first light (Paragraph [0052], Fig. 1A, Song teaches the eye tracking system (100) may include one or more light sources (105) configured to emit light (102) to illuminate one or two eyes (115) of a person, such as a user of the eye tracking system.) ; and a second light source configured to output a second light (Paragraph [0052], Fig. 1A, Song teaches a second light source (105).) , wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : cause the first polarized light to be transmitted by allowing the first light to pass through the polarization filter (Paragraph [0073], Song teaches the polarizer (213) may be a linear polarizer configured to substantially transmit a linearly polarized light having a first polarization. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets “or” to mean only one of the limitations are required.) or cause the third polarized light to be transmitted by allowing the second light to pass through the polarization filter . In regards to Claim 9, Song in view of Sharma teaches the electronic device of claim 8, wherein a wavelength region of the first light and a wavelength region of the second light are different from each other (Paragraph [0095], Song teaches the first p-polarized having a wavelength range within the first wavelength range, and the second s-polarized light having a wavelength range within the second wavelength range.) . In regards to Claim 11, Song in view of Sharma teaches the electronic device of claim 1, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : obtain a feature point related to an eye from the second coded image (Col. 9, lines 15-32, Sharma teaches obtaining cornea-position information (cornea glints from eye tracking images 682, 683, and/or 684. (682 is the “second” eye-tracking image).) , PNG media_image2.png 243 290 media_image2.png Greyscale and obtain the gaze information of the user based on the feature point (Col. 9, lines 15-41, Sharma teaches determining a gaze direction of the eye based at least in part on a pupil-center coordinate such as the pupil center. A gaze direction may also be determined based at least in part on a cornea position where a cornea-center is calculated. The cornea glints from eye-tracking images provide the data to determine the cornea position.) . In regards to Claim 13, Song in view of Sharma teaches the electronic device of claim 1, wherein the metalens comprising a surface formed of a plurality of nanostructures arranged on a substrate (Paragraph [0044], Song teaches polarization selective optical elements may be fabricated based on isotropic or anisotropic materials, and may include sub-wavelength structures, a birefringent material, a photo-reactive holographic material, or a combination thereof.) , wherein the metalens is configured to refract light received from the polarization change element into different paths by modulating a phase of the light received from the polarization change element based on one of a polarization of the light received from the polarization change element (Paragraphs [0044], [0047], Song teaches a Pancharatnam-Berry phase (“PBP”) element (polarization selective optical element) which may modulate a circularly polarized light based on a phase profile provided through a geometric phase. The PBP grating may be configured to operate in a positive state to forwardly diffract a first circularly polarized light with a first predetermined handedness in a positive diffraction angle, and operate in a negative state to forwardly diffract a second circularly polarized light having a second predetermined handedness opposite to the first predetermined handedness in a negative diffraction angle.) and a wavelength of the light received from the polarization change element (Paragraph [0059], Song teaches the PBP elements may be configured to operate for lights having different wavelength ranges.) . In regards to Claim 14, Song in view of Sharma teaches the electronic device of claim 13, wherein the metalens is further configured to refract light received from the polarization change element into different paths based on at least one of a size of each nanostructure of the plurality of nanostructures (Paragraph [0044], Song teaches the polarization selective optical elements such as PBP may be fabricated to include sub-wavelength structures.) , a shape of each nanostructure of the plurality of nanostructures (Paragraph [0044], Song teaches an optic axis of a PBP element may have a spatially varying orientation in at least one in-plane direction and may also have a spatially varying orientation in an out-of-plane direction. The term “optic axis” may refer to a direction in a crystal.) , and an arrangement of the plurality of nanostructures on the substrate (Paragraph [0044], Song teaches the PBP may be fabricated based on isotropic or anisotropic materials.). In regards to Claim 15, Song teaches method (Paragraph [0051], Song teaches an eye tracking method.) comprising: obtaining a first coded image by receiving first polarized light refracted into a first path by a metalens (Paragraphs [0075], [0080], [0077], Song teaches generating different images of the eye (115) respectively based on the first signal light. A first perspective view of the eye may be obtained from a first image generated based on the first signal light. For example, using the pixelated polarized camera, each photo diode may receive light of a predetermined polarization, such as light of 0-degree polarization direction. A first eye tracking image may be generated based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction. The polarization selective optical assembly, which may include a metasurface, may steer the first signal light as a light at a first steering angle toward the optical sensor (and photo diode).) , wherein the metalens is configured to refract incident light into different paths according to a polarization direction of the incident light (Paragraph [0077], Song teaches a polarization selective steering assembly which may include a metasurface steering element. The polarization selective steering assembly may steer the first signal light as a light (228) in a first steering angle, and steer a second light (229) as a light in a second steering angle different from the first steering angle. Light (228) and light (229) may be received by different detecting regions of the optical sensor.) ; identifying whether gaze information of a user may be obtained from the first coded image ; based on identifying that the gaze information of the user may not be obtained from the first coded image : obtaining a second coded image by receiving second polarized light refracted into a second path by the metalens (Paragraph [0080], Song teaches generating a second eye tracking image based on a signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction.) ; and obtaining the gaze information of the user from the second coded image (Paragraphs [0080], [0045], Song teaches generating separate eye tracking images based on the received lights of different polarizations (0-degree, 45-degree, 90-degree, 135-degree). For example, a second eye tracking image may be generated based on a signal provided by the photo diodes received by the second linearly polarized light of the 90-degree polarization direction. Different perspective views of the eye may be generated based on the second signal light, thereby providing a spatial multiplexing, multi-view eye tracking. The image of the eye can be used to extract eye tracking information, such as gaze direction.) . Song does not explicitly disclose identifying whether gaze information of a user may be obtained from the first coded image; (based on) identifying that the gaze information of the user may not be obtained from the first coded image. Sharma is in the same field of art of eye-tracking by capturing images of the eye and performing image processing on the images in order to determine the position of the eye. Further, Sharma teaches identifying whether gaze information of a user may be obtained from the first coded image (Col. 11, lines 6-23, Sharma teaches detecting a blink event of an eye by analyzing eye-tracking images captured by a camera where the eye-tracking images do not include bright cornea glints because the eyelid has closed over the cornea, for example. A subsequent gaze direction of the eye may be determined based on the subsequent eye- tracking image.); (based on) identifying that the gaze information of the user may not be obtained from the first coded image (Col. 11, lines 6-23, Sharma teaches capturing a subsequent eye-tracking image that includes the pupil while the eye is illuminated at a first intensity of non-visible light after detecting a blink event of an eye. A subsequent gaze direction of the eye may be determined based on the subsequent eye-tracking image.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Song by detecting whether or not a blink occurred when the eye-tracking image was taken by analyzing the eye-tracking image(s) captured by a camera, for example, where the eye-tracking images do not include bright cornea glints because the eyelid has closed over the cornea, and taking a subsequent eye tracking image if a blink is detected, that is taught by Sharma, to make the invention that detects whether an image may be used for gaze tracking and if it cannot, takes a subsequent eye-tracking image; thus, one of ordinary skilled in the art would be motivated to combine the references since a single tracking signal generated by a single optical sensor (camera) may provide insufficient information for determining accurate eye tracking information and the accuracy in gaze tracking based on a single tracking signal may be poor (Song, Paragraph [0043]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 16, Song discloses an electronic device (Paragraph [0054], Fig. 1A, Song teaches an eye tracking system (100).) comprising: a first light source configured to output a first light (Paragraph [0052], Fig. 1A, Song teaches one or more light sources (105). The light source (105) may emit the light (102).) ; a second light source configured to output a second light (Paragraph [0052], Fig. 1A, Song teaches a second light source (105).) ; a metalens configured to refract incident light into different paths according to a wavelength of the incident light (Paragraph [0059], Song teaches the polarization selective optical element may include one or more PBP elements (122) configured to operate for lights having different wavelengths. The PBP element may be configured to operate in a positive state for a circularly polarized light having a predetermined handedness, and operate in a negative state for a circularly polarized light having handedness opposite to the predetermined handedness.) ; an image sensor configured to receive light refracted by the metalens (Paragraphs [0054], [0077], Song teaches an optical sensor (110). The polarization selective steering assembly may steer the first signal light as a light (228) in a first steering angle and steer the second signal light as a light (229) in a second steering angle different from the first steering angle. Light (228) and (229) may be received by different detecting regions of the optical sensor.) ; at least one processor (Paragraph [0056], Fig. 1, Song teaches a processor (121).) ; and at least one memory storing at least one instruction that, when executed by the at least one processor, cause the electronic device to (Paragraph [0056], Song teaches a storage device (123) which may be a memory. The storage device (123) may be configured to store data or information, including computer-executable program instructions or codes, which may be executed by the processor.) : obtain, through the image sensor, a first coded image (Paragraph [0080], Song teaches generating a first eye tracking image based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction.) , wherein the first coded image is based on the first light refracted into a first path by the metalens (Paragraph [0077], Song teaches the polarization selective steering assembly may steer the first signal light (206) as a light (228) in a first steering angle.) , and based on identifying that gaze information of a user may not be obtained from the first coded image , obtain gaze information of the user from a second coded image obtained through the image sensor (Paragraphs [0080], [0045], Song teaches separate images may be generated based on the received lights of different polarizations (0-degree, 45-degree, 90-degree, 135-degree). For example, a second eye tracking image may be generated based on a signal provided by the photo diodes received by the second linearly polarized light of the 90-degree polarization direction. Different perspective views of the eye may be generated based on the second signal light, thereby providing a spatial multiplexing, multi-view eye tracking. The image of the eye can be used to extract eye tracking information, such as gaze direction.) , wherein the second coded image is based on the second light refracted into a second path by the metalens (Paragraph [0080], Song teaches generating a second eye tracking image based on a signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction.) . Song does not explicitly disclose (based on) identifying that gaze information of a user may not be obtained from the first coded image. Sharma is in the same field of art of eye-tracking by capturing images of the eye and performing image processing on the images in order to determine the position of the eye. Further, Sharma teaches identifying that gaze information of a user may not be obtained from the first coded image (Col. 11, lines 6-23, Sharma teaches detecting a blink event of an eye by analyzing eye-tracking images captured by a camera where the eye-tracking images do not include bright cornea glints because the eyelid has closed over the cornea, for example. A subsequent gaze direction of the eye may be determined based on the subsequent eye-tracking image.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Song by first determining whether a gaze direction can be determined from the first eye-tracking image that is taught by Sharma, to make the invention that captures eye-tracking images until one is sufficient to track eye gaze; thus, one of ordinary skilled in the art would be motivated to combine the references since in the event of a blink, the eyelid is closed over the cornea and therefore, the eye-tracking image cannot be used to identify a cornea glint. A subsequent eye-tracking image may then be captured that includes the pupil while the eye is illuminated (Sharma, Col. 11, lines 6-23). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. In regards to Claim 17, Song in view of Sharma teaches the electronic device of claim 16, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : based on identifying that the gaze information of the user may be obtained from the first coded image (Col. 8, lines 25-36, Col. 11, lines 1-5, Sharma teaches capturing a first eye-tracking image that includes the pupil of the eye. The eye tracking image has sufficient contrast to determine a difference between iris and pupil by way of image processing.) , obtain the gaze information of the user from the first coded image (Col. 9, lines 42-58, Sharma teaches using the pupil-center coordinate of the first-eye tracking image to assist in determining the gaze direction.). In regards to Claim 18, Song in view of Sharma discloses the electronic device of claim 16, wherein a wavelength region of the first light and a wavelength region of the second light are different from each other (Paragraph [0095], Song teaches the first p-polarized having a wavelength range within the first wavelength range, and the second s-polarized light having a wavelength range within the second wavelength range.) . In regards to Claim 19, Song in view of Sharma teaches the electronic device of claim 16, further comprising a third light source configured to output a third light (Col. 8, lines 25-61, Fig. 6A, Sharma teaches an array of illuminators (436A-D) which emit non-visible illumination light toward the eye. The Examiner interprets reference character “436C” to be the “third illuminator.”) , wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to: (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) based on identifying that the gaze information of the user may not be obtained from the second coded image (Col. 8, lines 37-61, Col. 11, lines 6-23, Sharma teaches the second eye-tracking image (682) may not have the contrast sufficient to determine the pupil-position of the eye. Also, a blink event of the eye may be detected by analyzing eye-tracking images captured by a camera.) , obtain the gaze information of the user from a third coded image obtained through the image sensor (Col. 8, lines 37-61, Sharma teaches determining cornea-position from cornea glints included in the third eye-tracking image (683). A gaze direction may be determined based on a cornea position where a cornea-center is determined by calculating center of a sphere of a cornea. The cornea glints from the eye-tracking image (683) provides the data to determine the cornea position and corresponding cornea-center.), wherein the third coded image is based on the third light refracted into a third path by the metalens (Paragraph [0080], Song teaches receiving light of a predetermined polarization, such as light of 0-degree, 45-degree, 90-degree, or 135-degree polarization. Separate images may be generated based on the received lights of different polarizations. Therefore, the first eye tracking image may be generated based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction, the second eye-tracking image may be generated based on the signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction.) . In regards to Claim 20, Song in view of Sharma discloses the electronic device of claim 16, wherein the first coded image is formed by the first light reaching a first point within an imaging surface of the image sensor along the first path (Paragraph [0080], Song teaches through using the pixelated polarized camera (230), each photo diode (265) may receive light of a predetermined polarization, such as light of 0-degree polarization direction. Separate images may be generated based on the received lights of different polarizations. A first eye tracking image may be generated based on a signal provided by the photo diodes receiving the first linearly polarized light of the 0-degree polarization direction. The pixelated polarized camera (230) may receive the first signal light by the different photo diodes corresponding to the micro-polarizers having a first polarization direction (i.e., 0-degree polarization direction).) , and wherein the second coded image is formed by the second light reaching a second point within the imaging surface of the image sensor along the second path (Paragraph [0080], Song teaches through using the pixelated polarized camera (230), each photo diode (265) may receive light of a predetermined polarization, such as light of 90-degree polarization direction. Separate images may be generated based on the received lights of different polarizations. A second eye tracking image may be generated based on a signal provided by the photo diodes receiving the second linearly polarized light of the 90-degree polarization direction. The pixelated polarized camera (230) may receive the second signal light by the different photo diodes corresponding to the micro-polarizers having a second polarization direction (i.e., 90-degree polarization direction).) . 07-21 AIA Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Song et al. (U.S. Patent Pub. No. 2022/0270645, hereafter referred to as Song) in view of Sharma et al. (U.S. Patent No. 11,205,069, hereafter referred to as Sharma) in further view of Wheelwright et al. (U.S. Patent No. 10,768,371, hereafter referred to as Wheelwright) . Regarding Claim 10, Song in view of Sharma discloses the electronic device of claim 1, further comprising: a first light source configured to output a first light (Paragraph [0052], Fig. 1A, Song teaches one or more light sources (105). The light source (105) may emit the light (102).) ; and a second light source configured to output a second light (Paragraph [0052], Fig. 1A, Song teaches a second light source (105).) , PNG media_image3.png 711 620 media_image3.png Greyscale wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : cause the first polarized light to be transmitted by allowing the first light to pass through the polarization filter (Paragraph [0073], Song teaches the polarizer (213) may be a linear polarizer configured to substantially transmit a linearly polarized light having a first polarization.) , cause the polarization change element to transmit the first polarized light (Paragraphs [0074], [0072], Song teaches the polarization switch (212) may operate in a non-switching state to substantially maintain the polarizations of the linearly polarized light having the first polarization. Note that the polarization switch may be disposed between the waveplate and the polarizer (213). The configuration shown in Fig. 2B is merely one example.) or to change the first polarized light to the second polarized light and transmit the second polarized light (Paragraphs [0073], [0072], Song teaches alternatively, a polarization switch (212) may operate in switching state to convert the linearly polarized light having the first polarization to a linearly polarized light having a second polarization.) , cause a third polarized light of a third direction to be transmitted by allowing the second light to pass through the polarization filter, cause the polarization change element to transmit the third polarized light or to change the third polarized light to a fourth polarized light of a fourth direction and transmit the fourth polarized light , and obtain, through the image sensor, the gaze information of the user from a third coded image based on the first polarized light, the second polarized light, the third polarized light, or the fourth polarized light (Col. 8, lines 37-61, Col. 5, lines1-3, Fig. 5A & 6A, Sharma teaches obtaining a third-eye tracking image (683) while illuminators emit non-visible illumination light at the second intensity that is lower than the first intensity. The third eye-tracking image may be used to determine a cornea position from cornea glints included in the eye-tracking images. Combiner optical element (440) may include a polarization selective volume hologram that reflects a first polarization orientation of the cornea-reflected light and passes polarization orientations that are other than the first polarization orientation toward the camera to generate an image.) . Song in view of Sharma does not explicitly disclose caus e (ing) a third polarized light of a third direction to be transmitted by allowing the second light to pass through the polarization filter, caus e (ing) the polarization change element to transmit the third polarized light or to change the third polarized light to a fourth polarized light of a fourth direction and transmit the fourth polarized light. Wheelwright is in the same field of art of performing eye tracking based on magnitudes of image light captured over a plurality of time instants and determining information such as eye position information and angle of an eye-gaze. Further, Wheelwright teaches caus e (ing) a third polarized light of a third direction to be transmitted by allowing the second light to pass through the polarization filter (Col. 7, lines 38-50, Wheelwright teaches transforming the image light of the second polarization into a second portion of image light having a third polarization by directly transmitting the image light of the second polarization.) , caus e (ing) the polarization change element to transmit the third polarized light or to change the third polarized light to a fourth polarized light of a fourth direction and transmit the fourth polarized light (Claim 1, Wheelwright teaches transforming the image light of the first polarization into a first portion of image light of a third polarization, transforming the image light of the second polarization to a second portion of image light of a third polarization, and directing both the first and second portions of image light toward an eye-box. The Examiner interprets directing the first and second portions of image light having a third polarization to be “transmitting.” Under BRI, the Examiner interprets “or” to mean only one of the limitations is required.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Song in view of Sharma by transmitting a polarized light of a third direction and further allowing the third polarized light through and directing it toward the eye that is taught by Wheelwright, to make the invention that transmits lights of a plurality of directions towards the eye to generate an eye-tracking image having sufficient information to determine the user’s eye gaze; thus, one of ordinary skilled in the art would be motivated to combine the references since a single eye tracking signal generated by a single optical sensor may provide insufficient information for determining accurate eye tracking information. For example, the accuracy in gaze tracking based on the single tracking signal may be poor with a single eye tracking signal (Song, Paragraph [0043]). By generating multiple images of the eye using a plurality of optical sensors and a plurality of signal lights, multiple perspective views of the eye can be created, thereby improving the accuracy of eye tracking, especially when the eye moves (Song, Paragraph [0058]). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention . 07-21 AIA Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Song et al. (U.S. Patent Pub. No. 2022/0270645, hereafter referred to as Song) in view of Sharma et al. (U.S. Patent No. 11,205,069, hereafter referred to as Sharma) in further view of Lee et al. (U.S. Patent No. 11,874,474, hereafter referred to as Lee) . Regarding Claim 12, Song in view of Sharma discloses the electronic device of claim 1, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to (Paragraph [0136], Song teaches a computer processor for performing any or all of the steps, operations, or processes described.) : obtain a restored image related to an eye from the second coded image, and obtain the gaze information of the user based on the restored image. Song in view of Sharma does not explicitly disclose obtain(ing) a restored image related to an eye from the second coded image, and obtain(ing) the gaze information of the user based on the restored image. Lee is in the same field of art of eye gaze tracking. Further, Lee teaches obtain(ing) a restored image related to an eye from the second coded image (Col. 21, lines 25-35, Lee teaches generating a third image by subtracting the intensity of the corresponding pixels in the first image and second image, which primarily includes information of the secondary signal light, thereby providing a second perspective view of the eye.) , and obtain(ing) the gaze information of the user based on the restored image (Col. 23, lines 12-33, Lee teaches generating a third image based on the secondary signal lights and second image enables time-multiplexing multi-view eye-tracking. The image of the eye may be used to extract desired information such as gaze direction.) . Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Song in view of Sharma by performing processing on the second captured image to generate a third image to more accurately extract eye tracking information that is taught by Lee, to make the invention that extracts object (eye)-tracking information with enhanced accuracy; thus, one of ordinary skilled in the art would be motivated to combine the references since performing further processing on the first image and the second image that is generated based on a mixture of the primary signal light and the secondary signal lights, such that a third image that is generated based on the secondary signal lights can be obtained to enable a time-multiplexing multi-view eye tracking (Lee, Col. 23, lines 1-33). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Pertinent Prior Art 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Ouderkirk et al. (U.S. Patent No. 10,852,817 B1) Thukral et al. (U.S. Patent Pub. No. 2015/0242680 A1) Arar et al. (NPL “Robust real-time multi-view eye tracking,” 2017) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYDNEY L BLACKSTEN whose telephone number is (571)272-7651. The examiner can normally be reached 8:30am-5pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /SYDNEY L BLACKSTEN/Examiner, Art Unit 2674 /ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674 Application/Control Number: 18/778,045 Page 2 Art Unit: 2674 Application/Control Number: 18/778,045 Page 3 Art Unit: 2674 Application/Control Number: 18/778,045 Page 4 Art Unit: 2674 Application/Control Number: 18/778,045 Page 5 Art Unit: 2674 Application/Control Number: 18/778,045 Page 6 Art Unit: 2674 Application/Control Number: 18/778,045 Page 7 Art Unit: 2674 Application/Control Number: 18/778,045 Page 8 Art Unit: 2674 Application/Control Number: 18/778,045 Page 9 Art Unit: 2674 Application/Control Number: 18/778,045 Page 10 Art Unit: 2674 Application/Control Number: 18/778,045 Page 11 Art Unit: 2674 Application/Control Number: 18/778,045 Page 12 Art Unit: 2674 Application/Control Number: 18/778,045 Page 13 Art Unit: 2674 Application/Control Number: 18/778,045 Page 14 Art Unit: 2674 Application/Control Number: 18/778,045 Page 15 Art Unit: 2674 Application/Control Number: 18/778,045 Page 16 Art Unit: 2674 Application/Control Number: 18/778,045 Page 17 Art Unit: 2674 Application/Control Number: 18/778,045 Page 18 Art Unit: 2674 Application/Control Number: 18/778,045 Page 19 Art Unit: 2674 Application/Control Number: 18/778,045 Page 20 Art Unit: 2674 Application/Control Number: 18/778,045 Page 21 Art Unit: 2674 Application/Control Number: 18/778,045 Page 22 Art Unit: 2674 Application/Control Number: 18/778,045 Page 23 Art Unit: 2674 Application/Control Number: 18/778,045 Page 24 Art Unit: 2674 Application/Control Number: 18/778,045 Page 25 Art Unit: 2674 Application/Control Number: 18/778,045 Page 26 Art Unit: 2674 Application/Control Number: 18/778,045 Page 27 Art Unit: 2674 Application/Control Number: 18/778,045 Page 28 Art Unit: 2674 Application/Control Number: 18/778,045 Page 29 Art Unit: 2674 Application/Control Number: 18/778,045 Page 30 Art Unit: 2674 Application/Control Number: 18/778,045 Page 31 Art Unit: 2674 Application/Control Number: 18/778,045 Page 32 Art Unit: 2674 Application/Control Number: 18/778,045 Page 33 Art Unit: 2674
Prosecution Insights