DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 12 is objected to because of the following informalities: line 10 recites “thecamera” but a space should be placed between these words such that it instead recites “the camera.” Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9 and 10, and therefore claim 11 which depends therefrom, are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 9, line 2 recites “generalized first gaze information,” however, claim 1 from which claim 9 ultimately depends, recites “first gaze information,” that is used to determine “second gaze information,” thus suggesting that the first gaze information of claim 1 is “generalized” as it is used to determine second gaze information. Therefore, it is unclear and indefinite whether the “generalized first gaze information,” is intending to recite antecedence back to the “first gaze information” of claim 1, or if it is a different, separate gaze information.
Further, regarding claim 10, the phrase "preferentially" renders the claim indefinite because it is unclear whether the limitation(s) following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–2, 5–6, 9, 12–15, and 18–19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al., KR 20230157145A (with reference to the provided machine English language translation, herein “Kim”) in view of Liu et al., US Patent Application Publication No. US 2026/0153925 A1 (herein “Liu”).
Regarding claims 1 and 14, with deficiencies of Kim noted in square brackets [], and substantive differences between the claims noted in curly brackets {}, and with claim 1 as exemplary, Kim teaches {an operating method of an electronic device, comprising – claim 1/ An electronic device, comprising: a processor configured to – claim 14} (Kim Abstract, gaze direction detection method of a gaze direction detection device, where page 11, 3rd and 4th full paragraphs teach embodiments including a processor executing instructions):
acquiring an image comprising a person (Kim page 3, ¶6, using a camera, an image 100 of a scene with a user (person) standing in front of the camera is input (acquiring), where the image has the face of the user);
extracting a normalized image comprising a face of the person from the image (Kim page 3, ¶¶8–10, user’s face area is adjusted through a normalization process resulting in a normalized image);
extracting feature information about the face from the normalized image (Kim page 3, ¶¶8–11, key facial features such as the user’s eyes, nose and mouth are predicted, and then converted to the coordinate system of the normalized image);
outputting, based on the feature information, pose information represented in a camera coordinate system which is a coordinate system of a camera acquiring the image (Kim page 3, ¶8, the user’s head pose is detected based on the camera coordinate system using the key facial features) and first gaze information represented in a facial coordinate system which is a coordinate system of the face (Kim page 3, ¶¶11–12, the gaze direction information is inferred using the related information (including 104 – the key facial features) in the normalized image coordinate system (facial coordinate system which facial features are also found within, thus “a coordinate system of the face”)); and
determining, based on the pose information and the first gaze information, second gaze information represented in the camera coordinate system (Kim page 3, ¶12, the gaze direction information inferred from the normalized image coordinate system is converted to the gaze direction in the input camera coordinate system by inversely transforming using the coordinate transformation relationship between the original input image and the normalized image, and the user's gaze direction (300) in the input camera coordinate system) that comprises a second unit vector from an origin of the facial coordinate system toward a gaze target represented in the camera coordinate system (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301 connecting the user’s gaze position 106 and the gaze point 201, where fig. 3, and page 3, ¶11, teach point 106 as an origin point of the normalized image coordinate system in which pose 105 is defined),
wherein the pose information comprises: [rotation information] and position information about the face (Kim page 3, ¶¶7–8, fig. 2, pose information 103, comprised of vectors originating from user position point 104 on the user’s face),
wherein the first gaze information comprises: a first unit vector (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301) [represented by a rotation of eyes from the origin of the facial coordinate system] toward the gaze target, in the facial coordinate system (Kim page 3, ¶¶ 6–11, figs. 2 and 3, gaze direction in the normalized image coordinate system (facial coordinate system) is determined with respect to a user’s eyes looking at an object of the gaze at gaze point 200).
While Kim teaches on page 6, ¶5 that the head pose is inferred by detecting geometric transformation relationships (rotation, movement) between 3D position values, thus suggesting that pose information, in a broader sense, comprises those geometric relationships including rotation and 3D position values, nonetheless, Kim does not explicitly teach the pose information comprising rotation information. However, Liu teaches the pose information comprising rotation information (Liu ¶17, acquiring a rotation matrix and a displacement matrix as the head pose data).
Further, Kim does not explicitly teach where Liu teaches gaze information comprises: a vector represented by a rotation of eyes from the origin of the facial coordinate system (Liu ¶¶30–42, left and right eye rotation matrices (a matrix is comprised of multiple vectors) are acquired from (thus comprised within) the head pose data, determined from the origin of left and right-eye coordinate systems respectively (facial coordinate system)).
Therefore, taking the teachings of Kim and Liu together as a whole, it would have been obvious to a person having ordinary skill in the art (herein “PHOSITA”) before the effective filing date of the claimed invention to have modified the gaze information of Kim to include vectors of eye rotation as disclosed by Liu at least because doing so would reduce the amount of data processing and improve the processing speed. Liu ¶241.
Regarding claims 2 and 15, with claim 2 as exemplary, Kim teaches wherein the outputting of the pose information represented in the camera coordinate system and the first gaze information represented in the facial coordinate system based on the feature information comprises: outputting the first gaze information based on the feature information and on a network that is trained to detect generalized first gaze information and correct the generalized first gaze information based on features of the eyes of the person (Kim page 9, fig. 11, full paragraphs nos. 2–12, CNN network for simultaneous head pose and gaze direction inference that includes gaze direction detection unit 502 that detects the gaze direction from extracted features from the normalized face image, where ¶8 teaches that the learner/CNN minimizes the loss between the detected inferred gaze and a true value gaze over iterations (correct the generalized first gaze information), and where page 3, ¶8 teaches the facial features to include the user’s eyes).
Regarding claims 5 and 18, Kim teaches further comprising: identifying a position of the face from the position information (Kim page 5, ¶9, and page 7, full ¶¶1–4, the face position detection unit detects the position of the face and a normalized center of face position is determined therefrom), and detecting the gaze target (Kim page 3, ¶6, and page 9, last paragraph, the object of gaze as gaze point 200 (gaze target), which is accurately acquired during the data acquisition process) along the second gaze information from the position of the face (Kim page 3, ¶12, and page 7, full ¶¶ 3–4, the gaze direction information inferred from the normalized image coordinate system is converted to the gaze direction in the input camera coordinate system by inversely transforming using the coordinate transformation relationship between the original input image and the normalized image about the center of the face).
Regarding claims 6 and 19, with claim 6 as exemplary, while Kim teaches specifically unit vectors as gaze vectors (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301) and gaze information as it relates toward the gaze target (Kim page 3, ¶6, and page 9, last paragraph, the object of gaze as gaze point 200 (gaze target)), Kim does not teach the remainder of the limitations of claim 6. Liu teaches further comprising: outputting, based on the feature information (Liu ¶160, gaze vectors determined from left and right eye features), third gaze information comprising a generalized third vector from the origin of the facial coordinate system (Liu ¶¶286–287, fig. 7, camera coordinate system adjusted to be parallel to the face coordinate system which has an origin of the center point of the face), in the camera coordinate system (Liu ¶¶120–124, additional gaze vectors determined for both the left eye and right eye in a camera coordinate system).
Therefore, taking the teachings of Kim and Liu together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the gaze information of Kim to include further third gaze information as disclosed by Liu at least because doing so would provide a more accurate gaze vector and thus gaze point. Liu ¶377.
Regarding claim 9, Kim teaches further comprising: acquiring data for training a network configured to correct generalized first gaze information based on features of the eyes of the person (Kim page 9, ¶¶2–8, and 12–13, learning system comprised of a CNN (network) that minimizes loss of the gaze direction (thus to correct gaze information) which is trained by constructing a training dataset including gaze point and head pose information, where page 3, ¶8 teaches the head pose being detected based on user’s eyes features).
Regarding claim 12, with deficiencies of Kim noted in square brackets [], Kim teaches an operating method of an electronic device, comprising: (Kim Abstract, gaze direction detection method of a gaze direction detection device):
acquiring an image comprising a person (Kim page 3, ¶6, using a camera, an image 100 of a scene with a user (person) standing in front of the camera is input (acquiring), where the image has the face of the user);
extracting a normalized image comprising a face of the person from the image (Kim page 3, ¶¶8–10, user’s face area is adjusted through a normalization process resulting in a normalized image);
extracting feature information about the face from the normalized image (Kim page 3, ¶¶8–11, key facial features such as the user’s eyes, nose and mouth are predicted, and then converted to the coordinate system of the normalized image);
outputting, based on the feature information, pose information represented in a camera coordinate system which is a coordinate system of a camera acquiring the image (Kim page 3, ¶8, the user’s head pose is detected based on the camera coordinate system using the key facial features) and first gaze information represented in a facial coordinate system which is a coordinate system of the face (Kim page 3, ¶¶11–12, the gaze direction information is inferred using the related information (including 104 – the key facial features) in the normalized image coordinate system (facial coordinate system which facial features are also found within, thus “a coordinate system of the face”));
outputting, based on the feature information, second gaze information represented in the camera coordinate system (Kim page 3, ¶12, the gaze direction information inferred from the normalized image coordinate system is converted to the gaze direction in the input camera coordinate system by inversely transforming the user's gaze direction (300), which is based on the key facial features) that comprises a generalized unit vector from an origin of the facial coordinate system of the person toward a gaze target represented in the camera coordinate system (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301 connecting the user’s gaze position 106 and the gaze point 201, where fig. 3, and page 3, ¶11, teach point 106 as an origin point of the normalized image coordinate system in which pose 105 is defined); and
[determining third gaze information comprising a third] unit vector (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301) [from the origin of the facial coordinate system] toward the gaze target (Kim page 3, ¶6, and page 9, last paragraph, the object of gaze as gaze point 200 (gaze target)) [in the camera coordinate system, based on the pose information, the first gaze information, and the second gaze information,]
wherein the pose information comprises: [rotation information] and position information about the face (Kim page 3, ¶¶7–8, fig. 2, pose information 103, comprised of vectors originating from user position point 104 on the user’s face),
wherein the first gaze information comprises: a first unit vector (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301) [represented by a rotation of eyes from the origin of the facial coordinate system] toward the gaze target, in the facial coordinate system (Kim page 3, ¶¶ 6–11, figs. 2 and 3, gaze direction in the normalized image coordinate system (facial coordinate system) is determined with respect to a user’s eyes looking at an object of the gaze at gaze point 200).
While Kim teaches on page 6, ¶5 that the head pose is inferred by detecting geometric transformation relationships (rotation, movement) between 3D position values, thus suggesting that pose information, in a broader sense, comprises those geometric relationships including rotation and 3D position values, nonetheless, Kim does not explicitly teach the pose information comprising rotation information. However, Liu teaches the pose information comprising rotation information (Liu ¶17, acquiring a rotation matrix and a displacement matrix as the head pose data).
Further, Kim does not explicitly teach where Liu teaches gaze information comprises: a vector represented by a rotation of eyes from the origin of the facial coordinate system (Liu ¶¶30–42, left and right eye rotation matrices (a matrix is comprised of multiple vectors) are acquired from (thus comprised within) the head pose data, determined from the origin of left and right-eye coordinate systems respectively (facial coordinate system)).
Still further, Kim does not explicitly teach where Liu teaches determining third gaze information comprising a third vector from the origin of the facial coordinate system (Liu ¶¶286–287, fig. 7, ¶¶313 and 318, gaze vectors determined from left and right eye features which are used to find a final gaze vector, and camera coordinate system adjusted to be parallel to the face coordinate system which has an origin of the center point of the face), in the camera coordinate system (Liu ¶¶120–124, additional gaze vectors determined for both the left eye and right eye in a camera coordinate system), based on the pose information, the first gaze information, and the second gaze information (Liu ¶¶314, 318 and 324, final gaze vector determined from the head-camera rotation matrix, which is determined from yaw and pitch angle of the head (pose information) and the gaze vector of the left and right eye).
Therefore, taking the teachings of Kim and Liu together as a whole, it would have been obvious to a person having ordinary skill in the art (herein “PHOSITA”) before the effective filing date of the claimed invention to have modified the gaze information of Kim to include the vectors of eye rotation, and determining additional (third) gaze informations as disclosed by Liu at least because doing so would reduce the amount of data processing and improve the processing speed. Liu ¶241.
Regarding claim 13, Kim teaches a non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the operating method of claim 1 (Kim page 11, third and fourth full paragraphs, embodiments of the disclosure implemented by a program stored in a non-transitory computer-readable medium as instructions that are executed on a device or a computer).
Claims 3–4 and 16–17 are rejected under 35 U.S.C. 103 as being unpatentable over Kim in view of Liu, as set forth above regarding the independent claims, further in view of Haro, US Patent No. US 11,694,419 B2 (herein “Haro”).
Regarding claims 3 and 16, Kim teaches wherein the first gaze information comprises: the first unit vector that (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301). Kim does not explicitly teach where Haro teaches is corrected based on features of the eyes of the person to be personalized for the person (Haro col. 14, l. 63 – col. 15, l. 17, eye gaze corrections made according to calibration specific to a unique user’s position and eye movements (personalized features of the eyes)).
Therefore, taking the teachings of Kim and Haro together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the gaze direction vector of Kim to include correction per calibration to a specific user’s eyes as disclosed by Haro at least because doing so would provide for robust and seamless eye gaze depictions for a multitude of users. See Haro col 16, ll. 26–30.
Regarding claims 4 and 17, Kim teaches wherein the second gaze information comprises: the second unit vector that (Kim page 4, 7th full paragraph, fig. 3, gaze direction defined as the unit direction vector of vector 301) is personalized for the person as features of the eyes of the person are reflected (Haro col. 14, l. 63 – col. 15, l. 17, eye gaze determined according to calibration specific to a unique user’s position and eye movements (personalized features of the eyes), where the calibration includes requesting the user to focus on a certain object and move their eyes with the object as the object moves (reflecting eye movements with respect to the target object)).
Therefore, taking the teachings of Kim and Haro together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have the gaze direction vector of Kim to include personalization per calibration to a specific user’s eyes as disclosed by Haro at least because doing so would provide for robust and seamless eye gaze depictions for a multitude of users. See Haro col 16, ll. 26–30.
Allowable Subject Matter
Claims 7 and 20 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Specifically, claims 7 and 20 recite determining a second gaze information (which per antecedent basis in the independent claims, the second gaze information is in the camera coordinate system) by selecting based on the rotation information (where rotation information also features antecedence in the independent claims requiring the rotation information to be part of the pose information), one from a set of two choices: 1) third gaze information (with antecedent basis provided in claims 6 and 19 from which claims 7 and 20 respectively depend) and 2) fourth gaze information comprising a personalized unit vector based on the pose information and the first gaze information, and determining the selected third or fourth gaze information to be the second gaze information. The closest cited prior art includes Kim and Liu as applied above to claim 6. However, none of Kim or Liu, or any of the other examiner cited art of record whether considered along or in an obvious combination, teach or suggest to a PHOSITA selecting, based on the rotation information, between two different gaze information, one being a generalized/standard/reference gaze, and another being a personalized gaze. The Linden et al., “Learning to Personalize in Appearance-Based Gaze Tracking,” arXiv:1807.00664v3 [cs.CV] Sep. 2, 2019, reference cited in the IDS filed March 3, 2026, teaches determining a personalized gaze, but does not teach selecting between this personalized gaze and a third generalized gaze, based on rotation information. Cited reference Wang et al., US Patent Application Publication No. US 2024/0221163 A1, teaches selecting a reference gaze but does not teach or suggest selecting between the reference gaze and a personalized gaze based on rotation information from pose information. Therefore, none of the cited art of record, whether considered alone or in an obvious combination to a PHOSITA teach or suggest the limitations of claims 7 and 20.
Claim 8 is also objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Specifically, claim 8 recites determining second gaze information by a weighted sum of the third gaze information and fourth gaze information, based on the rotation information, where the fourth gaze information comprises a personalized unit vector determined based on the pose information and the first gaze information. The closest cited prior art includes Kim and Liu as applied above to claim 6. However, none of Kim or Liu, or any of the other examiner cited art of record whether considered along or in an obvious combination, teach or suggest to a PHOSITA determining gaze information by a weighted sum of two different gaze information, one being a generalized/standard/reference gaze, and another being a personalized gaze. The Linden et al., “Learning to Personalize in Appearance-Based Gaze Tracking,” arXiv:1807.00664v3 [cs.CV] Sep. 2, 2019, reference cited in the IDS filed March 3, 2026, teaches determining a personalized gaze, but does not teach determining a weighted sum between this personalized gaze and a third generalized gaze, based on rotation information. Cited reference Wang et al., US Patent Application Publication No. US 2024/0221163 A1, teaches selecting a reference gaze but does not teach or suggest determining a weighted sum between the reference gaze and a personalized gaze based on rotation information from pose information. Therefore, none of the cited art of record, whether considered alone or in an obvious combination to a PHOSITA teach or suggest the limitations of claim 8.
Claim 10, and therefore claim 11 which depends therefrom, would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims. Specifically, claim 10 recites acquiring the third gaze information as the data (per antecedence to claim 9, this data is for training a network to correct generalized first gaze information) by generating a plurality of areas for a pupil rotation range of the person based on the features of the eyes, and acquiring the data preferentially by assigning a weight to an area from which data is not acquired among the plurality of areas. The closest cited prior art includes Kim and Liu as applied above to claim 9, including the details of Kim’s training of a CNN (network) to output an accurate gaze value. However, none of Kim or Liu, or any of the other examiner cited art of record whether considered alone or in an obvious combination, teach or suggest to a PHOSITA determining training data for a gaze correction by generating areas for a pupil rotation range, and assigning a weight to an area from which data is not acquired among the plurality of areas. Therefore, none of the cited art of record, whether considered alone or in an obvious combination to a PHOSITA teach or suggest the limitations of claim 10 and therefore claim 11 which depends from claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Linden et al., “Learning to Personalize in Appearance-Based Gaze Tracking,” arXiv:1807.00664v3 [cs.CV] Sep. 2, 2019, cited in the IDS filed March 3, 2026, is directed towards determining a personalized gaze vector.
Wang et al., US Patent Application Publication No. US 2024/0221163 A1, directed towards selecting a reference gaze for measuring a deviation angle of a gaze position.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights