DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The IDS(s) has/have been considered and placed in the application file.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Status of Claims
Claims 1-11 and 13-21 are pending in this application and have been considered below. Claim 12 is cancelled. The preliminary amendment filed January 10, 2025, has been entered.
Claim Objections
Claim 4, 5, 9, 11, 14, 17-18, and 21 is/are objected to because of informalities. The examiner recommends the following changes.
Claim 4, 17, and 21 should be “a the light spot center” or change its dependence to claim 2, where the condition is introduced.
Claim 5, line(s) 2, should be “a inter-frame light spot moving distance” to correct the antecedent basis issue.
Claim 9, line(s) 6, should be “a light spot center constraint condition” or change its dependence to claim 8, where the condition is introduced.
Claim 11, line(s) 1, should be “a second target position” or change its dependence to claim 10, where the second target position is introduced.
Claim 14, line(s) 2, should be “a computer” to correct the antecedent basis issue.
Claim 18, line(s) 2, should be “a inter-frame light spot moving distance” to correct the antecedent basis issue.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim(s) 7 recite “a plurality of the first target positions;” whereas claim 1 defines “a first target position” It is unclear whether claim 7 requires a plurality of local detection areas, a plurality of positions within a single local detection area, or a different position from the single highest-brightness position defined in claim 1.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4-5, 8-9, 13-14, 17-18, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cohen, US 2020/0241635 A1 (hereinafter “COHEN”) in view of Stuart et al., US 2022/0301217 A1 (hereinafter “STUART”).
Claims 1, 13, and 14.
COHEN and STUART disclose an electronic device, comprising: at least one processor; and a memory storing a program; wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform a light spot tracking method (COHEN ¶ 50: “The local processing and data module 260 may comprise a hardware processor, as well as digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to assist in the processing, caching, and storage of data.”), and a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to perform a light spot tracking method (COHEN ¶ 73: “programming (e.g., instructions in a non-transitory computer-readable medium)”) …
COHEN and STUART further disclose a light spot tracking method, comprising:
extracting an eye image of a target frame and an eye image of a to-be-detected frame from an eye image sequence, wherein the eye image of the target frame contains a light spot, and the acquisition time of the eye image of the target frame is earlier than the acquisition time of the eye image of the to-be-detected frame (COHEN ¶ 154: “determination of glint positions at an earlier frame can assist determining the expected positions of the glints in subsequent frames.” COHEN ¶¶ 89-90 discloses extracting eye images with glints i.e. a light spot.);
acquiring a local detection area from the eye image of the to-be-detected frame based on the light spot (COHEN ¶ 154: “limit the frame-by-frame search region to a small number of pixels (e.g., 2 to 10 pixels) around the previous glint positions.”); and
determining a light spot recognition result of the eye image of the to-be-detected frame in response to a pixel brightness judgment result of a first target position located in the local detection area (COHEN ¶ 105: “As an example, the glint detection module 714 may search for bright regions within the eye tracking image, sometimes referred to herein as "blobs" or local intensity maxima that are in the vicinity of the user's pupil or iris.” COHEN ¶ 168.), wherein
COHEN and discloses all of the subject matter as described above except for specifically teaching “in the local detection area.” However, STUART in the same field of endeavor teaches in the local detection area (STUART claim 3: “search the second region of the second image for respective locations exhibiting local maxima intensity values in the second region.” ¶150: “glint detection and labeling module 1014 may limit its search for glints to such a region.”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine COHEN and STUART before the effective filing date of the claimed invention. The motivation for this combination of references would have been to search COHEN's small previous position region for local maxima (COHEN ¶ 154) instead of scanning the whole image for maxima (COHEN¶ 168), thus improving glint detection efficiency, because STUART teaches limiting the glint search to an identified region and locating local maxima intensity therein (STUART ¶ 150; claim 3). Applying a known regional maxima technique the same way in COHEN's known base device would predictably exclude irrelevant maxima. This is the use of a known technique to improve similar devices in the same way (MPEP 2143(C)).
Claims 4, 17, and 21.
COHEN and STUART disclose the method according to claim 1, wherein the eye image sequence is acquired by an image acquisition device, and the acquiring a local detection area from the eye image of the to-be-detected frame based on the light spot comprises: determining a size parameter of the local detection area based on an acquisition frame rate of the image acquisition device (COHEN¶ 154: “because the glints do not move appreciably between the frames when taken at a relatively high frame rate … limit the frame-by-frame search region to a small num her of pixels ( e.g., 2 to 10 pixels) around the previous glint positions.” This teaches setting the size of the local region from the acquisition frame rate. COHEN recites glint image frame rates in a range from 50 fps to 1000 fps (COHEN ¶6)); determining the local detection area based on the light spot center of the eye image of the target frame and the size parameter of the local detection area (COHEN ¶ 129: “the location of the center of the glint can be determined to subpixel accuracy”; ¶ 154: “only a portion of the glint image (e.g., a 5x5 group of pixels) may be stored in a temporary memory buffer 615 for processing by an associated CPU 612.” This teaches a sized region taken about the prior light spot center.).
Claims 5 and 18.
COHEN and STUART disclose the method according to claim 4, wherein the size of the local detection area is positively correlated with the inter-frame light spot moving distance (COHEN ¶ 154. Because a higher frame rate yields smaller inter-frame glint movement and permits a smaller search region, the region size varies with the inter-frame movement distance.).
Claim 8.
COHEN and STUART disclose the method according to claim 1, wherein the determining a light spot recognition result of the eye image of the to-be-detected frame in response to a pixel brightness judgment result of a first target position located in the local detection area comprises: if a pixel brightness of the first target position located in the local detection area satisfies a light spot center constraint condition, determining that the light spot recognition result includes: that the local detection area and a to-be-detected light spot have an intersection (COHEN ¶ 158: “In the case of no occlusion, each glint may have approximately the same intensity from frame to frame.” This teaches that a light spot whose brightness satisfies the condition is the tracked light spot found within the region.), wherein the to-be-detected light spot is a light spot of the eye image of the target frame that is tracked in the eye image of the to-be-detected frame (COHEN ¶ 169: “identified glints (e.g., position, intensity, etc.) can be received from block 1716, and the block 1714 can use this information to identify a search region in the glint image”; Fig. 17); if the pixel brightness of the first target position located in the local detection area does not satisfy the light spot center constraint condition, determining that the light spot recognition result includes: that the light spot of the eye image of the target frame disappears in the eye image of the to-be-detected frame (COHEN ¶ 158: “if there is at least partial occlusion, the intensity of the glint will rapidly decrease … remove that glint from the eye-tracking analysis.” This teaches the failing branch, in which the tracked light spot is treated as absent from the later frame.).
Claim 9.
COHEN and STUART disclose the method according to claim 1, wherein the number of light spots of the eye image of the target frame is multiple, and the method further comprises: storing a plurality of light spot center coordinates of the eye image of the target frame in a light spot sequence array (COHEN ¶ 155: “the "constellation" of four glints depicted in FIG. 12 may tend to move at a substantially common velocity from frame to frame as the eye moves.” This teaches maintaining the coordinates of multiple tracked light spots as a labeled set.); if the first target position located in the local detection area satisfies the light spot center constraint condition, updating the light spot center coordinates corresponding to the local detection area in the light spot sequence array using the coordinates of the first target position (COHEN ¶ 154: “Glints that are present in the image portion may be quickly and efficiently identified and their positions determined.” This teaches writing the newly determined position into the maintained set.); if the first target position located in the local detection area does not satisfy the light spot center constraint condition, occupying a position in the light spot sequence array corresponding to the local detection area using an empty array (COHEN ¶ 158: the system will "remove that glint from the eye-tracking analysis". This teaches vacating the entry for a light spot that is no longer recognized.).
Claim(s) 2-3, 15-16, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over COHEN in view of STUART, and further in view of Ivarsson et al. (US 2018/0270436 A1, hereinafter “IVARSSON”).
Claims 2, 15, and 19.
COHEN and STUART disclose the method according to claim 1, wherein a geometric center of the local detection area coincides with a light spot center (COHEN ¶ 154: “limit the frame-by-frame search region to a small number of pixels ( e.g., 2 to 10 pixels) around the previous glint positions … a portion of the glint image (e.g., a 5x5 group of pixels)”).
COHEN and STUART do not explicitly teach aligning the center of the region with the center of the light spot. IVARSSON teaches this (IVARSSON ¶ 111: this positioning includes aligning the center of the ROI with the center of the pupil … this positioning includes aligning the center of the ROI with the center of the pupil”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine COHEN, STUART, and IVARSSON before the effective filing date of the claimed invention. The motivation for this combination of references would have been because a region centered on the light spot keeps the tracked light spot within a region as it moves between frames. This is use of a known technique to improve similar devices in the same way (MPEP 2143(C)) since IVARSSON supplies the known technique of aligning the region’s center with the glint’s center therefore the references position a bounded region of an infrared eye image on a located clint, so the result would have been predictable.
Claims 3, 16, and 20.
COHEN, STUART, and IVARSSON disclose the method according to claim 2, wherein the light spot center is a position with the highest brightness in the light spot (COHEN ¶ 168: “non-maxima in the glint image are suppressed or removed, which can assist in finding just the glint peaks.”); or, the light spot center is a geometric center of the light spot, and the size of the local detection area is larger than the size of the light spot (COHEN ¶ 129: “the location of the center of the glint can be determined to subpixel accuracy”; ¶ 125 “The full width half maximum (FWHM) of the glint 1006a is about 15 pixels.”).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over COHEN in view of STUART, and further in view of Yan et al. (CN 110930351 A, hereinafter “YAN”).
Claim 6.
COHEN and STUART disclose the method according to claim 1, further comprising: detecting the local detection area using a (COHEN ¶ 129: “the location of the center of mass of the glint can be determined and relied upon in a capacity similar to that of the location of the center of the glint.”), and obtaining a pixel brightness of the first target position (¶ 154: “remove that glint from the eye-tracking analysis.” ¶158: “remove that glint from the eye-tracking analysis.”).
COHEN and STUART do not explicitly teach a grayscale centroid algorithm. YAN teaches this (YAN teaches gray center of gravity method i.e. grayscale centroid algorithm).
Therefore, it would have been obvious to one of ordinary skill in the art to combine COHEN, STUART, and YAN before the effective filing date of the claimed invention. The motivation for this combination of references would have been to “realize the high-precision, fast, and autonomous detection and center positioning of the light spot in dynamic video” (Yan, p. 1 Background.) Applying a known centering method to a known detector (MPEP 2143(C)).
Claim(s) 10 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over COHEN in view of STUART, and further in view of lshii et al. (US 2018/0350070 A1, hereinafter "ISHII").
Claim 10.
COHEN and STUART disclose the method according to claim 8, wherein the light spot center constraint condition includes: (ISHII ¶ 55: “The bright-spot detector 22 detects pixels that are included in the eye area and whose brightness values are, larger than or equal to a predetermined threshold. The predetermined threshold may be, for example, 80% to 90% of the largest value of the brightness values in the eye area.”); wherein the (ISHII claim 1: “determining ... a degree of overlapping between the one of the bright spot areas and the pupil in accordance with brightness on a circumference of the one of the bright spot areas.” This teaches taking the reference brightness from positions at the edge of the region.).
It would have been obvious before the effective filing date of the claimed invention to reference COHEN's glint-intensity monitoring to edge pixels of the glint region rather than to an absolute value (COHEN¶ 158), because ISHII teaches that bright spots are substantially saturated and assessed by brightness on a circumference of the spot area (ISHII ¶ 54; claim 1). Applying ISHII's known circumferential-brightness technique the same way to COHEN's known base device would predictably improve occlusion detection. This is the use of a known technique to improve similar devices in the same way (MPEP 2143(C)).
Claim 11.
COHEN, STUART, and ISHII disclose the method according to claim 8, wherein a distance between the second target position and the first target position satisfies: d<d'<dmax, where d represents a distance between the first target position and a contour position of the local detection area, d' represents a distance between the first target position and the second target position, and dmax represents a maximum distance between the first target position and the second target position (ISHII ¶ 65: “the search-line setter 24 calculates a distance between the pixel located on the search line of interest and having the highest brightness and the barycenter of the bright spot area, and when the distance is smaller than a predetermined threshold, the search-line setter 24 may determine that the search line of interest overlaps the bright spot area.” ISHII sets that threshold as “a value obtained by adding a predetermined offset value (for example, one to three pixels) to the spread distance of the bright spot area,” which bounds the separation between the two positions as recited.).
Allowable Subject Matter
Claim(s) 7 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten to overcome the rejection under 35 U.S.C. 112(b) and rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights