DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment and written response filed 12/31/2025 have been entered and considered.
Claim 19 was cancelled.
Claim 20 was amended.
Claim 21 was added.
Claims 1-18 and 20-21 are pending.
Response to Arguments
Applicant's arguments filed 12/31/2025 have been fully considered but they are not persuasive.
The Brouns reference:
Applicant argues Brouns “teaches predicting a value for a particular camera frame using a predicted value for the previous camera frame. Brouns does not teach or suggest predicting a value for a particular camera frame using a sequence of predicted values.” And Brouns fails to disclose “determining the processing function of each image recursively over the sequence of images, on the basis of the sequence of estimates” as recited in claim 1.
Upon further review of the reference and in light of applicant’s arguments, the examiner respectfully disagrees as follows: as evidenced in Methods section and figure 1 in Brouns, at least three images are involved in the eye tracking process: previous frame; current frame and next frame. These three frames are in a sequence. Furthermore, the claimed “sequence of estimates of the processing functions of at least some of images” are clearly shown in figure 1 for the computed values of a sequence of processing tasks, from step 0: estimation of pupil characteristics from previous frame, step 1: crop to search area, step 2: Haar-like feature detection, … to step 10 new estimation of pupil characteristics to next frame. Additionally, these computed/estimated values of the processing steps are the claimed “sequence of estimates”, which are used to determine “estimation of pupil characteristics” of the image frame as an input to the current frame and output for the next frame in a recursive manner for Brouns’ eye tracking algorithm over a plurality of video frames. Therefore, it is clear that Brouns teaches the claim feature of “determining the processing function of each image recursively over the sequence of images, on the basis of the sequence of estimates”.
Claim Objections
Claim 21 is objected to because of the following informalities: a typographical error is found. “based of” in line 4 should be “basis of”. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1- 18, 20 and 21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Brouns (“Robust Video-Based Eye Tracking Using Recursive Estimation of Pupil Characteristics”, see IDS filed 11/02/2023).
As per claim 1, Brouns discloses a computer-implemented image processing method comprising a determination of an image processing function of each image of an image sequence (Brouns teaches an eye tracking method that includes a pupil detection algorithm for a sequence of image frames including a plurality of processing tasks as shown in figure 1) comprising the following steps: (i) determining a sequence of estimates of the processing functions of at least some of the images (see methods in page 2 and figure 1 for a plurality of processing tasks for detecting pupil in the image frames); and determining the processing function of each image recursively over the sequence of the images, on the basis of the sequence of estimates (see page 4 for 2.1 feature value prediction section that pupil characteristics are estimated in successive frames with a recursive estimator).
As per claim 2, Brouns discloses wherein the estimate of the processing function of a current image of the image sequence is determine in step (i) from: the current image; a neighboring image preceding or following the current image in the image sequence, and whose processing function has been determined beforehand; and the processing function of the neighboring image (see figure 1 for estimation of pupil characteristics from previous frame, current frame and next frame).
As per claim 3, Brouns discloses wherein step (i) comprises a comparison between the current image and the neighboring image (see section 2.1 Feature value prediction).
As per claim 4, Brouns discloses wherein the comparison comprises a determination of a vector field corresponding to a displacement of pixels between the current image and the neighboring image (see section 2.1 feature value prediction).
As per claim 5, Brouns discloses wherein the vector field is computed by optical flow (the computed vectors are continuously updated throughout frames).
As per claim 6, Brouns discloses wherein the estimates of the processing function of the current image is determined in step (i) by a composition of the vector field with the processing function of the neighboring image (as explained above and in section 2.1, the vectors are computed and continuously updated throughout frames during pupil tracking).
As per claim 7, Brouns discloses wherein the estimate of the processing function of the current image is algorithmically determined in step (i) by a Kalman filter (see section 2.1 for Kalman filter).
As per claim 8, Brouns discloses wherein an execution of steps (i) and (ii) beings a determination of the processing function of a first image of the image sequence from input data comprising this first image (the previous frame in figure 1 may be the claimed “first image” of the image sequence).
As per claim 9, Brouns discloses wherein the processing function of an image to be processed is at least second in the image sequence, and wherein the processing function estimate has been determined beforehand, and is determined in step (ii) from input data comprises: the image to be processed; the estimate of the processing function of the image to be processed; and the processing function of an image preceding the image to be processed in the image sequence (see figure 1).
As per claim 10, Brouns discloses wherein the input data further comprises: several of the images preceding the image to be processed in the image sequence and/or several of the processing functions of the images preceding the image to be processed in the image sequence (section 2.1 feature value predictions teaches estimation calculation from a plurality of previous images).
As per claim 11, Brouns discloses wherein the processing function of the image to be processed is determined algorithmically in step (ii) from input data (see section 2.1).
As per claim 12, Brouns discloses wherein step (ii) is carried out by a neural network which has been developed and trained prior to step (i) and (ii) in order to determine the processing function of the image to be processed at step (ii) on the basis of the input data (pager 17: last paragraph; page 27: last paragraph: Brouns teaches training a convolutional neural network based on 3600 pupils).
As per claim 13, Brouns discloses wherein the image processing function associates a model and/or a structure with a collection of pixels from the images (the segmented pupil region a collection of pixels from the images)
As per claim 14, Brouns discloses wherein the image processing function defines an image segmentation (see figure 1 for pupil segmentation)
As per claim 15, Brouns discloses an eye-tracking method comprising the following steps: (a) providing a sequence of images of an eye (abstract: eye tracking in “successive camera frames”); (b) segmenting the images at least in the vicinity of a representation of the iris of the eye (pages 2-3 and figure 1: crop image to a smaller search area); (c) determining the position of a limbus of the eye on the basis of the segmentations of the images of step (b) (figure 1: see processing tasks 2-7 for determining the position of a limbus through segmentations of the images); and determining a position of the eye on the basis of the position of the limbus of the eye determined at step (c) (see tasks 8-10 for eye tracking), wherein step (b) is implemented by a computer-implemented image processing method comprising a determination of an image processing function of each image of an image sequence comprising the following steps: determining a sequence of estimates of the processing functions of at least some of the images; and determining the processing function of each image recursively over the sequence of images, on the basis of the sequence of estimates, wherein the image processing function defines an image segmentation (as shown in figure 1, tasks 2-7 are the claimed “processing functions” for each image frame recursively for image segmentations).
As per claim 16, Brouns discloses wherein step © comprises determining a position characteristic of a pupil of the eye on the basis of the segmentation of the images of step (b) and the position of the eye is determined at step (b) on the basis of the position characteristics of the pupil of the eye and of the position of the limbus of the eye determined at step (c) (see figure 1: tasks 3-9 for determining pupil position and limbus position).
As per claim 17, Brouns discloses wherein step (ii) is carried out by a neural network which has been developed and trained prior to step (i) and (ii) in order to determine the processing function of the image to be processed at step (ii) on the basis of the input data; and prior to step (b), comprising: a method for training the neural network by back-propagating an error gradient at the level of the neural network on the basis of a sequence of test images of an eye looking at a target located at a predetermined position on a screen (pager 17: last paragraph; page 27: last paragraph: Brouns teaches training a convolutional neural network based on 3600 pupils and a convolutional neural network inherently includes a back-propagating loss function for calculating the error gradient for network optimization).
For claims 18 and 20, Brouns teaches a computer-like system, which inherently includes a non-transitory computer-readable medium storing a computer program.
As per claim 21, Brouns discloses wherein the sequence of images includes at least a first image and a second image, wherein the first image precedes the second image in the sequence of images, and wherein determining the processing function of each image recursively over the sequence of images, on the basis of the sequence of estimates further includes: determining a second processing function of the second image on the basis of: the sequence of estimates, and a first processing function of the first image (as explained above, the previous frame may be the claimed “first image” and “current frame” is the claimed “second image”, and the estimation of the pupil characteristics of each frame is calculated based on a sequence of computed values from steps 0-10, and the “estimation of the pupil characteristics”, which is used as an input for a current frame and output to a next frame, is recursively computed for each image frame for eye tracking).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TOM Y LU whose telephone number is (571)272-7393. The examiner can normally be reached Monday - Friday, 9AM - 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272 - 7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TOM Y LU/Primary Examiner, Art Unit 2667
Prosecution Insights