DETAILED ACTION
Response to Arguments
Applicant's arguments filed on 07/23/20266 have been fully considered but they are not persuasive.
Arguments are not commensurate in scope with the claims. Applicant argues that Mathôt fails to disclose an interaction element “wherein the interaction element comprises pixel information for a plurality of pixels” (Remarks, p. 8). That language is not recited in claim 1, 15 or 20. Claim 4 “(Orginal)” and remains dependent from claim 1. The subject matter indicated as allowable was not incorporated into any independent claim. Arguments direction to limitations not recited in the claims are not persuasive.
Applicant’s “transient phase assignment” argument reads limitations into the claims. The claims do not require the first and second regions to be spatially fixed, predetermined, or constant in illumination state. They require only that each region have different illumination characteristics. Mathôt maintains the two groups in opposite brightness states at all times [Mathôt: p.12: “Selection Algorithm”: “Letters are divided into two groups: bright and dark backgrounds”].
Applicant’s characterization of Mathôt is contrary to the reference. Applicant asserts Mathôt’s items “all share the same oscillating brightness pattern.” The groups oscillating in antiphase [Mathôt: p.12: “Selection Algorithm”: “For the letter group that has changed from bright to dark, L is multiplied by PPSD”; “For the letter group that has changed from dark to bright, L is divided by PPSD”].
The added limitations is expressly disclosed. Mathôt evaluates the direction of the pupillary response against the luminance state of the region under consideration [Mathôt: p.3: “Results”: “If pupil size decreased, the participant likely intended to select a letter that changed from darkness to brightness”], and applies opposite operations to the two regions based on the luminance of each [Mathôt: p.12: “Selection Algorithm”: “because we expect the pupil to constrict if the target is part that group”]. This is the recited determination “based on different illumination characteristics” and “based on luminance in the first region.”
Duration of the determination is not claimed. Mathôt determines the interaction event while the interaction element is presented [Mathôt: p.12: “Selection Algorithm”: “The analysis is performed on-line, while the participant performs the task”].
Claims 15 and 20. No separate argument is presented. The rejection is maintained.
Claims 6, 7, 9, 11, and 13-14. No deficiency in Mathôt has been shown. Applicant identifies no element of any dependent claim and addressed no relied-upon portion of Crispin or Palti-Wasserman. A general allegation that the claim defines a patentable invention, without specifically pointing out how the claim language distinguishes over the references, does not comply with 37 C.F.R. 1.111(b).
Double patenting. A request to hold the rejection in abeyance is not a reply on the merits. The rejection is maintained. Applicant is advised that the limitation added to claims 1, 15, and 20 corresponds to the step recited in claims 1, 15, and 20 of US 12,411,598 B2, and the amendment therefore brings the claims nearer to the reference patent claims. A timely filed terminal disclaimer under 37 CRG 1.321(c) may be used to overcome the rejection.
Allowable subject matter. Claims 4, 5, 18, and 19 have not been rewritten in independent form. The indication of allowable subject matter is maintained.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 8, 10, 12, 15-17, and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by [Mathôt S, Melmi J-B, van der Linden L, Van der Stigchel S (2016) The Mind-Writing Pupil: A Human-Computer Interface Based on Decoding of Covert Attention through Pupillometry. PLoS ONE 11 (2): e0148805. doi:10.1371/journal.pone.0148805, published February 5, 2016.] (hereinafter “Mathôt”).
Regarding claim 1:
Mathôt discloses
1. A method comprising: at a device [Mathôt : page 11: ¶ “Materials and Availability”: “using our HCI with supported systems (currently tested with EyeLink and EyeTribe eye trackers,
and Windows and Linux operating systems)”; page 11: ¶ “Software and Apparatus”: “Stimuli were presented on a 21" ViewSonic p227f CRT monitor (1280 x 1024 px, 85 Hz) running Ubuntu Linux 14.04.” Examiner: Under the broadest reasonable interpretation (BRI); the supported system which runs Windows or Linux OS inherently requires a computer to enable it. The combination of eye tracker (EyeLink 1000), display (ViewSonic p227f CRT monitor), the software (e.g., Ubuntu Linux or Windows OS) and computer (e.g., PC or Mac Book or the like) constitutes “a device”.] comprising a processor [Mathôt : page 11: ¶ “Materials and Availability”: “using our HCI with supported systems (currently tested with EyeLink and EyeTribe eye trackers, and Windows and Linux operating systems)”; Examiner: Under the BRI, the computer running the OS inherently comprises a processor. All general-purpose computers such as Mac Books or PCs have at least a processor and a memory.]:
obtaining physiological data [Mathôt: page 11: ¶ “Software and Apparatus”: “Eye position and pupil size were recorded monocularly with an EyeLink 1000 … , a video-based eye tracker sampling at 1000 Hz.”] associated with a pupil [Mathôt: page 11: ¶ “Software and Apparatus”: “ Eye position and pupil size were recorded … The right eye was recorded, unless the left eye provided a better signal”: “pupil size”] during presentation of an interaction element [Mathôt: page 4, Fig.1; page 11: ¶ “General Stimuli and Procedure”: “Items were presented in a circular configuration at an eccentricity of 9.2° (Fig 1). Items consisted of colored letters against a circular background (r = 3.1°).”; Examiner: Under BRI, the entire colored letters and corresponding circular background that are displayed simultaneously constitutes the “interaction element.” The claim does not limit “interaction element” to a single graphical icon; Figs.4A-C of the present application show only non-limiting examples.], the interaction element comprising a first region [Mathôt: page 4: Fig.1; page 12: ¶ “Selection Algorithm”: “Letters are divided into two groups: bright and dark backgrounds”; Examiner: Under BRI, the first group or the second group, or vice versa, constitutes “a first region.”] and a second region [Mathôt: page 4: Fig.1; page 12: ¶ “Selection Algorithm”: “Letters are divided into two groups: bright and dark backgrounds”; Examiner: Under BRI, the first group or the second group, or vice versa, constitutes “a second region.”], each region having different illumination characteristics [Mathôt: page 4: Fig.1; page 12: ¶ “Selection Algorithm”: “Letters are divided into two groups: bright and dark backgrounds”; page 11: ¶ “General Stimuli and Procedure”: “When only two items were presented, each item was accompanied by a mirror-symmetric placeholder (see Fig 1a; this configuration was chosen because pilot experiments showed it to be the most effective of several tested configurations; see S1 Appendix). The backgrounds alternated between brightness (97.0 cd/m2) and darkness (5.1 cd/m2) in cycles of 1.25 s (0.8 Hz). Each cycle consisted of a smooth brightness transition of 0.5 s, followed by 0.75 s of constant brightness (Fig 1b).”];
determining, based on the obtained physiological data [Mathôt: page 12: ¶ “After each cycle, a proportional pupil-size difference (PPSD) is determined (see Pupil-size measurement)”], a pupillary response during the presentation of the interaction element [Mathôt: page 3: ¶ “We measured
median pupil size during the last 0.25 s of each cycle, and used the following logic to determine
which letter the participant intended to select”];
determining, based on the different illumination characteristics [Mathôt: p.12: “Selection Alogrithm:” “Letters are divided into two groups: bright and dark backgrounds”; Examiner: The bright/dark assignments of the two regions is the input that which operation is applied.], whether the pupillary response corresponds to directing attention to the first region [Mathôt: p.3: “Results”: If pupil size decreased, the participant likely intended to select a letter that changed from darkness to brightness (‘b’ in Fig 1b); if pupil size increased, the participant likely intended to select a letter that changed from brightness to dark ness (‘a’ in Fig 1b)”] based on luminance in the first region [Mathôt: [p.12: “Selection Alogrith”: “For the letter group that has changed from bright to dark, L is multi plied by PPSD (because we expect the pupil to dilate if the target is part that group). For the let ter group that has changed from dark to bright, L is divided by PPSD (because we expect the pupil to constrict if the target is part that group)”; Examiner: The operation applied to a given region, multiplication versus division is selected according to the luminance of that region, and the resulting likelihood L is that region’s measure of whether the pupillary response corresponds to attention directed to it.];
and determining an interaction event [Mathôt: page 12, ¶ “Selection Algorithm: “Cycling continues until the proportional difference between the Ls for both groups exceeds a threshold T (L1/L2 > T or L1/L2 < 1/T), after which the group with the highest L is designated as the winner”; Examiner: The “interaction event” reads on Mathôt’s selection event under BRI; the claim does not require any particular form of selection or downstream action.] during the presentation of the interaction element [Mathôt: page 3: ¶ “Phases 1–3: Selecting a Predefined Stimulus”; “In the first part of the experiment, participants learned to select one of two (Phase 1), four (Phase 2), or eight (Phase 3) letters (see Fig 1). Letters were presented within circles that oscillated between brightness and darkness in cycles of 1.25 s. Participants selected a letter by covertly attending to it, while keeping the eyes on the central fixation dot”] based on determining that the pupillary response [Mathôt: page 3: ¶ “Phases 1–3: Selecting a Predefined Stimulus”: “Pupillary responses. Fig 2a shows the average pupil size during a cycle, as a function of whether the attended stimulus changed from bright to dark (blue line) or dark to bright (orange line); this is based on the average of all cycles (N = 112) for a single participant during Phase 1.”] corresponds to directing attention to the first region during the presentation of the interaction element [page 9: ¶ “Discussion”: “We have introduced a new human-computer interface (HCI) that is based on decoding of covert attention through pupillometry. Participants select a letter by covertly attending to it, without making any overt (eye) movement. Letters are presented within circles of oscillating brightness. Small changes in pupil size reflect the brightness changes of the attended stimulus [23], and this allows us to determine which stimulus the participant intends to select–in real time, independent of movement (other than pupil-size changes), and without physical contact.”; Examiner: The “first region” reads on whichever of the two bright/dark groups contains the user’s attended target on a given cycle; the claim does not require the first region to be predetermined or fixed.].
Regarding claim 2:
Mathôt discloses:
2. The method of claim 1,
wherein the different illumination characteristics of each region of the interaction element [Mathôt: page 12: ¶ “Selection Algorithm”: “Letters are divided into two groups: bright and dark backgrounds”] comprises one or more dark regions and one or more bright regions [Mathôt: page 11: ¶ “General Stimuli and Procedure”: “The backgrounds alternated between brightness (97.0 cd/m2) and darkness (5.1 cd/m2) in cycles of 1.25 s (0.8 Hz). Each cycle consisted of a smooth brightness transition of 0.5 s, followed by 0.75 s of constant brightness (Fig 1b).”; Examiner: The darkness state corresponds to the “one or more dark regions” and the brightness state corresponds to the “one or more bright regions”.].
Regarding claim 3:
Mathôt discloses:
3. The method of claim 1,
wherein each region of the interaction element comprises a level of luminance [Mathôt: page 11: ¶ “General Stimuli and Procedure”: “The backgrounds alternated between brightness (97.0 cd/m2) and darkness (5.1 cd/m2) in cycles of 1.25 s (0.8 Hz).”; Examiner: Each of the two regions Mathôt’s interaction element is presented at a defined cd/m2 value which constitutes “a level of luminance” under BRI] and the different illumination characteristics of each region [Mathôt: page 12: ¶ “Selection Algorithm”: “Letters are divided into two groups: bright and dark backgrounds.”] are based on the level of luminance of each region with respect to an illuminance threshold level [Mathôt: page 11: ¶ “General Stimuli and Procedure”: “The backgrounds alternated between brightness
(97.0 cd/m2) and darkness (5.1 cd/m2) in cycles of 1.25 s (0.8 Hz). Each cycle consisted of a
smooth brightness transition of 0.5 s, followed by 0.75 s of constant brightness (Fig 1b).”; Examiner: Mathôt’s binary classification of the two regions as “bright” and dark” inherently establishes an illuminance threshold level separating the two classifications. Specifically, any luminance value between 5.1 cd/m2 (dark state) ad 97.0 cd/m2 (bright state) constitutes an “illuminance threshold level” under BRI, with 97.0 cd/m2 being above the threshold and thus characterized as “bright”, and the 5.1 cd/m2 region being below the threshold and thus characterized as “dark”. The claim does not require that the thresholds be explicitly recited as a numerical value or that it be applied dynamically; it requires only that the different illumination characteristics be “based on” the level of luminance with respect to such threshold, which is the case in Mathôt’s binary bright/dark classification.].
Regarding claim 8:
Mathôt discloses:
8. The method of claim 1,
wherein the pupillary response is: a direction of the pupillary response [Mathôt: page 3: ¶ “Results Phases 1-3: Selectin a Predefined Stimulus”: “If pupil size decreased, the participant likely intended to select a letter that changed from darkness to brightness (‘b’ in Fig 1b); if pupil size increased, the participant likely intended to select a letter that changed from brightness to dark ness (‘a’ in Fig 1b)”; page 12: ¶ “Selection Algorithm”: “For the letter group that has changed from bright to dark, L is multi plied by PPSD (because we expect the pupil to dilate if the target is part that group). For the letter group that has changed from dark to bright, L is divided by PPSD (because we expect the pupil to constrict if the target is part that group)”; Examiner: Mathôt’s selection algorithm operates on the directional sign of the pupillary response, constriction (decrease) vs dilation (increase), which constitutes “a direction of the pupillary response” as claimed.];
a velocity of the pupillary response [Examiner: Not mapped because claim is in the alternative.];
or pupillary fixations [Examiner: Not mapped because claim is in the alternative.].
Regarding claim 10:
Mathôt discloses:
10. The method of claim 1, wherein the physiological data comprises an image of an eye [Mathôt: page 11: ¶ “Software and Apparatus”: “Eye position and pupil size were recorded monocularly with an EyeLink 1000 (SR Research, Mississauga, ON, Canada), a video-based eye tracker sampling at 1000 Hz”; Examiner: A video-based eye tracker inherently captures images of the eye to derive eye position and pupil size.] or electrooculography (EOG) data [Examiner: Not mapped because claim is in the alternative.].
Regarding claim 12:
Mathôt discloses:
12. The method of claim 1,
wherein determining the pupillary response during the presentation of the interaction element is based on determining a variability of the pupillary response [Mathôt: page 12: ¶ “Selection Algorithm“: “After each cycle, a proportional pupil-size difference (PPSD) is determined (see Pupil-size measurement).”; page 12: ¶ “Pupil Measurement”: “The proportional pupil-size difference on cycle i (PPSD(i)) is defined as PPSD(i) = … Here, PS(i) is the median pupil size during the last 250 ms of cycle i (see Fig 1b)”; Examiner: Under BRI, “a variability of the pupillary response” reads on Mathôt’s PPSD, which expressly captures the cycle-to-cycle change in pupil size, i.e., the variability of the pupillary response from one cycle to the next.] to a threshold [Mathôt: page 12: ¶ “Selection Algorithm”: “Cycling continues until the proportional difference between the Ls for both groups exceeds a threshold T (L1/L2 > T or L1/L2 < 1/T), after which the group with the highest L is designated as the winner.”; Examiner: Mathôt’s threshold T is applied to the likelihood ratio L1/L2, which itself derived from and updated PPSD variability of the pupillary response on each cycle. The pupillary-response determination is therefore “based on determining a variability of the pupillary response” as recited.].
Regarding claim 15:
The limitations of claim 15 have been addressed in the discussion of claim 1 above.
Regarding claim 16:
The limitations of claim 16 have been addressed in the discussion of claim 2 above.
Regarding claim 17:
The limitations of claim 17 have been addressed in the discussion of claim 3 above.
Regarding claim 20:
The limitations of claim 20 have been addressed in the discussion of claim 1 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 6-7, 11, and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over [Mathôt S, Melmi J-B, van der Linden L, Van der Stigchel S (2016) The Mind-Writing Pupil: A Human-Computer Interface Based on Decoding of Covert Attention through Pupillometry. PLoS ONE 11 (2): e0148805. doi:10.1371/journal.pone.0148805, published February 5, 2016.] (hereinafter “Mathôt”) in view of [Crispin Sterling et al., WO 2020159784 A1] (hereinafter Crispin) .
Regarding claim 6:
Mathôt discloses:
6. The method of claim 1.
However, Mathôt does not expressly discloses:
wherein the interaction event is classified using a machine learning technique based on the pupillary response and the different illumination characteristics of each region.
Crispin discloses:
wherein the interaction event is classified using a machine learning technique based on the pupillary response and the different illumination characteristics of each region [Crispin: ¶ 0008: “determining the intention of the user includes applying a machine learning technique trained to identify patterns in physiological data corresponding to user intentions or user expectations”; ¶ 0063: “a machine learning model (e.g., a trained neural network) is applied to identify patterns in physiological data, including identification of physiological responses to a visual characteristic 30 associated with an object 20”; ¶ 0064: “the device 10 may learn patterns specific to the particular user 25. For example, the device 10 may learn from determining that peak pattern 810 represents an indication of interest or intent of the user 25 in response to a particular visual characteristic 30 and use this information to subsequently identify the similar peak pattern 820 as another indication of interest or intent of the user 25”; Examiner: Crispin expressly teaches].
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Mathôt’s selection algorithm to classify the interaction event using machine language technique as taught by Crispin, because Crispin teach that an ML classifier improves pupillary-response-based interaction system by adapting to individual user variability, and substituting an ML classifier for Mathôt’s fixed-parameter likelihood update is known substitution yielding the predictable results of a more robust interaction-even classifier.
Regarding claim 7:
Mathôt discloses:
7. The method of claim 1.
However, Mathôt does not expressly disclose:
further comprising: adjusting content in response to determining the interaction event.
Crispin discloses:
further comprising: adjusting content in response to determining the interaction event [Crispin: ¶ 0005: “Based on the determined intention of the user to interact with the object, the device may initiate interaction with (e.g., selection of) the object”; ¶ 0047: “As shown in the example of Fig. 5, a first series of trials may present a user with a blue button that changes to pink after the user clicks the button. Prior to clicking the button in Trial 1, the user may present a baseline physiological characteristic (e.g., naive pupil diameter of 2 mm). As the user performs additional trials, the user may develop an expectation of the interface feedback condition and, upon presentation of the object (e.g., the button), the user may present a physiological response that reflects the expectation of the interface feedback condition (e.g., an excited or expected pupil diameter). For example, in Trial N, the user may present an excited pupil diameter of 4 mm prior to interacting with the button due to the expectation of the user that the button will change pink after clicking it.”; ¶ 0052: “the device may automatically perform the interaction with object in response to the identified intent or expectation of the user. As another example, if a user has been conditioned to expect that a dial button on his phone turns from gray to green after being clicked, and the phone has identified that the user exhibits an excited pupillary diameter of 3 mm in expectation of the dial button turning green (e.g., prior to selecting the dial button), then the phone may use a detection of the 3 mm excited pupillary diameter to make the call and display the characteristic of the dial button turning green (e.g., without the user physically clicking the dial button)”].
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Mathôt’s pupillary-response selection method to adjust on-screen content in response to the determined interaction event as taught by Crispin, because Crispin teaches that triggering a content-state change in response to a pupil-based interaction event is the standard means by which such systems provide feedback and complete the user’s intended interaction, and combining Mathôt’s selection mechanism with Crispin’s content-adjustment response yields the predictable result of a complete pupil-driven HCI loop.
Regarding claim 11:
Mathôt discloses:
11. The method of claim 1.
However, Mathôt does not expressly disclose:
wherein the physiological data comprises head movements.
Crispin discloses:
wherein the physiological data comprises head movements [Crispin: ¶ 0070: “In some implementations, the location and features of the head 27 of the user 25 (e.g., an edge of the eye, a nose or a nostril) are extracted by the device 10”; ¶ 0071: “By tracking the eyes 45, some implementations reduce the need to re-calibrate the user 25 after the user 25 moves their head 27. In some implementations, the device 10 uses depth information to track the pupil's 50 movement, thereby enabling a reliable present pupil diameter 55 to be calculated based on a single calibration of user 25. Utilizing techniques such as pupil-center-comeal reflection (PCCR), pupil tracking, and pupil shape, the device 10 may calculate the pupil diameter 55, as well as a gaze angle of the eye 45 from a fixed point of the head 27, and use the location information of the head 27 in order to re-calculate the gaze angle. In addition to reduced recalibrations, further benefits of tracking the head 27 may include reducing the number of light projecting sources and reducing the number of cameras used to track the eye 45”].
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Mathôt’s pupillary-response selection method to additionally obtain head-movement data as taught by Crispin, because Crispin teaches that combining head-position data with gaze and pupil signals improves identification to the on-screen object the user is attending to (Crispin ¶ [0070]), and adding IMU-derived head-movement data is a routing sensor-fusion combination yielding the predictable result of an interaction-event determination informed by both pupil and head-movement data.
Regarding claim 13:
Mathôt discloses:
3. The method of claim 1.
However, Mathôt does not expressly disclose:
wherein the device is a head-mounted device (HMD).
Crispin discloses:
wherein the device is a head-mounted device (HMD) [Crispin: Fig.10; ¶ 0056: “In some implementations, as illustrated in Figure 1, the device 10 is a handheld electronic device (e.g., a smartphone or a tablet). In some implementations the device 10 is a laptop computer or a desktop computer. In some implementations, the device 10 has a touchpad and, in some implementations, the device 10 has a touch-sensitive display (also known as a“touch screen” or“touch screen display”). In some implementations, the device 10 is a wearable head mounted display (HMD)”; ¶ 0034 and 0095].
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to implement Mathôt’s pupillary-response selection method to an head-mounted device as taught by Crispin, because Crispin expressly identifies HMDs as a suitable platform for pupillary-response-based interaction systems and intergrates the same eye-tracking hardware (light source, camera, controller) that Mathôt uses, yielding the predictable result of porting Mathôt’s selection algorithm onto a wearable HMD platform with no change to underlying selection mechanism.
Regarding claim 14:
Mathôt discloses:
14. The method of claim 1.
However, Mathôt does not expressly disclose:
wherein the presentation of the interaction element is an extended reality (XR) experience.
Crispin discloses:
wherein the presentation of the interaction element is an extended reality (XR) experience [Crispin: ¶ 0086: “In some implementations, the one or more displays 912 are capable of presenting MR content, including VR or AR content”].
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to present Mathôt’s interaction element as part of an XR experience as taught by Crispin, because Crispin expressly identifies XR (including AR, VR and MR) as a suitable presentation context for pupillary-response-based interaction systems (Crispin ¶ [0086]) and the same selection mechanism operates identically on 2D and XR-rendered selectable items, yielding the predictable result of Mathôt’s selection algorithm operating in an XR context.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over [Mathôt S, Melmi J-B, van der Linden L, Van der Stigchel S (2016) The Mind-Writing Pupil: A Human-Computer Interface Based on Decoding of Covert Attention through Pupillometry. PLoS ONE 11 (2): e0148805. doi:10.1371/journal.pone.0148805, published February 5, 2016.] (hereinafter “Mathôt”) in view of [Palti-Wasserman; Daphna, US 20150355815 A1].
Regarding claim 9:
Mathôt discloses:
9. The method of claim 1.
However, Mathôt does not expressly disclose:
wherein the pupillary response is derived from a saccade characteristic.
Palti-Wasserman discloses:
wherein the pupillary response is derived from a saccade characteristic [Palti-Wasserman: ¶ 0016: “According to some embodiments, the eye-response may include eye-movement trajectory, eye-movement direction, eye-movement frequency, pupil activity, pupil size, blinking speed, blinking frequency, gaze, saccades, drift, fixation, nystagmus, convergence, rolling, smooth pursuit, or any combination thereof”].
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have included the concept above of Palti-Wasserman in the invention of Mathôt to yield the predictable result of providing an alternative or additional data in the determination of the pupillary response.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is AUTO-PROCESSED and APPROVED IMMEDIATELY upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. US 12/411,598 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because as shown in the comparison table below it is clear that all the elements of the application claim are to be found in patent claim (as the application claim fully encompasses patent claim). The difference between the application claim and the patent claim lies in the fact that the patent claim includes many more elements and is thus much more specific. Thus the invention of the patent claim is in effect a “species” of the “generic” invention of the application claim. It has been held that the generic invention is “anticipated” by the “species”. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993). Since application claim is anticipated by the patent claim, it is not patentably distinct from the patent claim.
Appl. No.: 19/292,228
Pat. No.: US 12/411,598 B2
1. A method comprising: at a device comprising a processor:
obtaining physiological data associated with a pupil during presentation of an interaction element, the interaction element comprising a first region and a second region, each region having different illumination characteristics;
determining, based on the obtained physiological data, a pupillary response during the presentation of the interaction element;
determining, based on the different illumination characteristics, whether the pupillary response corresponds to directing attention to the first region based on luminance in the first region;
and determining an interaction event during the presentation of the interaction element based on determining that the pupillary response corresponds to directing attention to the first region during the presentation of the interaction element.
2. The method of claim 1, wherein the different illumination characteristics of each region of the interaction element comprises one or more dark regions and one or more bright regions.
3. The method of claim 1, wherein each region of the interaction element comprises a level of luminance and the different illumination characteristics of each region are based on the level of luminance of each region with respect to an illuminance threshold level.
4. The method of claim 1, wherein the presentation of the interaction element comprises pixel information for a plurality of pixels and determining that the pupillary response corresponds to directing attention to the first region of the interaction element comprises: determining an estimated perceived luminance for each pixel in the first region based on the pixel information.
5. The method of claim 4, wherein determining an interaction event comprises: determining scene-induced pupil response variation characteristics for the regions of the interaction element; and determining the interaction event during the presentation of the interaction element based on the scene-induced pupil response variation characteristics for each region of the interaction element.
6. The method of claim 1, wherein the interaction event is classified using a machine learning technique based on the pupillary response and the different illumination characteristics of each region.
7. The method of claim 1, further comprising: adjusting content in response to determining the interaction event.
8. The method of claim 1, wherein the pupillary response is: a direction of the pupillary response; a velocity of the pupillary response; or pupillary fixations.
9. The method of claim 1, wherein the pupillary response is derived from a saccade characteristic.
10. The method of claim 1, wherein the physiological data comprises an image of an eye or electrooculography (EOG) data.
11. The method of claim 1, wherein the physiological data comprises head movements.
12. The method of claim 1, wherein determining the pupillary response during the presentation of the interaction element is based on determining a variability of the pupillary response to a threshold.
13. The method of claim 1, wherein the device is a head-mounted device (HMD).
14. The method of claim 1, wherein the presentation of the interaction element is an extended reality (XR) experience.
15. A device comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:obtaining physiological data associated with a pupil during presentation of an interaction element, the interaction element comprising a first region and a second region, each region having different illumination characteristics;determining, based on the obtained physiological data, a pupillary response during the presentation of the interaction element; and
determining, based on the different illumination characteristics, whether the pupillary response corresponds to directing attention to the first region based on luminance in the first region;
determining an interaction event during the presentation of the interaction element based on determining that the pupillary response corresponds to directing attention to the first region during the presentation of the interaction element.
16. The device of claim 15, wherein the different illumination characteristics of each region of the interaction element comprises one or more dark regions and one or more bright regions.
17. The device of claim 15, wherein each region of the interaction element comprises a level of luminance and the different illumination characteristics of each region are based on the level of luminance of each region with respect to an illuminance threshold level.
18. The device of claim 15, wherein the presentation of the interaction element comprises pixel information for a plurality of pixels and determining that the pupillary response corresponds to directing attention to the first region of the interaction element comprises: determining an estimated perceived luminance for each pixel in the first region based on the pixel information.
19. The device of claim 18, wherein determining an interaction event comprises: determining scene-induced pupil response variation characteristics for the regions of the interaction element; and determining the interaction event during the presentation of the interaction element based on the scene-induced pupil response variation characteristics for each region of the interaction element.
20. A non-transitory computer-readable storage medium, storing program instructions executable by one or more processors on a device to perform operations comprising: obtaining physiological data associated with a pupil during presentation of an interaction element, the interaction element comprising a first region and a second region, each region having different illumination characteristics; determining, based on the obtained physiological data, a pupillary response during the presentation of the interaction element; and
determining, based on the different illumination characteristics, whether the pupillary response corresponds to directing attention to the first region based on luminance in the first region;
determining an interaction event during the presentation of the interaction element based on determining that the pupillary response corresponds to directing attention to the first region during the presentation of the interaction element.
1. A method comprising: at a device comprising a processor:
obtaining physiological data associated with a pupil during presentation of an interaction element, the interaction element comprising a first region and a second region, each region having different illumination characteristics;
determining, based on the obtained physiological data, a pupillary response during the presentation of the interaction element;
determining that the pupillary response corresponds to attention response characteristics associated with attention to the first region of the interaction element based on the different illumination characteristics between the first region and the second region;
and determining an interaction event during the presentation of the interaction element based on determining that the pupillary response corresponds to directing attention to the first region during the presentation of the interaction element.
2. The method of claim 1, wherein the different illumination characteristics of each region of the interaction element comprises one or more dark regions and one or more bright regions.
3. The method of claim 1, wherein each region of the interaction element comprises a level of luminance and the different illumination characteristics of each region are based on the level of luminance of each region with respect to an illuminance threshold level.
4. The method of claim 1, wherein the presentation of the interaction element comprises pixel information for a plurality of pixels and determining that the pupillary response corresponds to directing attention to the first region of the interaction element comprises: determining an estimated perceived luminance for each pixel in the first region based on the pixel information.
5. The method of claim 4, wherein determining an interaction event comprises: determining scene-induced pupil response variation characteristics for the regions of the interaction element; and determining the interaction event during the presentation of the interaction element based on the scene-induced pupil response variation characteristics for each region of the interaction element.
6. The method of claim 1, wherein the interaction event is classified using a machine learning technique based on the pupillary response and the different illumination characteristics of each region.
7. The method of claim 1, further comprising: adjusting content in response to determining the interaction event.
8. The method of claim 1, wherein the pupillary response is: a direction of the pupillary response; a velocity of the pupillary response; or pupillary fixations.
9. The method of claim 1, wherein the pupillary response is derived from a saccade characteristic.
10. The method of claim 1, wherein the physiological data comprises an image of an eye or electrooculography (EOG) data.
11. The method of claim 1, wherein the physiological data comprises head movements.
12. The method of claim 1, wherein determining the pupillary response during the presentation of the interaction element is based on determining a variability of the pupillary response to a threshold.
13. The method of claim 1, wherein the device is a head-mounted device (HMD).
14. The method of claim 1, wherein the presentation of the interaction element is an extended reality (XR) experience.
15. A device comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: obtaining physiological data associated with a pupil during presentation of an interaction element, the interaction element comprising a first region and a second region, each region having different illumination characteristics; determining, based on the obtained physiological data, a pupillary response
during the presentation of the interaction element;
determining that the pupillary response corresponds to attention response characteristics associated with attention to the first region of the interaction element based on the different illumination characteristics between the first region and the second region; and
determining an interaction event during the presentation of the interaction element based on determining that the pupillary response corresponds to directing attention to the first region during the presentation of the interaction element.
16. The device of claim 15, wherein the different illumination characteristics of each region of the interaction element comprises one or more dark regions and one or more bright regions.
17. The device of claim 15, wherein each region of the interaction element comprises a level of luminance and the different illumination characteristics of each region are based on the level of luminance of each region with respect to an illuminance threshold level.
18. The device of claim 15, wherein the presentation of the interaction element comprises pixel information for a plurality of pixels and determining that the pupillary response corresponds to directing attention to the first region of the interaction element comprises: determining an estimated perceived luminance for each pixel in the first region based on the pixel information.
19. The device of claim 18, wherein determining an interaction event comprises: determining scene-induced pupil response variation characteristics for the regions of the interaction element; and determining the interaction event during the presentation of the interaction element based on the scene-induced pupil response variation characteristics for each region of the interaction element.
20. A non-transitory computer-readable storage medium, storing program instructions executable by one or more processors on a device to perform operations comprising: obtaining physiological data associated with a pupil during presentation of an interaction element, the interaction element comprising a first region and a second region, each region having different illumination characteristics; determining, based on the obtained physiological data, a pupillary response during the presentation of the interaction element;
determining that the pupillary response corresponds to attention response characteristics associated with attention to the first region of the interaction element based on the different illumination characteristics between the first region and the second region; and
determining an interaction event during the presentation of the interaction element based on determining that the pupillary response corresponds to directing attention to the first region during the presentation of the interaction element.
Allowable Subject Matter
Claims 4-5 and 18-19 would be allowable if rewritten to overcome the rejection(s) set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 4:
The prior art does not teach or suggest either singularly or in combination the at least claimed “wherein the presentation of the interaction element comprises pixel information for a plurality of pixels and determining that the pupillary response corresponds to directing attention to the first region of the interaction element comprises: determining an estimated perceived luminance for each pixel in the first region based on the pixel information”, in combination with the other recited claim features.
Regarding claim 5:
Claim 5 depends on claim 4 and is found allowable for at least the same reason as discussed above.
Regarding claim 18:
The limitations of claim 18 have been addressed in the discussion of claim 4 above.
Regarding claim 19:
Claim 19 depends on claim 18 and is found allowable for at least the same reason as discussed above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Koosha Sharifi-Tafreshi whose telephone number is (571)270-5897. The examiner can normally be reached Mon - Fri 8AM to 5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nitin Patel can be reached at (571) 272-7677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KOOSHA SHARIFI-TAFRESHI/ Primary Examiner, Art Unit 2628
Prosecution Insights