DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-2 and 4-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Devani (U.S. Patent Application Publication No. 2023/0052100) hereinafter referred to as Devani; in view of Medathati et al. (Medathati, Naga Venkata Kartheek, Ruta Desai, and James Hillis. "Towards inferring cognitive state changes from pupil size variations in real world conditions." ACM Symposium on Eye Tracking Research and Applications. 2020) hereinafter referred to as Medathati; in view of Giovinazzo et al. (U.S. Patent Application Publication No. 2019/0191995) hereinafter referred to as Giovinazzo.
Regarding claim 1, Devani teaches a method comprising:
monitoring pupils (¶[0051] certain stimulus including a pupil of a user) of individuals in environments (¶[0051] field of view in front of the camera);
capturing, as received sensor data, sensor data associated with pupillary activity of the individuals (Fig. 3);
generating baselines via consolidation of pupillary activity data comprising the pupillary activity curves, individual ones of the baselines representing a known physiological curve without at least one of the physiological conditions (¶[0042], ¶[0060]), the baselines being identified via baseline tags and being stored with the baseline tags (¶[0077]) as another portion of the library data in the library databases (¶[0054], ¶[0138]);
training a machine learning (ML) model using the library data (¶[0079]);
receiving current sensor data associated with current pupillary activity associated with a simultaneous scan of a pair of pupils of an individual in response to at least one of external or internal stimuli (¶[0085], Fig. 5-1, Fig. 5-2-A, Fig. 5-2-B);
generating a current pupillary activity curve associated with the current sensor data (¶[0085], Fig. 501, element 502 plot);
performing, by a curve classification algorithm and utilizing the ML model, a comparison between the current pupillary activity curve and the pupillary activity curves based on the baselines and mathematically generated classifications (¶[0078-0081]); and
determining a physiological condition identifier associated with a result of the comparison (¶[0118], for example, determining psychosensory response throughout, as a biomarker).
While Devani teaches a trained machine learning model for the purposes of classifying pupillary activity into cognitive states, Devani is silent as to the training data and fails to be explicit with respect to collecting, as pupillary activity curves, individual ones of datasets in a group of database files comprising, as an array or a string of numbers, data of the received sensor data, individual ones of the pupillary activity curves representing a change in a pupil characteristic over time, the data being represented by a two-dimensional line; generating, without intermediate metric computations, classification models representing mathematically generated classifications of the pupillary activity curves based on physiological conditions associated with the pupillary activity curves, the mathematically generated classifications being identified via classification tags and being stored with the classification tags as a portion of library data in library databases, the library data comprising the pupillary activity curves. Devani further does not teach outputting a physiological condition identifier associated with a result of the comparison.
Attention is brought to the Medathati reference, which teaches collecting, as pupillary activity curves, individual ones of datasets in a group of database files comprising, as an array or a string of numbers, data of the received sensor data, individual ones of the pupillary activity curves representing a change in a pupil characteristic over time, the data being represented by a two-dimensional line (§ 4.2 Event-related Pupillary Response (EPR), including §§ 4.2.1, 4.2.2, Figs. 1-4)
generating, without intermediate metric computations (this is interpreted as additional computations of metrics, p. 6, col. 2, ¶ 4, baseline raw pupil dilation, normalized), classification models representing mathematically generated classifications of the pupillary activity curves based on physiological conditions associated with the pupillary activity curves (§ 5.2 ML methods for classification), the mathematically generated classifications being identified via classification tags and being stored with the classification tags as a portion of library data in library databases (§ 5 Event Detection Using Binary Classification, “supervised machine learning”, and labels for the training data denoted by participant responses), the library data comprising the pupillary activity curves (p. 6, col. 1, ¶ 2 epochs extracted by sliding windowing operation).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify the machine learning system of Devani to include a machine learning training technique, as taught by Medathati, because Medathati teaches a process that confirms “advanced ML methods hold the promise of using this signal to infer cognitive state variation in relatively naturalistic real world scenario,” (Medathati, § Discussion and Conclusion, col. 2, ending ¶).
Devani as modified further does not teach outputting a physiological condition identifier associated with a result of the comparison.
Attention is drawn to the Giovinazzo reference, which teaches a machine learning model (¶¶[0096-0097]) for analyzing a pupillary light reflex and outputting a physiological condition identifier (¶¶[0101-0106]) associated with a result of a comparison between baseline and current pupillary light reflex (¶¶[0098-0099]).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify the pupil response analysis of Devani as modified to include outputting a physiological condition, as taught by Giovinazzo, because Gionvinazzo teaches a list of benefits including speed of results, and portability, e.g. (Giovinazzo ¶¶¶0019-0027]).
Regarding claim 2, Devani as modified teaches the method of claim 1.
Devani further teaches wherein collecting the datasets further comprises calibrating the pupillary activity curves by:
collecting light level data associated with the environments in which the sensor data is captured (¶[0089] video data captures changes in ambient light);
collecting video data representing the pupillary activity (¶[0089] video data);
transforming the video data into time series curve data associated with changes of diameters of the pupils over time (¶[0090]); and
collecting, as the pupillary activity curves which include pupillary light reflex (PLR) curves, the datasets based on the light level data, the video data, and the time series curve data (¶¶[0089-0090]).
Regarding claim 4, Devani as modified teaches the method of claim 1.
Devani teaches further comprising: maintaining the library data in the library databases by performing i) ongoing analysis of additional sensor data associated with additional individuals (¶[0064]), ii) ongoing collection of additional pupillary activity curves based on the additional sensor data (¶[0076]), iv) updating of the baselines as updated baselines based on the additional pupillary activity curves (¶[0096], ¶[0104]), and v) updating of the library data based on the additional pupillary activity curves, the additional classification models, and the updated baselines, the additional classification models being utilized to relatively increase a level of accuracy of ongoing identification of additional physiological conditions associated with the additional pupillary activity curves (¶[0042] longitudinal data collection, ¶[0060], ¶[0125]).
Devani does not teach iii) ongoing generation of additional classification models based on the additional pupillary activity curves, the additional classification models being utilized to relatively increase a level of accuracy of ongoing identification of additional physiological conditions associated with the additional pupillary activity curves
Gionvinazzo further teaches iii) ongoing generation of additional classification models based on the additional pupillary activity curves, the additional classification models being utilized to relatively increase a level of accuracy of ongoing identification of additional physiological conditions associated with the additional pupillary activity curves (¶¶[0107-0115]).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify the pupil response analysis of Devani as modified to include improving the machine learning model over time, as taught by Giovinazzo, because Gionvinazzo teaches a list of benefits including speed of results, accuracy, and portability, e.g. (Giovinazzo ¶¶¶0019-0027]).
Regarding claim 5, Devani as modified teaches the method of claim 1.
Devani further teaches wherein performing, by the ML model, the comparison between the current pupillary activity curve and the pupillary activity curves further comprises:
matching at least one characteristic of the individual with at least one corresponding characteristic of individual ones of a first subset of the individuals associated with the pupillary activity curves, the at least one characteristic comprising at least one of an eye color (¶[0059], ¶[0067] used for more robust baseline for comparison); and
performing, by the ML model, the comparison between the current pupillary activity curve and a second subset of the pupillary activity curves associated with the first subset of the individuals (¶[0101]).
Giovinazzo also teaches matching individual characteristics including eye color and patient sex (¶[0092]).
Regarding claim 6, Devani as modified teaches the method of claim 1.
Devani further teaches wherein performing, by the ML model, the comparison between the current pupillary activity curve and the pupillary activity curves further comprises: outputting, by the ML model, a predictive performance measurement associated with the current pupillary activity curve (¶[0126]), the predictive performance measurement indicating a probability of a future likelihood of the individual experiencing at least one physiological condition (¶[0116]).
Devani does not teach the at least one physiological condition comprising at least one of a coma or a vegetative state.
Giovinazzo further teaches the at least one physiological condition comprising at least one of a coma or a vegetative state (¶[0227]).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify the pupil response analysis of Devani as modified to include known pupil responses in a coma, as taught by Giovinazzo, because Gionvinazzo teaches a list of benefits including speed of results, accuracy, and portability, e.g. (Giovinazzo ¶¶¶0019-0027]).
Regarding claim 7, Devani as modified teaches the method of claim 1.
Devani further teaches wherein performing, by the ML model, the comparison between the current pupillary activity curve and the pupillary activity curves further comprises: analyzing, by the ML model, the current pupillary activity curve (¶[0126]); and identifying, in response to the analyzing of the current pupillary activity curve, at least one physiological condition with which the physiological condition identifier is associated (¶[0116], ¶[0118]).
Devani does not teach the at least one physiological condition comprising at least one of an injury, intoxication, deception, a mental state, fatigue, dementia, toxins, a disease, a heart condition, or diabetes.
Giovinazzo further teaches at least one physiological condition comprising at least one of an injury, intoxication, deception, a mental state, fatigue, dementia, toxins, a disease, a heart condition, or diabetes (¶[0055], long list of conditions in ¶¶[0123-0164] and more throughout).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify the pupil response analysis of Devani as modified to include known pupillary responses for additional conditions, as taught by Giovinazzo, because Gionvinazzo teaches a list of benefits including speed of results, accuracy, and portability, e.g. (Giovinazzo ¶¶¶0019-0027]).
Regarding claims 8-15/17-20, the claims are directed to a system and computer readable medium comprising substantially the same subject matter as the method of claims 1-2 and is rejected under substantially the same sections of Devani, Medathati, and Giovinazzo.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Devani, Medathati, and Giovinazzo as applied to claim 1 above, and further in view of Rotenstreich et al. (U.S. Patent Application Publication No. 2023/0360220) hereinafter referred to as Rotenstreich.
Regarding claim 3, Devani as modified teaches the method of claim 1.
Devani as modified does not teach wherein: performing, by the ML model, the comparison between the current pupillary activity curve and the pupillary activity curves further comprises performing, via the ML model, support vector machine (SVM) analysis of the current pupillary activity curve based on the pupillary activity curves in the library databases; and
outputting the physiological condition identifier further comprises outputting principal component analysis (PCA) data indicating a PCA condition with which the physiological condition identifier is associated.
Attention is brought to the Giovinazzo reference, which teaches performing, by the ML model, the comparison between the current pupillary activity curve and the pupillary activity curves further comprises performing, via the ML model, support vector machine (SVM) analysis of the current pupillary activity curve based on the pupillary activity curves in the library databases (¶[0096]).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify the pupil response analysis of Devani as modified to include a support vector machine, as taught by Giovinazzo, because Gionvinazzo teaches a list of benefits including speed of results, accuracy, and portability, e.g. (Giovinazzo ¶¶¶0019-0027]).
Devani as modified does not teach outputting the physiological condition identifier further comprises outputting principal component analysis (PCA) data indicating a PCA condition with which the physiological condition identifier is associated.
Attention is drawn to the Rotenstreich reference, which teaches outputting the physiological condition identifier further comprises outputting principal component analysis (PCA) data indicating a PCA condition with which the physiological condition identifier is associated (¶[0214]).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify the pupil response analysis of Devani as modified to include principal component analysis, as taught by Rotenstreich, because Rotenstreich teaches additional analysis features complement and augment image classification by machine learning (Rotenstreich ¶[0214]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMANDA L STEINBERG whose telephone number is (303)297-4783. The examiner can normally be reached Mon-Fri 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Unsu Jung can be reached at (571) 272-8506. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMANDA L STEINBERG/ Examiner, Art Unit 3792