DETAILED ACTION
Claims 1-20 are currently pending.
Response to Arguments
Applicant’s arguments, see pages 9-11, filed 6/15/26, with respect to the rejections of claims 1-20 under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, have been fully considered and are persuasive. The rejections of claims 1-20 under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, have been withdrawn.
Applicant's remaining arguments filed 6/15/26 have been fully considered but they are not persuasive.
The Applicant argues on page 9 of the response in essence that: The amendment to paragraph [82] is a modest clarification that the “main limitation of this study” sentence refers only to the narrow physiological-only experiments shown in FIGS. 4A- 8D. This clarification is fully supported by the original disclosure, which distinguishes those experiments from the broader multimodal system and method affirmatively disclosed in the Summary of the Invention ([012]), the system embodiment ([055]-[059] and FIG. 1), and the dataset description ([060]-[064]).
The amendment to the specification has been objected because it introduces new matter into the disclosure. The amendment is not explicitly or inherently present in the original disclosure, and is not a correction of an obvious error. See MPEP 2163.07. Thus, the amendment to the specification has not been entered.
The Applicant argues on page 11 of the response in essence that: The Examiner's denial of priority is respectfully traversed. The present specification provides written description support for the claims, as set forth in Section Il above. Accordingly, the claims are entitled to the benefit of the January 27, 2021, filing date of U.S. Provisional Application No. 63/142,010.
It is not sufficient that the present specification provides written support for the claims. To comply with the written description requirement of 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph, or to be entitled to an earlier priority date or filing date under 35 U.S.C. 119, 120, 365, or 386, each claim limitation must be expressly, implicitly, or inherently supported in the originally filed disclosure. When an explicit limitation in a claim "is not present in the written description whose benefit is sought it must be shown that a person of ordinary skill would have understood, at the time the patent application was filed, that the description requires that limitation." Hyatt v. Boone, 146 F.3d 1348, 1353, (Fed. Cir. 1998). Application No. 63/142,010 fails to disclose the limitation of claim 1 of “providing the one or more biopotential signals and the one or more video images collected from the patient of interest as inputs to the trained neural network model” and “applying the trained neural network model to the one or more biopotential signals-collected from the patient of interest and to the one or more video images collected from the patient of interest to identify a pain level of the patient of interest while also accounting for the non-pain affect state as a discrete class.”
The Applicant argues on page 12 of the response in essence that: Uddin does not disclose or suggest a neural network model.
Uddin discloses using the Extreme Gradient Boosting classification algorithm of Chen and Guestrin (page 3). The algorithm of Chen and Guestrin called XGBoost is a neural network. See Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785-794, 2016. Furthermore, Lanzkowsky discloses that the detection and processing of facial expression are achieved through neural network processing (paragraph 12).
The Applicant argues on page 13 of the response in essence that: Lanzkowsky does not cure these deficiencies. Lanzkowsky describes real-time, single- patient multimodal monitoring but does not disclose merging a pain dataset with an affect dataset from multiple subjects, collapsing discrete affects into a single "Affect (A)" class, or creating and training on the specific four-class pain-affect dataset (Baseline/Low Level Pain/High Level Pain /Affect) that includes merged image data.
In response to Applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Uddin discloses merging a pain dataset with an affect dataset from multiple subjects, collapsing discrete affects into a single "Affect (A)" class, or creating and training on the specific four-class pain-affect dataset (Baseline/Low Level Pain/High Level Pain /Affect) that includes merged image data.
The Applicant argues on page 13 of the response in essence that: There is no teaching or motivation in the cited references to combine Uddin's physiological-only study with Lanzkowsky's single-patient system in the precise manner claimed.
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to identify a pain level by analyzing a face of the patient. The motivation for doing so would have been to improve the accuracy of pain detection by accounting for recognized emotions of the patient.
Response to Amendment
The amendment filed 6/15/26 is objected to under 35 U.S.C. 132(a) because it introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment shall introduce new matter into the disclosure of the invention. The added material which is not supported by the original disclosure is as follows:
Applicant’s addition and deletion of language in paragraph 82 is improper as it seeks to introduce new matter into the disclosure of the invention.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 120 as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed application, Application No. 63/142,010, fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application.
Application No. 63/142,010 fails to disclose the limitation of claim 1 of “providing the one or more biopotential signals and the one or more video images collected from the patient of interest as inputs to the trained neural network model” and “applying the trained neural network model to the one or more biopotential signals-collected from the patient of interest and to the one or more video images collected from the patient of interest to identify a pain level of the patient of interest while also accounting for the non-pain affect state as a discrete class.” Application No. 63/142,010 states on page 8, paragraph 17 that “The bioVid emotion dataset contains three biopotential signals: skin conductance level or electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of trapezius muscle, and videos of participants’ frontal face.” However, Application No. 63/142,010 states on page 9, paragraph 17 that “In this work, we focus only on the biopotential signals as studies in healthcare indicated that biopotential signals are major objective indicators of pain and other affect.” It is apparent from the disclosure in Application No. 63/142,010 that while videos are included in the bioVid emotion dataset, the disclosure in Application No. 63/142,010 does not describe applying the neural network model to the video images collected from the patient to identify a pain level of the patient.
Further, Applicant states on page 12 of the remarks of 6/15/26 that “Uddin does not disclose or suggest a neural network”. As the disclosure of Application No. 63/142,010 and Uddin share a mostly identical disclosure, Applicant’s statement is evidence that Application No. 63/142,010 lacks disclosure of a neural network. Applicant states on page 13 of the remarks of 6/15/26 that “Uddin never used image data in any model and expressly identified the combination of face images with physiological signals as a limitation of its study and subject for future work.” Applicant’s statement is evidence that Application No. 63/142,010 lacks disclosure of providing video images as input to the trained neural network model.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature Md Taufeeq Uddin et al. "Accounting for Affect in Pain Level Recognition" November 2020. pages 1-11. Machine Learning for Health (hereafter “Uddin”) and Lanzkowsky US Publication 2019/0313966 (hereafter “Lanzkowsky”).
Referring to claims 1, 12 and 16, Uddin discloses a computer-implemented method for identifying a pain level of a patient of interest, the method comprising:
establishing a pain-affect dataset by merging a pain dataset comprising data acquired from a plurality of patients in response to a stimulus for eliciting pain with an affect dataset comprising data acquired from the plurality of patients in response to a stimulus to elicit a non-pain affect state in the plurality of patients (page 2, We curate a new dataset by merging the publicly available bioVid pain Walter et al.; (2013) and bioVid emotion Zhang et al. (2016) datasets), wherein the affect dataset includes a plurality of discrete affects merged into an affect class and wherein the pain-affect dataset includes a baseline class, a low level pain class, a high level pain class and the affect class (page 2-3, To do so, we propose to incorporate affect in PL recognition model as a category (e.g., A in our studied dataset) along with multiple pain levels (e.g., LLP -low-level pain, HLP - high-level pain in our studied dataset; see Appendix A Section 2 for details). Depending on the context, baseline (BL) category could be incorporated in affect category as BL is likely to be a neutral/relaxed affect state);
training a neural network model using the established pain-affect dataset (page 3, To investigate
the performance of PLR models in above mentioned cases, we built the models in unimodel and multimodel settings, i.e., we trained and tested PLR model on EDA, ECG, EMG separately, and their combination (EDA + ECG + EMG));
collecting one or more biopotential signals of the patient of interest (page 3, To create a feature vector for a given sample, we downsampled the biopotential signals by computing the moving average using a sliding window with 80% overlap);
providing the one or more biopotential signals collected from the patient of interest as inputs to the trained neural network model (page 3, To investigate the performance of PLR models in above mentioned cases, we built the models in unimodel and multimodel settings, i.e., we trained and tested PLR model on EDA, ECG, EMG separately, and their combination (EDA + ECG + EMG));
applying the trained neural network model to the one or more biopotential signals collected from the patient of interest to identify a pain level of the patient of interest while also account for the non-pain affect state as a discrete class (page 4, In case 5, we take the affect into account in our PLR model).
While Uddin discloses identifying a pain level of the patient of interest, Uddin does not disclose expressly applying the trained neural network model to the one or more video image collected from the patient of interest to identify a pain level of the patient of interest.
Lanzkowsky discloses capturing one or more video images of a face of a patient of interest (paragraph 63, In some embodiments, operation 101 includes collecting images of the patient while the patient is interacting with the diagnostic system 202. These images may be video or may be individual still photographic images from cameras 205. The image may include a facial expression); and
providing the one or more video images collected from the patient of interest as inputs to the trained neural network model (paragraph 11, Operation 104 includes combining and analyzing collected data to determine pain state information);
applying the trained neural network model to the one or more video images collected from the patient of interest to identify a pain level of the patient of interest (paragraph 12, The detection and processing of facial expression are achieved through various methods such as optical flow, hidden Markov models, neural network processing or active appearance models).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to identify a pain level by analyzing a face of the patient. The motivation for doing so would have been to improve the accuracy of pain detection by accounting for recognized emotions of the patient. Therefore, it would have been obvious to combine Lanzkowsky with Uddin to obtain the invention as specified in claims 1, 12 and 16.
Referring to claims 2, 13 and 17, Uddin discloses wherein one or more of the plurality of discrete affects are selected from amusement, anger, disgust, fear and sadness (page 8, In the bioVid emotion dataset, video clips from movies were used to elicit spontaneous discrete emotions including amusement (Am), anger (An), disgust (Di), fear (F), and sadness (S)).
Referring to claim 3, Uddin discloses wherein the pain dataset further comprises data acquired from the plurality of patients in response to no stimulus for eliciting pain to establish the baseline class (page 8, There are five pain levels in the pain dataset including baseline).
Referring to claims 4, 14 and 18, Uddin discloses wherein the pain dataset is a bioVID pain dataset (page 2, We curate a new dataset by merging the publicly available bioVid pain Walter et al.; (2013) and bioVid emotion Zhang et al. (2016) datasets).
Referring to claim 5, Uddin discloses wherein the bioVid pain dataset comprises face image data and data collected from one or more biopotential signals of the plurality of patients (page 8, In the merged bioVid pain-affect dataset, we only selected the common biopotential signals (e.g., EDA, ECG, EMG from trapezius muscle) and videos).
Referring to claim 6, Uddin discloses wherein the one or more biopotential signals are selected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle, EMG of a corrugator muscle and EMG of a zygomaticus muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face. In addition to the above-mentioned modalities, bioVid pain dataset contains EMG signal collected from corrugator and zygomaticus muscles).
Referring to claims 7, 15 and 19, Uddin discloses wherein the non-pain affect dataset is a bioVid emotion dataset (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face).
Referring to claim 8, Uddin discloses wherein the bioVid emotion dataset comprises face image data and data collected from one or more biopotential signals of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face).
Referring to claim 9, Uddin discloses wherein the biopotential signals are selected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face).
Referring to claims 10 and 20, Uddin discloses wherein the pain dataset is a bioVid pain dataset comprising image data and data collected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle, EMG of a corrugator muscle and EMG of a zygomaticus muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face. In addition to the above-mentioned modalities, bioVid pain dataset contains EMG signal collected from corrugator and zygomaticus muscles), the non-pain affect dataset is a bioVid affect dataset comprising image data and data collected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face), and wherein the pain-affect dataset comprises merged image data and data collected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle of the plurality of patients (page 8, In the merged bioVid pain-affect dataset, we only selected the common biopotential signals (e.g., EDA, ECG, EMG from trapezius muscle) and videos).
Referring to claim 11, Uddin discloses wherein the one or more biopotential signals collected from the patient of interest are selected from electrodermal activity (EDA), electrocardiogram (ECG) and electromyogram (EMG) of a trapezius muscle of the patient of interest (page 8, In the merged bioVid pain-affect dataset, we only selected the common biopotential signals (e.g., EDA, ECG, EMG from trapezius muscle) and videos).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights