DETAILED ACTION
Claims 1-20 are currently pending.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/19/26 has been entered.
Response to Arguments
Applicant's arguments filed 3/19/26 have been fully considered but they are not persuasive.
The specification has now been amended (see above) to clarify that paragraph [0082] referred only to the narrow physiological experiments shown in FIGS. 4-8. The as-filed disclosure provides abundant support for the multimodal invention.
The amendment to the specification has been objected because it introduces new matter into the disclosure. Applicant’s original disclosure does not provide support for “providing the one or more biopotential signals and the one or more video images collected from the patient of interest as inputs to the trained neural network model” and “applying the trained neural network model to the one or more biopotential signals-collected from the patient of interest and to the one or more video images collected from the patient of interest to identify a pain level of the patient of interest while also accounting for the non-pain affect state as a discrete class” as discussed in rejections under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, below.
Response to Amendment
The amendment filed 3/19/26 is objected to under 35 U.S.C. 132(a) because it introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment shall introduce new matter into the disclosure of the invention. The added material which is not supported by the original disclosure is as follows:
Applicant’s addition and deletion of language in paragraph 82 is improper as it seeks to introduce new matter into the disclosure of the invention.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 120 as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed application, Application No. 63/142,010, fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application.
Application No. 63/142,010 fails to disclose the limitation of claim 1 of “providing the one or more biopotential signals and the one or more video images collected from the patient of interest as inputs to the trained neural network model” and “applying the trained neural network model to the one or more biopotential signals-collected from the patient of interest and to the one or more video images collected from the patient of interest to identify a pain level of the patient of interest while also accounting for the non-pain affect state as a discrete class.” Application No. 63/142,010 states on page 8, paragraph 17 that “The bioVid emotion dataset contains three biopotential signals: skin conductance level or electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of trapezius muscle, and videos of participants’ frontal face.” However, Application No. 63/142,010 states on page 9, paragraph 17 that “In this work, we focus only on the biopotential signals as studies in healthcare indicated that biopotential signals are major objective indicators of pain and other affect.” It is apparent from the disclosure in Application No. 63/142,010 that while videos are included in the bioVid emotion dataset, the disclosure in Application No. 63/142,010 does not describe applying the neural network model to the video images collected from the patient to identify a pain level of the patient.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventors, at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “providing the one or more biopotential signals and the one or more video images collected from the patient of interest as inputs to the trained neural network model” and “applying the trained neural network model to the one or more biopotential signals-collected from the patient of interest and to the one or more video images collected from the patient of interest to identify a pain level of the patient of interest while also accounting for the non-pain affect state as a discrete class.” Applicant’s Specification lacks any discussion of how the captured image data of the patient is used to identify a pain level of the patient of interest. The Applicant’s Specification on page 13, paragraph 55 states that “The video image capture device 105 would be placed above the patient of interest in a manner to allow for the capture of facial expressions from the neonate patient.” However, there is no further discussion of analyzing facial expressions with the trained neural network model to identify a pain level. Further, the Applicant’s Specification states that “The main limitation of this study is that it is limited to physiological signals only. In future work, behavioral data can be explored, such as face images and facial action units, along with the combination of physiological and behavioral data.” Based on this statement, it appears that analyzing facial expressions is not described in the present disclosure. Thus, one of ordinary skill in the art would not have reasonably considered Applicant to be in possession of the claimed limitations based on Applicant’s limited description of capturing facial expressions. Claims 12 and 16 recite similar limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature Md Taufeeq Uddin et al. "Accounting for Affect in Pain Level Recognition" November 2020. pages 1-11. Machine Learning for Health (hereafter “Uddin”) and Lanzkowsky US Publication 2019/0313966 (hereafter “Lanzkowsky”).
Referring to claims 1, 12 and 16, Uddin discloses a computer-implemented method for identifying a pain level of a patient of interest, the method comprising:
establishing a pain-affect dataset by merging a pain dataset comprising data acquired from a plurality of patients in response to a stimulus for eliciting pain with an affect dataset comprising data acquired from the plurality of patients in response to a stimulus to elicit a non-pain affect state in the plurality of patients (page 2, We curate a new dataset by merging the publicly available bioVid pain Walter et al.; (2013) and bioVid emotion Zhang et al. (2016) datasets), wherein the affect dataset includes a plurality of discrete affects merged into an affect class and wherein the pain-affect dataset includes a baseline class, a low level pain class, a high level pain class and the affect class (page 2-3, To do so, we propose to incorporate affect in PL recognition model as a category (e.g., A in our studied dataset) along with multiple pain levels (e.g., LLP -low-level pain, HLP - high-level pain in our studied dataset; see Appendix A Section 2 for details). Depending on the context, baseline (BL) category could be incorporated in affect category as BL is likely to be a neutral/relaxed affect state);
training a neural network model using the established pain-affect dataset (page 3, To investigate
the performance of PLR models in above mentioned cases, we built the models in unimodel and multimodel settings, i.e., we trained and tested PLR model on EDA, ECG, EMG separately, and their combination (EDA + ECG + EMG));
collecting one or more biopotential signals of the patient of interest (page 3, To create a feature vector for a given sample, we downsampled the biopotential signals by computing the moving average using a sliding window with 80% overlap);
providing the one or more biopotential signals collected from the patient of interest as inputs to the trained neural network model (page 3, To investigate the performance of PLR models in above mentioned cases, we built the models in unimodel and multimodel settings, i.e., we trained and tested PLR model on EDA, ECG, EMG separately, and their combination (EDA + ECG + EMG));
applying the trained neural network model to the one or more biopotential signals collected from the patient of interest to identify a pain level of the patient of interest while also account for the non-pain affect state as a discrete class (page 4, In case 5, we take the affect into account in our PLR model).
While Uddin discloses identifying a pain level of the patient of interest, Uddin does not disclose expressly applying the trained neural network model to the one or more video image collected from the patient of interest to identify a pain level of the patient of interest.
Lanzkowsky capturing one or more video images of a patient of interest (paragraph 63, In some embodiments, operation 101 includes collecting images of the patient while the patient is interacting with the diagnostic system 202. These images may be video or may be individual still photographic images from cameras 205. The image may include a facial expression); and
providing the one or more video images collected from the patient of interest as inputs to the trained neural network model (paragraph 11, Operation 104 includes combining and analyzing collected data to determine pain state information);
applying the trained neural network model to the one or more video images collected from the patient of interest to identify a pain level of the patient of interest (paragraph 12, The detection and processing of facial expression are achieved through various methods such as optical flow, hidden Markov models, neural network processing or active appearance models).
At the time of the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to identify a pain level by analyzing a face of the patient. The motivation for doing so would have been to improve the accuracy of pain detection by accounting for recognized emotions of the patient. Therefore, it would have been obvious to combine Lanzkowsky with Uddin to obtain the invention as specified in claims 1, 12 and 16.
Referring to claims 2, 13 and 17, Uddin discloses wherein one or more of the plurality of discrete affects are selected from amusement, anger, disgust, fear and sadness (page 8, In the bioVid emotion dataset, video clips from movies were used to elicit spontaneous discrete emotions including amusement (Am), anger (An), disgust (Di), fear (F), and sadness (S)).
Referring to claim 3, Uddin discloses wherein the pain dataset further comprises data acquired from the plurality of patients in response to no stimulus for eliciting pain to establish the baseline class (page 8, There are five pain levels in the pain dataset including baseline).
Referring to claims 4, 14 and 18, Uddin discloses wherein the pain dataset is a bioVID pain dataset (page 2, We curate a new dataset by merging the publicly available bioVid pain Walter et al.; (2013) and bioVid emotion Zhang et al. (2016) datasets).
Referring to claim 5, Uddin discloses wherein the bioVid pain dataset comprises face image data and data collected from one or more biopotential signals of the plurality of patients (page 8, In the merged bioVid pain-affect dataset, we only selected the common biopotential signals (e.g., EDA, ECG, EMG from trapezius muscle) and videos).
Referring to claim 6, Uddin discloses wherein the one or more biopotential signals are selected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle, EMG of a corrugator muscle and EMG of a zygomaticus muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face. In addition to the above-mentioned modalities, bioVid pain dataset contains EMG signal collected from corrugator and zygomaticus muscles).
Referring to claims 7, 15 and 19, Uddin discloses wherein the non-pain affect dataset is a bioVid emotion dataset (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face).
Referring to claim 8, Uddin discloses wherein the bioVid emotion dataset comprises face image data and data collected from one or more biopotential signals of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face).
Referring to claim 9, Uddin discloses wherein the biopotential signals are selected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face).
Referring to claims 10 and 20, Uddin discloses wherein the pain dataset is a bioVid pain dataset comprising image data and data collected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle, EMG of a corrugator muscle and EMG of a zygomaticus muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face. In addition to the above-mentioned modalities, bioVid pain dataset contains EMG signal collected from corrugator and zygomaticus muscles), the non-pain affect dataset is a bioVid affect dataset comprising image data and data collected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle of the plurality of patients (page 8, The bioVid emotion dataset contains three biopotential signals: EDA, ECG, EMG of the trapezius muscle, and videos of participants' frontal face), and wherein the pain-affect dataset comprises merged image data and data collected from electrodermal activity (EDA), electrocardiogram (ECG), electromyogram (EMG) of a trapezius muscle of the plurality of patients (page 8, In the merged bioVid pain-affect dataset, we only selected the common biopotential signals (e.g., EDA, ECG, EMG from trapezius muscle) and videos).
Referring to claim 11, Uddin discloses wherein the one or more biopotential signals collected from the patient of interest are selected from electrodermal activity (EDA), electrocardiogram (ECG) and electromyogram (EMG) of a trapezius muscle of the patient of interest (page 8, In the merged bioVid pain-affect dataset, we only selected the common biopotential signals (e.g., EDA, ECG, EMG from trapezius muscle) and videos).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights