DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-13 and 15-20 are pending.
Claim 14 is canceled.
Claims 3-6 and 18-19 are withdrawn from consideration.
Claims 1-2, 7-13, 15-17 and 20 are being examined.
Prior Art
D1: US20220051437A1 Ostadabbas et al
D2: US20200320345A1 Nikolenko et al
D3: US20170055896A1 Al-Ali et al
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-2, 7-8, 16-17 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 in view of D2.
Regarding claims 1, 16 and 20, D1 teaches a method comprising:
receiving subject training data comprising a plurality of images of subjects on a person support apparatus performing a plurality of actions;
(D1, "which includes images from participants while lying in a bed", [0011]; "pose data was collected from each participant, while lying in a bed and randomly changing their poses under three main categories of supine, left side, and right side.", [0175]; D1 teaches receiving images of subjects on a person support apparatus (bed), but only static positions. D2, "The real data may come from any source, including video, real dynamic images and real static images.", [0017]; D2 teaches subject image data includes dynamic, action-depicting content; together D1 and D2 teach receiving images of subjects on a bed performing actions)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate D2 into D1 in order to capture image data of subjects performing actions. The combination of D1 and D2 also teaches other enhanced capabilities.
The combination of D1 and D2 further teaches:
labeling the plurality of images based on the actions performed by the subjects to generate labeled subject training data;
(D1, "the collected LWIR pose images were labeled by finding 14 body joints in each", [0178]; D1 teaches labeling images via body-joint locations reflecting dynamic body configuration, but the resulting label is a static pose. D2, "the data that makes up the dataset, which is used for training the system, is also labeled as part of the dataset generation", "gesture recognition, navigation, interior mapping, medical application and imagery", [0019]; D2 teaches such joint-derived labels are applied to gesture/action recognition; together D1 and D2 teach labeling images based on actions performed; incorporating D2 into D1 would label images by the action performed)
generating synthetic training data comprising computer generated images of artificial subjects on an artificial person support apparatus performing a plurality of actions;
(D2, Fig. 1; "the synthetic data is generated using computer graphics to prepare artificial imagery with objects similar to the objects present in the seed data.", [0021]; D2 teaches generating CGI imagery of artificial subjects from real seed data; "gesture recognition, navigation, interior mapping, medical application and imagery", [0019]; D2 teaches CGI generation used for gesture/action recognition; D1, "which includes images from participants while lying in a bed", [0011]; D1 supplies the in-bed seed data; together D1 and D2 teach generating artificial subjects on an artificial bed performing actions; incorporating D2 into D1 would expand training-data diversity cost-effectively)
labeling the synthetic training data based on the actions performed by the artificial subjects to generate labeled synthetic training data; and
(D1, "which includes images from participants while lying in a bed", [0011]; D1 supplies the real subject images used as seed data. D2, "the synthetic data is generated using computer graphics to prepare artificial imagery with objects similar to the objects present in the seed data.", [0021]; D2 teaches generating artificial imagery of D1's subjects. "As the synthetic data is generated, labels are used to provide accurate information as part of the dataset that is used for training the system.", [0004]; D2 teaches labeling that synthetic data as it is generated; "gesture recognition, navigation, interior mapping, medical application and imagery", [0019]; D2 teaches such labels are applied to gesture/action recognition; together D1 and D2 teach labeling the artificial-subject imagery by action; incorporating D2 into D1 would generate labeled synthetic action data)
training a machine learning model based on the labeled subject training data and the labeled synthetic training data, using supervised learning techniques, to generate a trained model to predict an action performed by a subject based on image data associated with the subject.
(D1, "(c) with the model, estimating a human in-bed pose from an input image.", [0056]; D1 teaches training a model to predict a pose, not an action, from an image. D2, "the training dataset is made up of real data and synthetic data resulting in a hybrid dataset. The training dataset is provided to the system or model and used to train the system or model", [0031]; D2 teaches supervised training on a real+synthetic hybrid dataset; together D1 and D2 teach training a model on labeled real and synthetic data to predict an action from an image; incorporating D2 into D1 would predict an action performed by the subject)
Regarding claims 2 and 17, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the machine learning model comprises one or more convolutional neural networks.
(D1, “The dataset can be used with a variety of machine learning or artificial intelligence (AI) systems, algorithms and techniques, such as, without limitation, convolution pose machines and deep neural networks”, [0144]; “It includes two convolution layers with kernel 1×1 and channel 256 and 32 followed by 3 fully connected layers”, [0268]; D2, “deep learning based models such as Mask-RCNN or DeepMask”, [0038]; employed and tested machine learning models can include convolutional neural networks)
Regarding claim 7, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, further comprising:
receiving a real-time image of a first subject on a first person support apparatus;
(D1, "(b) transmitting to the processor one or more images of a human lying in a bed from one or more imaging devices oriented toward the bed", [0045]; "Observation Interpretation at Run-Time", [0233]; "The computer system can run the software with instructions to periodically capture LWIR/depth images", [0141]; periodically capturing and transmitting run-time (real-time) images of a human lying in a bed to the processing system)
inputting the real-time image into the trained model; and
predicting an action of the first subject based on an output of the trained model.
(D1, "the processor is in communication with the one or more imaging devices to receive one or more images of a human lying in the bed and is operative to determine a pose of the human lying in the bed via the trained model.", [0048]; D1 teaches inputting the real-time image into the trained model and obtaining a predicted output. As established regarding claim 1, D2, "gesture recognition, navigation, interior mapping, medical application and imagery", [0019], teaches that such a trained-model framework is applied to predict an action; because the trained model referenced here is the same model already modified per the D1/D2 combination of claim 1 to be trained on action-labeled data, applying that model to the real-time image via D1's disclosed run-time mechanism would predict an action of the first subject rather than only a pose; incorporating D2 into D1 so that the run-time output reflects the action the model was trained to predict)
Regarding claim 8, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination further teaches the method of claim 7, further comprising displaying information about the action of the first subject.
(D1, "The computer system can run the software with instructions to periodically capture LWIR/depth images and display a skeleton pose figure to the user.", [0141]; D1 teaches displaying a visual representation of the trained model's output to the user. As established regarding claims 1 and 7, the D1/D2 combination modifies that output to be a predicted action; applying D1's display mechanism to that modified output would display information about the action of the first subject rather than only a pose; incorporating D2 into D1 so the displayed information reflects the predicted action)
Claim(s) 9-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 in view of D2 and further in view of D3.
Regarding claim 9, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination does not expressly disclose but D3 teaches the method of claim 7, further comprising causing a handheld device to display information about the action of the first subject.
(D3, "portable clinician devices 114, such as, for example, tablets, PDAs or the like, may be used by caregivers to access the required patient-specific information while at the patient's bedside.", [0088]; D3 teaches causing a handheld device to display patient-specific information. "In this manner, a lookup table of patient orientation change actions can be created.", [0148]; D3 expressly characterizes the patient's detected orientation changes as "actions", so the patient-specific information of [0088] displayed to caregivers is information about an action of the subject)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the remote monitoring and alerting features of D3 into the in-bed human pose/action estimation system of D1 in order to practically utilize the estimated pose or action data to track the duration a patient remains in a specific position or performs a specific action, trigger warnings for dangerous positions, actions, or prolonged immobility to prevent pressure ulcers, and efficiently communicate this status to caregivers by displaying predetermined representative images of the patient's position or action on remote and handheld computing devices. The combination of D1, D2 and D3 also teaches other enhanced capabilities.
Regarding claim 10, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination of D1, D2 and D3 teaches the method of claim 7, further comprising causing a remote computing device to display information about the action of the first subject.
(D3, "This representation can be displayed on the patient monitor 106 or transmitted to a nurses' station or other processing node to enable caregivers to monitor the patient's position in bed.", [0138]; D3 teaches transmitting the patient's positional representation to a remote computing device for display. "Reference is made to a look-up table of events which includes a set of profiles of orientation change actions or activities.", [0153]; D3 confirms the transmitted orientation representation encompasses detected orientation-change "actions", so the remote device displays information about the subject's action)
Regarding claim 11, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination of D1, D2 and D3 teaches the method of claim 7, further comprising displaying a predetermined image associated with the action of the first subject.
(D3, "the graphical icons are used to visually depict the detected orientation of the patient. In particular, the icons of FIGS. 13A-F show, in stick figure-type format, the patient sitting, standing, and lying in the supine position (on the back), the prone position (on the belly), on the left side, and on the right side, respectively.", [0196]; D3 teaches displaying predetermined graphical icons associated with the detected patient orientation. "In this manner, a lookup table of patient orientation change actions can be created.", [0148]; D3 classifies such orientation changes as "actions", so the predetermined icons of [0196] are predetermined images associated with the action of the first subject)
Regarding claim 12, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination of D1, D2 and D3 teaches the method of claim 7, further comprising causing a handheld device to display a predetermined image associated with the action of the first subject.
(D3, "portable clinician devices 114, such as, for example, tablets, PDAs or the like, may be used by caregivers to access the required patient-specific information while at the patient's bedside.", [0088]; "the graphical icons are used to visually depict the detected orientation of the patient.", [0196]; "In this manner, a lookup table of patient orientation change actions can be created.", [0148]; D3 teaches transmitting predetermined orientation icons, which D3 itself classifies as depicting patient "actions", to handheld devices such as tablets/PDAs; it would have been obvious to a person having ordinary skill in the art to display the predetermined images depicting the patient's action on the handheld devices explicitly taught by D3, to allow a clinician to visually ascertain the patient's status on their portable device)
Regarding claim 13, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination of D1, D2 and D3 teaches the method of claim 7, further comprising causing a remote computing device to display a predetermined image associated with the action of the first subject.
(D3, "This representation can be displayed on the patient monitor 106 or transmitted to a nurses' station or other processing node to enable caregivers to monitor the patient's position in bed.", [0138]; "the graphical icons are used to visually depict the detected orientation of the patient.", [0196]; "In this manner, a lookup table of patient orientation change actions can be created.", [0148]; D3 teaches transmitting the predetermined orientation icons, classified by D3 as patient "actions", and representations of the patient's position to remote computing devices (nurses' stations); it would have been obvious to a person having ordinary skill in the art to display the predetermined images on the remote computing device to allow remote caregivers to visually monitor the patient's specific action)
Regarding claim 15, the combination of D1 and D2 teaches its/their respective base claim(s).
The combination of D1, D2 and D3 teaches the method of claim 7, further comprising:
determining whether the first subject is in a dangerous position; and
(D3, "the patient monitor determines the mobility status of the patient, e.g., whether the patient is ambulatory, standing, sitting, reclining, or falling.", [0017]; determining whether the subject is in a dangerous or prohibited position, such as falling)
upon determination that the first subject is in the dangerous position, outputting a warning.
(D3, "In certain aspects, the wireless monitoring system can include an alert system to alert the caregiver that the patient is falling, getting out of bed, or otherwise moving in a prohibited manner or in a manner that requires caregiver attention.", [0017]; outputting a warning (alert) upon determining that the subject is in a dangerous or prohibited position)
Response to Arguments
Applicant's arguments filed on 9/3/2026 with respect to one or more of the pending claims have been fully considered but they are not persuasive.
Regarding independent claim(s) 1, 16, 20, and dependent claims 7-13, Applicant, in the remarks, argues that the combination of the cited reference(s) fails to teach the newly amended limitations in the claims.
The Examiner respectfully disagreed. The office action has been updated to address applicant’s argument. See the updated review comments for details.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center. for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 9/19/2026
