DETAILED ACTION
This is in response to the applicant’s reply filed June 11, 2026. In the applicant’s reply; Claim 11 was previously canceled by a preliminary amendment by applicant. Claims 1, 2, 4-5, 7, 12-13, and 15-16 were amended, and claims 9-10 are newly canceled.
Claims 1-8 and 12-16 are pending in this application.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Examiner’s Responses to Applicant’s Remark
Applicants' amendments filed on June 11, 2026 have been fully considered. The amendments overcome the following rejections set forth in the office action mailed on March 11, 2026.
Applicant’s amendments overcome the objection to the title of the specification, and the objection is hereby withdrawn.
Applicant’s amendments overcome the rejections of claims 1-10 and 12-26 under 35 U.S.C. 101 for being directed to non-statutory subject matter, and the rejection is hereby withdrawn.
Applicant’s amendments overcome the rejections of claims 1-10 and 12-16 under 35 U.S.C. 112 sixth paragraph, and the rejection is hereby withdrawn.
Applicant's arguments with respect to claims 1-8 and 12-16 have been considered but are moot in view of the new ground of rejection, presented below and necessitated by applicant’s amendments.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Japan on March 31, 2022. It is noted, however, that applicant has not filed a certified copy of the PCT/JP2022/016540 application as required by 37 CFR 1.55. For purposes of examination, the certified copy must be filed in order for the application to be awarded this priority date.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 12-13 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Sareckis et al. (US PGPub US 20240216757, hereby referred to as “Sareckis”).
Consider Claim 1, 12 and 13.
Sareckis teaches:
[Claim 1] (currently amended) A movement information generation apparatus comprising: / [Claim 12] (original) A movement information generation method comprising, by a computer: / [Claim 13] (currently amended) A non-transitory computer-readable storage medium storing a program for causing a computer to execute: perform operations comprising: (Sareckis: The present embodiment provides a biological feedback measurement system and method for capturing, analyzing, and presenting users (11) exercise data. Comprises video cameras (10a, 10b, 10c) surrounding the user (11), capturing the user's (11) movements, processing, and predicting 3D coordinates of human skeleton joints. Comprise a synchronization system and automatic calibration algorithm uses a distance metric to estimate the distance between joints. A shared hub (12) with a camera automatic calibration algorithm is provided to map multiple human skeletons into a single common world coordinate system, processing, identifying, and correcting joint position prediction failures from a sequence of skeleton joint positions detected in consecutive video frames and a vector of features from the changes in coordinate values for each joint. The system uses a machine learning classifier to identify the type of exercise and a fuzzy logic-based system to map subjective evaluations to the parameters measured, displays medical professional evaluations, provides feedback to the user (11).)
1. at least one memory configured to store instructions; and at least one processor configured to execute the instructions to preform operations comprising: / 12. generating first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; / 13. image processing of generating first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; and (Serackis: [0027] An example reference to FIG. 1 an application of a preferred embodiment of the invention, depicting a user 11 in a room. The user 11 is surrounded by video cameras devices 10 a, 10 b, 10 c and performs a group of exercises recommended by the medical professional. Each video camera 10 a, 10 b, 10 c runs an image processing process where a machine learning model is used to predict 3D coordinates for each of 17 connected human skeleton joints. The predicted joint coordinates are saved in coordinate system with origin point position, individually for each camera 10 a, 10 b, 10 c. The coordinates together with additional information for camera synchronization from each camera 10 a, 10 b, 10 c are transmitted into a common hub 12 (computer device) where a special iterative camera automatic calibration algorithm is applied. The camera automatic calibration algorithm maps three human skeletons, obtained and saved in different coordinate systems (origin points) into one common world coordinate system. Joint position prediction failures are identified by a process introduced in this invention and the required corrections are made.)
1. an image processing unit that generates first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; / 12. and determining a difference between the first movement information and comparison movement information indicating movement being a comparison target, / 13. output processing of determining a difference between the first movement information and comparison movement information indicating movement being a comparison target, and (Serackis: [0027] A fixed number of joints with lower level of dynamical changes of the coordinates are selected by analyzing features of joints, collected from all cameras 10 a, 10 b, 10 c. Joints of the same type (position in the skeleton) are selected for camera automatic calibration—estimation of camera position in world coordinate system. During the iterative process, one camera is selected as a reference position ant position of other cameras 10 a, 10 b, 10 c are changed at each iteration to minimize the pre-defined cost function for joint positioning. Distance between joints can be estimated using one of available and known distance metrics. According to the selected distance metric, the cost function is prepared. Distance minimization can be performed using one of available and known optimization algorithms. When the distance between selected group of joints is minimized, the camera positions are treated as calibrated. The positions of the remaining skeleton joints (usually those, which had the highest dynamical changes of the coordinates in a sequence of video frames) are recalculated by selecting most expected position in each time stamp (related to video frame). The coordinate of the joint is selected by applying a weighted sum of joint coordinates predicted in different camera frames, selecting the highest weight for the coordinate, that was predicted from the video frame of the camera, situated at higher angle to the joint motion plane (the highest angle is received when the motion plane is parallel to the camera plane). [0028] In flowchart FIG. 2 AI based 3D pose estimation results showing how 3D joint coordinates, predicted, and saved in individual camera coordinate systems are fused into single common coordinate system to perform autocalibration of the randomly placed camera views. The whole view of scope of the invention is shown in the flowchart FIG. 3 .)
1. and an output unit that determines a difference between the first movement information and comparison movement information indicating movement being a comparison target, and outputs output information being information relating to the difference. / 12. and outputting output information being information relating to the difference. / 13. outputting output information being information relating to the difference, (Serackis: [0029] The final human skeleton with fused and corrected positions of all 17 joints are presented to human motion analysis and parameter extraction block. Human motion analysis block calculates parameters, that are indicated by a medical professional as important to monitor during a particular exercise. Such parameters can be a speed of upper limb motion, maximum angle, displacement of joints that represent shoulder positions, etc. Therefore, the system should recognize the type of the exercise, that the patient performs. [0030] In specified flowchart FIG. 4 is shown a schematic representation of exercise type recognition. The exercise type detection is performed using one of available machine learning based classifiers. The input features, presented to the classifier, are estimated using numerical indication which skeleton joints were moving, estimated angle changes between joints during a single exercise phase, motion plane angle in respect to the human body (e.g., chest) plane, motion magnitude and other complementary features. The exercise type detection is performed from a group of exercises that were recommended for the patient by a medical professional. The list of recommended exercises is stored in a database as a list, related with a patient identification code.)
1/12/13. wherein the first image includes a plurality of still images, the operations further comprise generating skeleton information relating to the subject for each of the plurality of still images, the skeleton information including a line connecting parts to each other, and the parts including a head and joint points of a human, the first movement information is generated using the skeleton information, and the first movement information includes at least one of:a position of at least one joint point during the specific movement,a motion vector of at least one joint point during the specific movement, anda relative position of at least two joint points during the specific movement. (Serackis: [0016] …taking a sequence of skeleton joint positions detected in consecutive video frames and computing a vector of features from the changes in coordinate values for each joint; analyzing the features of the joints from all cameras to identify the joints with the lowest levels of dynamical changes in coordinates; selecting joints of the same type (position in the skeleton) for camera automatic calibration; estimating the camera position in the world coordinate system; using a cost function to minimize the distance between the joint group; assigning positions of the remaining joints using the camera frames that best capture the motions of those joints; providing the human motion analysis and parameter extraction block with all indicated human skeleton joints in the right, fused positions; calculating parameters that a medical specialist has identified and must be followed during the exercise; using a machine learning classifier to identify the type of exercise the patient is working on; using numerical data such as movement of the skeleton joints, changes in the angle between joints during a single exercise phase, motion plane angle relative to the human body plane, motion magnitude and other associated features as input features to the classifier; selecting a set of parameters to be monitored during the exercise; using a fuzzy logic-based system to map subjective evaluations to the parameters that is measured; displaying visual simulations of medical professional evaluations and comparing current joint motion, angle changes and other parameters of past values related to the patient; and displaying the monitored joints on a 3D mannequin model that follows the patient's movements, as well as measured angles. [0027] The automatic calibration process takes a sequence of skeleton joint positions detected in consecutive video frames, calculates a vector of features from coordinate value changes for each joint. As a feature set, the features that represents joint movement dynamics (such as variance, standard deviation, difference between maximum and minimum value, etc.) are selected. A fixed number of joints with lower level of dynamical changes of the coordinates are selected by analyzing features of joints, collected from all cameras 10 a, 10 b, 10 c. Joints of the same type (position in the skeleton) are selected for camera automatic calibration—estimation of camera position in world coordinate system. During the iterative process, one camera is selected as a reference position ant position of other cameras 10 a, 10 b, 10 c are changed at each iteration to minimize the pre-defined cost function for joint positioning. Distance between joints can be estimated using one of available and known distance metrics. According to the selected distance metric, the cost function is prepared. Distance minimization can be performed using one of available and known optimization algorithms. When the distance between selected group of joints is minimized, the camera positions are treated as calibrated. The positions of the remaining skeleton joints (usually those, which had the highest dynamical changes of the coordinates in a sequence of video frames) are recalculated by selecting most expected position in each time stamp (related to video frame). The coordinate of the joint is selected by applying a weighted sum of joint coordinates predicted in different camera frames, selecting the highest weight for the coordinate, that was predicted from the video frame of the camera, situated at higher angle to the joint motion plane (the highest angle is received when the motion plane is parallel to the camera plane).)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1-8 and 12-16 are rejected under 35 U.S.C. § 103 as being unpatentable over Chau Chong Ye (US PGPub US20180199861A1, hereby referred to as “Ye”) in view of Deng et al. (US PGPub US2020242805, hereby referred to as “Deng”).
Consider Claim 1, 12 and 13.
Ye teaches:
[Claim 1] (currently amended) A movement information generation apparatus comprising: / [Claim 12] (original) A movement information generation method comprising, by a computer: / [Claim 13] (currently amended) A non-transitory computer-readable storage medium storing a program for causing a computer to execute: perform operations comprising: (Ye: abstract An example method for assisting a patient to conduct a physical therapy session can include: presenting information about the physical therapy session on a display, the information including one or more actions to be performed by the patient; capturing one or more images of the patient as the patient performs the one or more actions; detecting, by a computing device, a safety risk related to one or more medical devices associated with the patient during the physical therapy session; and providing feedback regarding the safety risk. Other methods can include: detecting, by a computing device, using the one or more images, a level of effort from the patient during the physical therapy session; and modifying a future physical therapy session for the patient based upon the level of effort; or allowing the patient to compete with other patients during the physical therapy session based upon the level of effort. [0048]-[0053], Figure 6)
1. at least one memory configured to store instructions; and at least one processor configured to execute the instructions to preform operations comprising: / 12. generating first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; / 13. image processing of generating first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; and (Ye: [0048]-[0053], Figure 6; [0049] Referring now to FIG. 6, the computing device 106 (the server computer 122 can be similarly configured) includes at least one central processing unit (“CPU”) 602, a system memory 608, and a system bus 622 that couples the system memory 608 to the CPU 602. The system memory 608 includes a random access memory (“RAM”) 610 and a read-only memory (“ROM”) 612. A basic input/output system contains the basic routines that help to transfer information between elements within the computing device 106, such as during startup, is stored in the ROM 612. The computing device 106 further includes a mass storage device 614. The mass storage device 614 is able to store software instructions and data.)
1. an image processing unit that generates first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; / 12. and determining a difference between the first movement information and comparison movement information indicating movement being a comparison target, / 13. output processing of determining a difference between the first movement information and comparison movement information indicating movement being a comparison target, and (Ye: [0017]-[0030], Figures 1-3, [0017] Referring now to FIGS. 1-2, an example system 100 is shown. In the system 100, a patient 110 is located in an environment 102, such as a hospital room, clinical setting, or bedroom. The environment 102 also includes one or more cameras 104 and a computing device 106. [0023] In this example, the camera 104 is an infrared camera configured to capture infrared images and/or video of the patient 110 on the patient support device 202. The camera 104 includes an infrared laser and a detector, such as a CMOS sensor, that captures three-dimensional imagery of the patient 110 and the surroundings (such as the medical devices surrounding and coupled to the patient). The images and/or video that are captured by the camera 104 can be processed locally or remotely, as described further below, to detect the actions of the patient 110. [0024] For example, as depicted, the camera 104 is programmed to transmit the captured infrared imagery to the computing device 106 and/or directly to a server computer 122 through a network 120. The server computer 122 can be a central server that is programmed to process the imagery and/or allow the caregivers 212 to monitor the patient 110)
1. and an output unit that determines a difference between the first movement information and comparison movement information indicating movement being a comparison target, and outputs output information being information relating to the difference. / 12. and outputting output information being information relating to the difference. / 13. outputting output information being information relating to the difference. (Ye: [0031]-[0036], [0032] Next, at operation 304, the patient's actions are captured as images and/or video as the patient performs the physical therapy session. As described above, this can be done using one or more cameras that are used to capture the movement of the patient as the patient performs the actions during the physical therapy session. [0033] At operation 306, the images and/or video are analyzed. As described above, this can include an automated analysis that determines such aspects as patient progress, compliance, and/or risk. Additional details on the aspects of this assessment are provided in FIG. 4 and described below. [0034] Finally, at operation 308, a follow-up plan is developed based upon the analysis of the patient's action and feedback. This plan can include further physical therapy sessions and/or additional feedback to the patient. The plan can be developed automatically by the computing device as the patient's actions are analyzed. Or, suggested feedback can be developed and presented to a caregiver for review. [0052] According to various embodiments, the computing device 106 may operate in a networked environment using logical connections to remote network devices through the network 120, such as a wireless network, the Internet, or another type of network. The computing device 106 may connect to the network 120 through a network interface unit 604 connected to the system bus 622. It should be appreciated that the network interface unit 604 may also be utilized to connect to other types of networks and remote computing systems. The computing device 106 also includes an input/output controller 606 for receiving and processing input from a number of other devices, including a touch user interface display screen, or another type of input device. Similarly, the input/output controller 606 may provide output to a touch user interface display screen or other type of output device.)
-; 1/12/13. wherein the first movement information includes at least one of the following - a position of at least one joint point during specific movement,- a motion vector of at least one joint point during specific movement, and - a relative position of at least two joint points during specific movement. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients.)
Even if Ye does not specifically teach:
wherein the first image includes a plurality of still images, the operations further comprise generating skeleton information relating to the subject for each of the plurality of still images, the skeleton information including a line connecting parts to each other, and the parts including a head and joint points of a human, the first movement information is generated using the skeleton information, and the first movement information includes at least one of:a position of at least one joint point during the specific movement,a motion vector of at least one joint point during the specific movement, anda relative position of at least two joint points during the specific movement.
Deng teaches:
[Claim 1] (currently amended) A movement information generation apparatus comprising: / [Claim 12] (original) A movement information generation method comprising, by a computer: / [Claim 13] (currently amended) A non-transitory computer-readable storage medium storing a program for causing a computer to execute: perform operations comprising: (Deng: abstract, Examples are disclosed herein that relate to automatically calibrating cameras based on human detection. One example provides a computing system comprising instructions executable to receive image data comprising depth image data and two-dimensional image data of a space from a camera, detect a person in the space via the image data, determine a skeletal representation for the person via the image data, determine over a period of time a plurality of locations at which a reference point of the skeletal representation is on a ground area in the image data, determine a ground plane of the three-dimensional representation based upon the plurality of locations at which the reference point of the skeletal representation is on the ground area in the image data, and track a location of an object within the space relative to the ground plane. [0045]-[0050], Figure 11)
1. at least one memory configured to store instructions; and at least one processor configured to execute the instructions to preform operations comprising: / 12. generating first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; / 13. image processing of generating first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; and (Deng: [0045]-[0050], Figure 11, [0045] FIG. 11 schematically shows a non-limiting embodiment of a computing system 1100 that can enact one or more of the methods and processes described above. Computing system 1100 is shown in simplified form. Computing system 1100 may take the form of one or more personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices. Computing system 1100 may be representative of a multi-camera system, a server computing system within the camera network, and/or a camera comprising integrated processing. [0046] Computing system 1100 includes a logic subsystem 1102 and a storage subsystem 1104. Computing system 1100 may optionally include a display subsystem 1106, input subsystem 1108, communication subsystem 1110, and/or other components not shown in FIG. 11. [0050] Storage subsystem 1104 may include removable and/or built-in devices. Storage subsystem 1104 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage subsystem 1104 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices. [0003]-[0004] [0003] Another example provides a computing system comprising instructions executable to receive a first image from a first camera and a second image from a second camera having an overlapping field of view with the first camera, detect a skeleton location of a person in the first image and detect the skeleton location of the person in the second image in a region of each image at which fields of view of the first camera and the second camera overlap, determine a geometric relationship between the cameras of the camera pair based upon comparing coordinates of the skeleton location of the person in the first camera and coordinates of the skeleton location of the person in the second camera, determine a camera connection graph based on the geometric relationships between the cameras of the camera pairs, determine a primary camera from the camera connection graph, determine coordinate mappings defining a spatial relationship between the primary camera and each of one or more other cameras of the plurality of cameras, and track an object that moves between fields of view of two or more cameras of the plurality of cameras based upon the coordinate mappings)
1. an image processing unit that generates first movement information indicating movement of a subject at first timing by processing a first image acquired by capturing the subject at the first timing; / 12. and determining a difference between the first movement information and comparison movement information indicating movement being a comparison target, / 13. output processing of determining a difference between the first movement information and comparison movement information indicating movement being a comparison target, and (Deng: [0004] Another example provides a method of tracking an object viewable by a pair of stationary cameras having an overlapping field of view, the method comprising receiving, from a first camera of a camera pair, first image data comprising first depth image data and first two-dimensional image data, receiving, from a second camera of the camera pair, second image data comprising second depth image data and second two-dimensional image data, detecting a person in the first image data and detecting the person in the second image data in a spatial region at which fields of view of the first camera and the second camera overlap, comparing a first skeletal representation of the person from the first depth data to a second skeletal representation of the person from the second depth data to determine a correspondence between the first skeletal representation and the second skeletal representation, based upon the correspondence from skeleton mapping, detecting visual features in the first two-dimensional image data and the visual features in the second two-dimensional image data, and building one or more additional correspondences using the visual features, determining coordinate mappings that relate a coordinate system of the first camera and a coordinate system of the second camera by using the new correspondence from visual features, and tracking an object that moves between fields of view of the first camera and the second camera based upon the coordinate mappings. This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.)
1. and an output unit that determines a difference between the first movement information and comparison movement information indicating movement being a comparison target, and outputs output information being information relating to the difference. / 12. and outputting output information being information relating to the difference. / 13. outputting output information being information relating to the difference, (Deng: [0020]-[0022], 0020]
In the example of FIG. 1, person 104 is within the fields of view of cameras 102 a, 102 b, and 102 c. Each of these cameras may include depth sensing technology configured to output a skeletal representation of the person as determined from acquired depth images, in which a person is represented by a model comprising a collection of nodes that represent locations of the human body and that are connected in a form that approximates the form of the human body. As another example, a skeletal representation of the person may first be determined from two-dimensional image data, where points of the skeletal representation in the two-dimensional image data may then be mapped to three-dimensional (3D) points using depth image information. [0021] FIGS. 3A-C shows example skeletal representations 300 a, 300 b, and 300 c of the person 104 in FIG. 1 as determined from image data acquired by each of cameras 102 a, 102 b and 102 c, respectively. Such skeletal representations may be computed onboard each camera and provided as camera output, or may be determined by another computing device (e.g. a camera system server) that receives image data from the cameras. The skeletal representations may be used for various calibration tasks. For example, as mentioned above, the skeletal representations may be used to determine a location of a ground plane in an area imaged by the cameras. In a camera system comprising depth image sensors, the depth image sensors may be used to form a depth map of an imaged area. Such a depth map may take the form of a surface reconstruction mesh computed based upon depth values at each pixel in a depth image. Various locations in the surface reconstruction mesh, such as a ground plane, may be identified, semantically labeled (e.g. describing the surface as a wall, floor/ground, table, etc.) and represented as a smooth plane in place of the mesh. However, determining which surfaces in the surface reconstruction mesh correspond to the ground plane may pose challenges, as the manual identification of a ground plane may require the labeling of a relatively large number of ground points in the depth data, which may be time-consuming. Where an insufficient number of ground points are identified, other surfaces and objects in the area may be mistaken for the ground.)
wherein the first image includes a plurality of still images, the operations further comprise generating skeleton information relating to the subject for each of the plurality of still images, the skeleton information including a line connecting parts to each other, and the parts including a head and joint points of a human, the first movement information is generated using the skeleton information, and the first movement information includes at least one of:a position of at least one joint point during the specific movement, a motion vector of at least one joint point during the specific movement, and a relative position of at least two joint points during the specific movement. (Deng: [0022] Thus, a skeletal representation of a human detected in the image data may be tracked as it moves through the imaged area to identify locations at which a reference point of the skeletal representation is determined to be on a ground area over time to thereby identify ground points. The reference point may be a relatively low node of the skeletal representation (e.g. a node that is close to the physical floor or ground). Depending upon the skeletal model being used, examples of suitable reference points may include an ankle or foot node, a midpoint between left and right foot nodes, or a midpoint between left and right ankle nodes. The locations of the reference point of the skeletal representation over time may then be considered as ground points. These ground points may be determined from a skeletal representation obtained from either two-dimensional image data or depth image data. [0023] FIG. 4 shows a diagram of an example set of ground points 402 in which each ground point represents a location of a reference point of a skeletal representation 404 over time as the corresponding person walked through the area. Upon detecting a sufficient number of ground points in the image data, a convex hull may be fit to the ground points in the image data, for example, using an algorithm such as the quickhull algorithm. Points within the convex hull, which include the ground points and other points, are then mapped to a 3D representation of the area as determined from depth image data. The result of mapping the ground points to the 3D representation is a point cloud of 3D points, and this point cloud may then be used to determine the ground plane The use of all points within the convex hull in the image form a bigger point set than the set of ground points alone, and their corresponding three-dimensional representations form a bigger point cloud, which may make the ground plane fitting more accurate. FIG. 5 shows an example convex hull 500 fit to the point cloud comprising the set of ground points 402. The convex hull 500 determined and the resulting point cloud of 3D points within the convex hull may be used to estimate a location of the ground plane with respect to the cameras. Thus, by observing the movement of a human within the area and the interactions of an ankle/foot/other reference node of the human with surfaces in the area as determined from depth images, the ground plane may be more easily determined compared to other ground plane detection methods. [0024] Human detection may also be used to determine the spatial relationships among cameras in the camera system 102. Referring briefly back to FIGS. 3A-3C, person 104 is in the fields-of- view 302, 304 and 306 of cameras 102 a, 102 b and 102 c respectively while standing in the location shown in FIG. 1, as these cameras have at least partially overlapping fields of view. Based on imaging the person at the same location from different perspectives, a camera coordinate transformation matrix may be determined for each pair of cameras in FIG. 1 (e.g. cameras 102 a/102 b, 102 a/102 c, and 102 b/102 c). By performing similar imaging and computations for other camera pairs (not shown) in the area with overlapping fields of view, spatial relationships between multiple cameras may be calibrated.)
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the computerized method and system for an interactive physical therapy session of Ye with the camera calibration architecture of Deng in generating a skeletal representation of a person of interest in image analysis. The determination of obviousness is predicated upon the following findings: both references are in the same field of endeavor of image analysis and the detection and tracking of a user in a physical capacity and setting, and One skilled in the art would have been motivated to modify the computerized physical therapy session of Ye in order to improve the overall person tracking and present a more interactive end-user model using skeletal representations. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Ye, while the teaching of Deng continues to output a skeletal representation of an end-user and continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of presenting a more accurate skeletal representation in the interactive end-user interface. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Consider Claim 2.
The combination of Ye and Deng teaches:
[Claim 2] (original) The movement information generation apparatus according to claim 1, wherein the comparison movement information is generated by processing a second image acquired by capturing the subject at second timing prior to the first timing. (Ye: [0031]-[0036], Figure 3 [0032] Next, at operation 304, the patient's actions are captured as images and/or video as the patient performs the physical therapy session. As described above, this can be done using one or more cameras that are used to capture the movement of the patient as the patient performs the actions during the physical therapy session. [0033] At operation 306, the images and/or video are analyzed. As described above, this can include an automated analysis that determines such aspects as patient progress, compliance, and/or risk. Additional details on the aspects of this assessment are provided in FIG. 4 and described below. [0034] Finally, at operation 308, a follow-up plan is developed based upon the analysis of the patient's action and feedback. This plan can include further physical therapy sessions and/or additional feedback to the patient. The plan can be developed automatically by the computing device as the patient's actions are analyzed. Or, suggested feedback can be developed and presented to a caregiver for review. [0052] According to various embodiments, the computing device 106 may operate in a networked environment using logical connections to remote network devices through the network 120, such as a wireless network, the Internet, or another type of network. The computing device 106 may connect to the network 120 through a network interface unit 604 connected to the system bus 622. It should be appreciated that the network interface unit 604 may also be utilized to connect to other types of networks and remote computing systems. The computing device 106 also includes an input/output controller 606 for receiving and processing input from a number of other devices, including a touch user interface display screen, or another type of input device. Similarly, the input/output controller 606 may provide output to a touch user interface display screen or other type of output device. [0036]-[0049], Figure 4. Deng: [0023] FIG. 4 shows a diagram of an example set of ground points 402 in which each ground point represents a location of a reference point of a skeletal representation 404 over time as the corresponding person walked through the area. Upon detecting a sufficient number of ground points in the image data, a convex hull may be fit to the ground points in the image data, for example, using an algorithm such as the quickhull algorithm. Points within the convex hull, which include the ground points and other points, are then mapped to a 3D representation of the area as determined from depth image data. The result of mapping the ground points to the 3D representation is a point cloud of 3D points, and this point cloud may then be used to determine the ground plane The use of all points within the convex hull in the image form a bigger point set than the set of ground points alone, and their corresponding three-dimensional representations form a bigger point cloud, which may make the ground plane fitting more accurate. FIG. 5 shows an example convex hull 500 fit to the point cloud comprising the set of ground points 402. The convex hull 500 determined and the resulting point cloud of 3D points within the convex hull may be used to estimate a location of the ground plane with respect to the cameras.)
Consider Claim 3.
The combination of Ye and Deng teaches:
[Claim 3] (original) The movement information generation apparatus according to claim 1, wherein the comparison movement information is generated by processing a third image acquired by capturing a reference person being a person different from the subject. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients.)
Consider Claim 4.
The combination of Ye and Deng teaches:
[Claim 4] (currently amended)The movement information generation apparatus according to claim 2, wherein the operations further comprise: generates generating the comparison movement information by processing the second image. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients. Deng: [0023] FIG. 4 shows a diagram of an example set of ground points 402 in which each ground point represents a location of a reference point of a skeletal representation 404 over time as the corresponding person walked through the area. Upon detecting a sufficient number of ground points in the image data, a convex hull may be fit to the ground points in the image data, for example, using an algorithm such as the quickhull algorithm. Points within the convex hull, which include the ground points and other points, are then mapped to a 3D representation of the area as determined from depth image data. The result of mapping the ground points to the 3D representation is a point cloud of 3D points, and this point cloud may then be used to determine the ground plane The use of all points within the convex hull in the image form a bigger point set than the set of ground points alone, and their corresponding three-dimensional representations form a bigger point cloud, which may make the ground plane fitting more accurate. FIG. 5 shows an example convex hull 500 fit to the point cloud comprising the set of ground points 402. The convex hull 500 determined and the resulting point cloud of 3D points within the convex hull may be used to estimate a location of the ground plane with respect to the cameras.)
Consider Claim 5.
The combination of Ye and Deng teaches:
[Claim 5] (currently amended) The movement information generation apparatus according to claim 1,wherein the first timing is timing after the subject performs rehabilitation training, and the output information includes a recommended movement (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients.)
Consider Claim 6.
Ye teaches:
[Claim 6] (original) The movement information generation apparatus according to claim 5, wherein the difference is generated for each of a plurality of parts of a body, and the output information includes the recommended movement for each of the plurality of parts. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients.)
Consider Claim 7.
The combination of Ye and Deng teaches:
[Claim 7] (currently amended) The movement information generation apparatus according to claim 5 further wherein the operations further comprise: acquiring stage information indicating a stage of rehabilitation training of the subject, wherein the output unit decides deciding the recommended movement by using the stage information. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients. Deng: [0023] FIG. 4 shows a diagram of an example set of ground points 402 in which each ground point represents a location of a reference point of a skeletal representation 404 over time as the corresponding person walked through the area. Upon detecting a sufficient number of ground points in the image data, a convex hull may be fit to the ground points in the image data, for example, using an algorithm such as the quickhull algorithm. Points within the convex hull, which include the ground points and other points, are then mapped to a 3D representation of the area as determined from depth image data. The result of mapping the ground points to the 3D representation is a point cloud of 3D points, and this point cloud may then be used to determine the ground plane The use of all points within the convex hull in the image form a bigger point set than the set of ground points alone, and their corresponding three-dimensional representations form a bigger point cloud, which may make the ground plane fitting more accurate. FIG. 5 shows an example convex hull 500 fit to the point cloud comprising the set of ground points 402. The convex hull 500 determined and the resulting point cloud of 3D points within the convex hull may be used to estimate a location of the ground plane with respect to the cameras.)
Consider Claim 8.
Ye teaches:
[Claim 8] (currently amended) The movement information generation apparatus according to claim 1,wherein the output information includes the first movement information and the comparison movement information. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients.)
Consider Claim 9.
Ye teaches:
[Claim 9] (currently amended) The movement information generation apparatus according to claim 1,wherein the first image includes a plurality of still images, and the image processing unit generates generating skeleton information relating to the subject for each of the plurality of still images, and generates the first movement information by using the skeleton information. (Ye: [0023] In this example, the camera 104 is an infrared camera configured to capture infrared images and/or video of the patient 110 on the patient support device 202. The camera 104 includes an infrared laser and a detector, such as a CMOS sensor, that captures three-dimensional imagery of the patient 110 and the surroundings (such as the medical devices surrounding and coupled to the patient). The images and/or video that are captured by the camera 104 can be processed locally or remotely, as described further below, to detect the actions of the patient 110. [0026] The imagery can, for example, be analyzed to determine whether or not the patient is performing the proper actions using the proper form. As described further below, the system 100 can be programmed to make recommendations and/or modifications to the physical therapy when the system determines that the patient is proceeding at a slower or faster rate of recovery. [0027] Further, the imagery can be used to determine compliance, such as determining if the patient performed the actions as required. For example, the patient may be required to perform a certain number of repetitions for each exercise during a session, and the system 100 can be programmed to determine whether or not the proper actions were performed. [0028] In yet other examples, the system 100 is programmed to determine risk associated with the physical therapy. This risk can take various forms. For example, the system can be programmed to determine how the patient's actions might impact the patient's health status and/or surroundings, such as medical devices in the vicinity of and/or coupled to the patient. The system 100 can provide feedback to the patient 110 and/or the caregivers 212, as described further below, if a safety concern is identified, such as actions performed by the patient that might displace or otherwise impact medical devices or other therapy being provided to the patient 110.)
Claim 9. (Canceled)
Claim 10. (Canceled)
Claim 11 (Canceled)
Consider Claim 14.
The combination of Ye and Deng teaches:
[Claim 14] (new) The movement information generation apparatus according to claim 5, wherein the recommended movement is optimized to the subject. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients.)
Consider Claim 15.
The combination of Ye and Deng teaches:
[Claim 15] (new) The movement information generation apparatus according to claim 1, wherein the operations further comprise: outputting a rehabilitation progress degree computed by a machine learning model. (Ye: [0031]-[0036], Figure 3, [0036] Referring now to FIG. 4, additional details regarding the analysis operation of operation 306 of the method 300 are shown. [0037]-[0041], Figure 4, [0037] At operation 402, the images and/or video of the patient performing the actions during the physical therapy session are analyzed. This can, for example, be accomplished by using pattern matching that compares the patient's movements to expected movements for the therapy. For example, if the patient is instructed to move her arm from a level horizontal position to an upright vertical position, the images and/or video can be analyzed to confirm that the patient performed the correct action(s) and the proper number of repetitions. [0038] At operation 404, any actions by the patient that fall outside of the requirements for the physical therapy session are identified. Continuing with the example above, if the patient fails to move her arm to a full vertical position with each repetition, this failure can be identified. Or, if the patient only performs 5 repetitions when 10 repetitions were requested, this failure is identified. [0039] Next, at operation 406, risks associated with the patient's actions during the physical therapy session are identified. These risks can include such issues as the patient's actions, such as over-extending or otherwise putting the patient at risk. The risk can also relate to the patient's medical therapies. For example, if the patient has a central line and is performing actions during physical therapy that might result in the dislodgement of the central line, the system can identify that risk. [0040] Finally, at operation 408, the feedback is provided to the patient and/or caregiver. The feedback can be anything from additional directions for the patient to perform actions in a different manner, selection of different physical therapy sessions, and/or warnings to the patient regarding risky actions that might impact medical therapies that are being provided to the patient (e.g., “Do not swing your arms across your ventilator tubes because the motion could result in dislodgement.”). [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients.)
Consider Claim 16.
The combination of Ye and Deng teaches:
[Claim 16] (new) The movement information generation apparatus according to claim 1, wherein the subject is an injured or a diseased individual. (Ye: [0041] Referring now to FIG. 5, an example method 500 is shown for gamifying the physical therapy process to encourage compliance by the patient. In this example, a social media platform is created to allow the patient to connect with other patients. [0042] At operation 502, information associated with the patient is accessed. This can be bibliographic information such as name, address, age, gender, injury state, disease state, etc. The privacy of the patient can be preserved by anonymizing the information and/or allowing the patient to control what, if any, information is shared. [0043] Next, at operation 502, the bibliographic information is used to match the patient with other similar patients. This can be accomplished in many ways similar to that done on other social media platforms. For example, if the patient suffers from congestive heart failure as a disease state, the system may match the patient with other congestive heart failure patients of a similar age or in a similar geography.)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAHMINA N ANSARI whose telephone number is (571)270-3379. The examiner can normally be reached on IFP Flex - Monday through Friday 9 to 5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O' NEAL MISTRY can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
TAHMINA N. ANSARI
Examiner
Art Unit 2672
2672
July 27, 2026
/TAHMINA N ANSARI/Primary Examiner, Art Unit 2674
Prosecution Insights