DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of JP2021-190692, filed in Japan on 11/25/2021.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 5/8/2024 and 4/10/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Claim Objections
Claims 1, 3, 5, 6, and 9-12 objected to because of the following informalities: The verbiage “taking exercise” is unclear. Appropriate correction is required. Examiner suggests changing to “performing exercise.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-16 are rejected under 35 U.S.C. 103 as being unpatentable over Tzvieli (U.S. Patent Pub. No. 2019/0046044) in view of Lowery (U.S. Patent Pub. No. 2023/0241453).
Regarding Claim 1, Tzvieli teaches an information processing apparatus comprising:
processing circuitry configured to: obtain a user video in which a user taking exercise is seen (Fig. 35a, Fig.35b; ¶246 FIG. 35a illustrates an embodiment of a system configured to estimate an aerobic activity parameter 688. The system includes at least one CAM (Camera as taught in Abstract) that is used to measure TH.sub.RBN 683 and a computer 686. Some embodiments of the system may optionally include additional elements, such as the frame 680, a head-mounted inward-facing video camera 682)
make, based on the user video, an estimation about an exercise load of the user with respect to an exercise tolerance of the user; and (¶247 The computer 686 is configured, in one embodiment, to calculate, based on TH.sub.RBN (taken by the at least one CAM), the aerobic activity parameter 688. Optionally, the aerobic activity parameter 688 is indicative of one or more of the following values: oxygen consumption (VO.sub.2), maximal oxygen consumption (VO.sub.2 max), and energy expenditure (EE). Optionally, the computer 686 may utilize additional inputs to calculate the aerobic activity parameter such as measurements of the heart rate (HR) of the user, values of the activity level of the user, and/or various statistics about the user (e.g., age, weight, height, gender, etc.); ¶266 user interface 689 may be used to alert the user responsive to an indication that the aerobic activity parameter has fallen below a threshold (e.g., when the rate of energy expenditure falls below a threshold) or when the aerobic activity parameter reaches a certain threshold (e.g., when the total energy expenditure during a session reaches a certain caloric goal).)
present information based on a result of the estimation about the exercise load of the user (¶266 A user interface 689 may be utilized to present the aerobic activity parameter 688 and/or present an alert related to the aerobic activity parameter 688.)
Tzvieli suggests but does not explicitly disclose make, based on the user video, an estimation about an exercise load of the user with respect to an exercise tolerance of the user.
Lowery is in the same field of art of image analysis. Further, Lowery teaches make, based on the user video, an estimation about an exercise load of the user with respect to an exercise tolerance of the user (Lowery, ¶12 display a score, based on the positions at a single point in time, of at least one of the first, second, third, and fourth body parts respectively, the velocity value, and the location value; and assign a level of difficulty above or below the threshold level of difficulty for the training movement instruction based on the score.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tzvieli by determining exercise load of the user that is taught by Lowery; thus, one of ordinary skilled in the art would be motivated to combine the references to provide instant feedback to improve body movements of the individual to increase the rate of neuroplasticity through enhanced replication of the exercise directions (Lowery ¶2).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, wherein the processing circuitry makes the
estimation about the exercise load of the user by applying an estimation model to input data based on the user video (Tzvieli, ¶251 The computer 686 may utilize various approaches in order to estimate aerobic activity parameters based on data that includes TH.sub.RBN 683 and/or values derived from TH.sub.RBN. In one embodiment, the computer 686 generates feature values based on data comprising TH.sub.RBN, and utilizes a model 687 to calculate the aerobic activity parameter 688 based on the feature values. Optionally, the model 687 is trained based on data indicative of aerobic activity of multiple users (e.g., data that includes physiological signals such as respiratory rate, heart rate, etc., of the multiple users). Additionally or alternatively, the model 687 is trained based on data that includes previous TH.sub.RBN of the multiple users and values of the aerobic activity parameter of the multiple users corresponding to when the previous TH.sub.RBN were taken.)
Regarding Claim 3, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 2, wherein the estimation model corresponds to a trained model or corresponds to a fine-tuned model or a distilled model of the trained model, the trained model being created by supervised learning using a training data set including items of input data and items of labeled data, the items of input data each including data on a subject video in which a subject taking exercise is seen, the items of labeled data being associated with the items of input data (Tzvieli, ¶251 Optionally, the model 687 is trained based on data indicative of aerobic activity of multiple users (e.g., data that includes physiological signals such as respiratory rate, heart rate, etc., of the multiple users). Additionally or alternatively, the model 687 is trained based on data that includes previous TH.sub.RBN of the multiple users and values of the aerobic activity parameter of the multiple users corresponding to when the previous TH.sub.RBN were taken. For example, the training data includes samples, each sample comprising: (i) feature values were generated from certain pervious TH.sub.RBN of a certain user taken during certain period of time, and (ii) a label generated based on a measurement of the value of the aerobic activity parameter of the certain user during the certain period of time (i.e., the value of VO.sub.2, VO.sub.2 max, or EE, as measured during the certain period of time).)
Regarding Claim 4, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 3, wherein the subject is a person identical to the user (Tzvieli, ¶259 The model 687 is trained on data that includes previous TH.sub.RBN of the user and/or other users. Training the model 687 typically involves generating samples based on the previous TH.sub.RBN and corresponding labels indicative of values of the aerobic activity parameter when the previous TH.sub.RBN were taken.)
Regarding Claim 5, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 2, wherein the items of input data each include user data on a physical condition of the user taking exercise (Tzvieli, ¶251 The computer 686 may utilize various approaches in order to estimate aerobic activity parameters based on data that includes TH.sub.RBN 683 and/or values derived from TH.sub.RBN. wherein TH.sub.RBN are indicative of an exhale stream of the user)
Regarding Claim 6, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 5, wherein the user data includes data on at least one of a skeleton, a facial expression, a skin color, respiration, and a heart rate of the user taking exercise (Tzvieli, ¶251 The computer 686 may utilize various approaches in order to estimate aerobic activity parameters based on data that includes TH.sub.RBN 683 and/or values derived from TH.sub.RBN. wherein TH.sub.RBN are indicative of an exhale stream of the user,)
Regarding Claim 7, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, further causing the processing circuitry to:
obtain data on a cardiopulmonary condition of the user; and (Tzvieli, ¶263 In one embodiment, the computer 686 calculates, based on TH.sub.RBN 683, n≥1 values x.sub.1 . . . x.sub.n, of observations of a parameter related to respiration such as the respiration rate, change to respiration rate, respiration volume, change to respiration volume, and the like)
analyze a relationship between the exercise load of the user and the cardiopulmonary condition of the user, wherein (Tzvieli, ¶263 For example, x.sub.i may be the increase to the respiration rate observed after moderate running for a period (e.g., five minutes). In another example, x.sub.i may be the change to respiration volume and/or average respiration volume during a half hour of cycling)
the processing circuitry presents information based on a result of analyzing the relationship (Tzvieli, ¶266 A user interface 689 may be utilized to present the aerobic activity parameter 688 and/or present an alert related to the aerobic activity parameter 688.)
Regarding Claim 8, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, further causing
the processing circuitry to analyze a change over time in the exercise load of the user, wherein (Tzvieli, ¶263 For example, x.sub.i may be the increase to the respiration rate observed after moderate running for a period (e.g., five minutes). In another example, x.sub.i may be the change to respiration volume and/or average respiration volume during a half hour of cycling)
the processing circuitry presents information based on a result of analyzing the change over time in the exercise load of the user (Tzvieli, ¶266 A user interface 689 may be utilized to present the aerobic activity parameter 688 and/or present an alert related to the aerobic activity parameter 688.)
Regarding Claim 9, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, further causing the processing circuitry to:
make an estimation about a sign of heart failure of the user based on at least one of the user video and user voice that is obtained by recording voice of the user taking exercise; and (Tzvieli, ¶531 In still another example, the physiological response, which is detected based (facial skin color changes) FSCC recognizable in IM.sub.ROI, is heart rate and/or breathing rate; ¶560 U.S. Pat. No. 8,768,438, titled “Determining cardiac arrhythmia from a video of a subject being monitored for cardiac function”, describes how a heart rate may be determined based on FSCC, which are represented in a PPG signal obtained from video of the user…Peak-to-peak pulse points are detected in the PPG signal, which may be analyzed to determine parameters such as heart rate, heart rate variability, and/or to obtain peak-to-peak pulse dynamics that can be indicative of conditions such as cardiac arrhythmia.)
present information based on a result of the estimation about the sign of heart failure of the user (Tzvieli, ¶118 A user interface (UI) may be utilized, in some embodiments, to notify the user and/or some other entity, such as a caregiver, about the physiological response and/or present an alert responsive to an indication that the extent of the physiological response reaches a threshold.)
Regarding Claim 10, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, further causing the processing circuitry to:
make an estimation about a mental condition of the user based on at least one of the user video and user voice that is obtained by recording voice of the user taking exercise; and (Tzvieli, ¶608 patterns and/or various extractable features from one user's thermal and/or FSCC data may not be easily transferable to another user, or even to the same user under different physiological and/or mental conditions…Personalized models can overcome some of the possible disadvantages of using normed physiological statistics, which paves the way for personalized training, detection, and therapies, which are able to account for arbitrary user-defined physiological and/or mental states corresponding to a wide variety of individual needs. Leveraging machine learning algorithms can enable assignment of arbitrary user-defined physiological and/or mental states to recorded thermal and/or FSCC data during day-to-day activities)
present information based on a result of the estimation about the mental condition of the user (Tzvieli, ¶118 A user interface (UI) may be utilized, in some embodiments, to notify the user and/or some other entity, such as a caregiver, about the physiological response and/or present an alert responsive to an indication that the extent of the physiological response reaches a threshold.)
Regarding Claim 11, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, wherein
the processing circuitry makes estimations about exercise loads of a plurality of users taking exercise, and (Tzvieli, ¶286 performance of one or more users may be monitored while they breathe in various patterns while performing a certain sequence of movements)
presents information based on a result of the estimation about the exercise load of the user presents information based on a result of the estimations about the exercise loads of the plurality of users to an instructor of the plurality of users (Tzvieli, ¶118 A user interface (UI) may be utilized, in some embodiments, to notify the user and/or some other entity, such as a caregiver, about the physiological response and/or present an alert responsive to an indication that the extent of the physiological response reaches a threshold.)
Regarding Claim 12, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, wherein
the processing circuitry makes estimations about exercise loads of a plurality of users taking exercise, and presents information on a user of which a result of an estimation about an exercise load satisfies a predetermined condition (Tzvieli, ¶286 performance of one or more users may be monitored while they breathe in various patterns while performing a certain sequence of movements, and the optimal breathing pattern (i.e., the breathing pattern that is synchronized with the certain sequence) may be determined based on detecting a breathing pattern for which the performance is maximized (e.g., farthest/most accurate driver hit)) from among the plurality of users, to an instructor of the user (Tzvieli, ¶118 A user interface (UI) may be utilized, in some embodiments, to notify the user and/or some other entity, such as a caregiver, about the physiological response and/or present an alert responsive to an indication that the extent of the physiological response reaches a threshold.)
Regarding Claim 13, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, wherein
the processing circuitry obtains a user video (Lowery, ¶49 The camera(s) 1222 can take several measurements per second of the body positions and can transfer its raw position and timing data to the computing system 1208) in which a user playing a video game is seen, the video game being a video game of which game progress is controlled in accordance with a body movement of a player (Lowery, ¶5 the system includes an exercise therapy video game and monitor system for physical targeted activation and cognitive skills improvement for learning fluency a user comprising: interactive exercise software designed to provide visual movement instructions to the user via a first gaming platform having a screen; a motion sensing device configured to track the body position and movements of the user and create and output raw data corresponding to the user's body positions and movements when the user is replicating the movements of the visual movements instructions; and a computing system networked to the interactive exercise software that is configured to input the raw data and evaluate the accuracy of the user's body positions and movements,)
makes an estimation about an exercise load of the user while the user is playing the video game, and (Lowery, ¶64 the disclosed system can test the user's vestibular brain functions, and, if the user's performance meets a predetermined threshold, the system can test the user's gross motor and cerebellum functions. It can continue working up the pyramid to the test each of the functions until it determines if and where a user needs improvement. Then, when the system determines an area that needs improvement, the assessment portion of the disclosed system can be paused and the training portion can commence.)
further causes the processing circuitry to determine at least one of a task relating to the video game, an incentive relating to the video game, and a game parameter about progress of the video game, at least one of the task, the incentive, and the game parameter being to be provided to the user in accordance with a result of the estimation about the exercise load of the user (Lowery, ¶5 a computing system networked to the interactive exercise software that is configured to input the raw data and evaluate the accuracy of the user's body positions and movements, and provide a score as instant feedback to the user by causing a comparison of the user's body positions and movements to the visual movement instructions to be displayed on a screen, wherein the feedback can improve the body positions and movements of the user.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tzvieli by determining a task relating to the video game and the users progress towards that task to determine the exercise load that is taught by Lowery; thus, one of ordinary skilled in the art would be motivated to combine the references to provide instant feedback to improve body movements of the individual to increase the rate of neuroplasticity through enhanced replication of the exercise directions (Lowery ¶2).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 14, Tzvieli in view of Lowery discloses the information processing apparatus according to claim 1, wherein
the processing circuitry obtains a user video in which a user playing a video game is seen, the video game being a video game of which game progress is controlled in accordance with a body movement of a player, and (Lowery, ¶5 the system includes an exercise therapy video game and monitor system for physical targeted activation and cognitive skills improvement for learning fluency a user comprising: interactive exercise software designed to provide visual movement instructions to the user via a first gaming platform having a screen; a motion sensing device configured to track the body position and movements of the user and create and output raw data corresponding to the user's body positions and movements when the user is replicating the movements of the visual movements instructions; and a computing system networked to the interactive exercise software that is configured to input the raw data and evaluate the accuracy of the user's body positions and movements,)
further causes the computer to:
make an estimation about a skeleton of the user based on the user video; and (Lowery, ¶49 one or more cameras 1222 housed in a housing 1224, as illustrated in FIG. 13, that tracks specific joints and/or body position and movements of the user as well as the time it takes a user to move into desired body positions and movements)
determine at least one of a task relating to the video game, an incentive relating to the video game, and a game parameter about progress of the video game, at least one of the task, the incentive, and the game parameter being to be provided to the user in accordance with a result of the estimation about the skeleton of the user (Lowery, ¶5 a computing system networked to the interactive exercise software that is configured to input the raw data and evaluate the accuracy of the user's body positions and movements, and provide a score as instant feedback to the user by causing a comparison of the user's body positions and movements to the visual movement instructions to be displayed on a screen, wherein the feedback can improve the body positions and movements of the user.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tzvieli by tracking joints of the user and determining a task relating to the video game that is taught by Lowery; thus, one of ordinary skilled in the art would be motivated to combine the references to provide instant feedback to improve body movements of the individual to increase the rate of neuroplasticity through enhanced replication of the exercise directions (Lowery ¶2).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 15, claim 15 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tzvieli further teaching on: A non-transitory computer-readable storage medium, storing computer-readable instruction thereon, which, when executed by processing circuitry, cause the processing circuitry to execute a method (Tzvieli, ¶633 At least some of the methods described herein are “computer-implemented methods” that are implemented on a computer, such as the computer (400, 410), by executing instructions on the processor (401, 411). Additionally, at least some of these instructions may be stored on a non-transitory computer-readable medium.)
Regarding claim 16, claim 16 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tzvieli further teaching on: A method wherein a computer is configured to perform the functions (Tzvieli, ¶633 At least some of the methods described herein are “computer-implemented methods” that are implemented on a computer, such as the computer (400, 410), by executing instructions on the processor (401, 411). Additionally, at least some of these instructions may be stored on a non-transitory computer-readable medium.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664