DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-4 and 6-9 is rejected under 35 U.S.C. 103 as being unpatentable over the article "AI-based Workout assistant and fitness guide" (hereinafter referred to as Taware) in view of US20180036591 (hereinafter referred to as King).
Regarding claim 1, Taware teaches a method for calculating calories burned during activity performance using Artificial Intelligence (AI) [See Taware page 1 section I. paragraph 2 which teaches that AI is used to calculate the calories burned during exercise],
PNG
media_image1.png
407
689
media_image1.png
Greyscale
the method comprising: receiving, in real-time, a video stream of a user performing an activity and a set of user profile attributes of the user [see above section I and figure 3 where a real time video stream is processed and page 2 section III which describes obtaining a user profile.]
PNG
media_image2.png
575
710
media_image2.png
Greyscale
PNG
media_image3.png
424
697
media_image3.png
Greyscale
wherein the video stream comprises a plurality of frames [see page 2 Fig. 3 above which teaches that the video input is converted into image frames]; creating, for each of the plurality of frames, [See above section I and III where profile information and pose information for each frame is given to the AI model]; Processing by an AI model, the [See Fig 3 above where poses are detected from the joint coordinates of the body, and the amount of repetitions is tracked, as well as section I which teach calculating how many calories were burned. This indicates that the activity parameters for the action happening in the frame are determined]; Taware also teaches identifying a set of target activity parameters associated with the target pose [See figure 3 above where pose data, which indicates a target pose is stored in the database] and comparing, by the AI model, each of the set of activity parameters with a corresponding activity parameter from the set of target activity parameters [see Taware page 1 section I. paragraph 2 above where the pose estimation is used to make sure that body posture is correct during the activity. Pose information can be reasonable interpreted as an activity parameter. See also section III part B under fig 3., where the pose is compared with reference data];
PNG
media_image4.png
228
651
media_image4.png
Greyscale
determining in real-time, contemporaneous to the user performing the activity, the accuracy of the workout [see Fig 3 and section III part B where the pose (where a correct pose can indicate a more efficient work out) is compared to reference data as the video data is processed]; and calculating, by the AI model, in real-time, contemporaneous to the user performing the activity, a count of calories burned by the user based on the workout [see above in section I, where the amount of calories burnt is tracked along with section III B, where the accuracy of the pose impacts the exercise duration, and therefore the count of calories burned].
Taware does not explicitly state that vectors are created for the data. Taware also does not explicitly state selecting a target user based on how similar they are to the user profile, where the target user corresponds to a benchmark user. Taware also does not teach a user efficiency level.
King does teach creating vectors [See King 0138 which teaches that vectors can be used to store information about users, including their performance],
PNG
media_image5.png
153
1074
media_image5.png
Greyscale
selecting, by the AI model [See 0163 of King where the AI is used to compare attributes and items]
PNG
media_image6.png
74
1067
media_image6.png
Greyscale
a target user from a plurality of target users based on similarity between the set of user profile attributes of the user and a set of target profile attributes of the target user, wherein the target user corresponds to a benchmark user [See paragraph 0071 of King where the similarity between the user profile and a target user is found. The user with the similar rating history acts as a benchmark user, as the metrics are already present in the database];
PNG
media_image7.png
170
1261
media_image7.png
Greyscale
King also teaches identifying a set of target activity parameters associated with the target user [See paragraph 0152 of King where the recommended workouts acquired from the target user are associated with parameters such as strength, endurance, intensity];
PNG
media_image8.png
274
1407
media_image8.png
Greyscale
Additionally, King also teaches obtaining the accuracy of the workout form compared to a target [See 0102 of King where the user’s form can be scored] and the user efficiency level [see 0121 of King].
PNG
media_image9.png
205
1300
media_image9.png
Greyscale
PNG
media_image10.png
400
1124
media_image10.png
Greyscale
Therefore it would have been obvious to one with ordinary skill in the art before the effective filing date to combine the fitness tracking method of Taware, which uses pose estimation from a live video, with the comparing to other user data of King, as they are in the same field of endeavor of Tracking user fitness. The motivation to combine would be to ensure that the fitness data accounts for outliers and a diverse population of users [See King 0008].
Regarding claim 2, Taware and King teach the method of claim 1, wherein the set of activity parameters comprises a type of activity, a pace of performing the activity [see 0067 below of King which indicates that heartrate is being tracked, and 0092 of King where the heartrate is associated with the pace of the activity],
PNG
media_image11.png
227
1128
media_image11.png
Greyscale
number of repetitions of the activity [see section I of Taware above where the number of repetitions are tracked], periodicity of the activity [See 0084 of King where reminders are set to indicate days since last workout. This indicates that how often the user performs a workout is monitored],
PNG
media_image12.png
195
1136
media_image12.png
Greyscale
duration of performing the activity [see 0067 of King below where the activity duration information is obtained],
PNG
media_image13.png
243
1124
media_image13.png
Greyscale
an accuracy of performing the activity [see section I of Taware where the accuracy of the body posture when an exercise is performed is tracked], and a degree of freedom of the activity [See section III B. and Fig. 4 of Taware which shows that the joint positions are collected and processed for pose estimation. This indicates that the degree of freedom for the activity is obtained].
Regarding claim 3, Taware and King teach the method of claim 1, wherein the set of profile attributes comprises age, gender, height, weight, and physical activities [See King 0063 for the user profile attributes].
Regarding claim 4, Taware and King teach the method of claim 1, wherein identifying the set of target activity parameters further comprises: receiving a video of the target user performing a target activity and the set of target profile attributes of the target user, wherein the video comprises a plurality of frames [see 0129 where the video of the target video can show trainers of a target gender and other attributes, indicating it incorporates the target user's profile attributes. Videos have a plurality of frames];
PNG
media_image14.png
273
1299
media_image14.png
Greyscale
creating, for each of the plurality of frames, a multimedia vector corresponding to the associated frame [See King 0124 below where vectors can be used as a predictive model and 0100 where the video is associated with gestures, indicating that the each of the frames has a corresponding vector];
PNG
media_image15.png
272
1289
media_image15.png
Greyscale
PNG
media_image16.png
232
1297
media_image16.png
Greyscale
and processing, by the AI model, the multimedia vector created for each of the plurality of frames to determine the set of target activity parameters associated with the target user [see paragraph 0094 where an attribute score can be created for each workout, incorporating content from the trainer [Which corresponds to the target user], as well as the associated metrics of the workout together, it would be obvious to use the classified gestures that were processed in frame by frame vectors with the trainer information in an AI model to accurately and efficiently determine the outcome attributes (target activity parameters) in that specific video].
PNG
media_image17.png
336
1193
media_image17.png
Greyscale
Regarding claim 6, Taware and King teach the method of claim 4, further comprising: storing the set of target activity parameters associated with the target user in a database. storing the target efficiency level within the database [See 0065 of King where information can be stored in a database. See also King 0121 above where the efficacy of the workout is stored in the user profile indicating a database is utilized to organize the data for future reference and retrieval].
PNG
media_image18.png
227
1130
media_image18.png
Greyscale
Regarding claim 7, Taware and King teach the method of claim 1, and determining, by the AI model, similarity of the set of user profile attributes with the corresponding set of target profile attributes of each of the plurality of target users [see 0071 of King above where the similarity between the user profile and target profiles is ranked]. Additionally, Kadam teaches that determining the similarity comprises: calculating, by the AI model, a similarity score between each of the set of user profile attributes and each of the corresponding set of target profile attributes [see 0123 of King where similar users are clustered based on profile attributes, where the less distance indicates higher similarity];
PNG
media_image19.png
249
1140
media_image19.png
Greyscale
and selecting, by the AI model, the target user based on the calculated similarity score, wherein the similarity score calculated for the target user is the highest [See 0123 of King above where the cluster with the closest match is selected].
Regarding claim 8, Taware and King teach the method of claim 1, further comprising: rendering, by the AI model via a Graphical User Interface (GUI), efficiency level of the user, the count of calories burned by the user, the set of user activity parameters, the set of target activity parameters on a user device [See 0147 of King which teaches a GUI. This is used to display information about prescribed physical activities, therefore it would have been obvious to communicate the efficiency of the workout as discussed above, as well as the count of calories burned and the user and target activity parameters].
PNG
media_image20.png
300
1131
media_image20.png
Greyscale
Claim 9 is similarly analyzed to claim 1 with King teaching the additional limitations of a processer and memory [see King 0038],
PNG
media_image21.png
167
1128
media_image21.png
Greyscale
Claim 5, and 10-18 are rejected under 35 U.S.C. 103 as being unpatentable over Taware in view of King in further view of “Calories Burned Prediction Using Machine Learning” (hereinafter referred to as Kadam).
Regarding Claim 5, Taware and King teach the method of claim 1, Taware and King teach determining, by the AI model, an estimated target calories for the target activity. [See Taware Introduction above where the calories burned are being counted based on the pose data, indicating that the calories counted are specific to the accuracy and intensity of the work out, indicating an activity type. See additionally, King which classifies the intensity of the exercise so that parameters associated with the target activity are tracked such as [0039 and 0123 below where an activity has an associated effort parameter which has metrics such as heart rate and power
PNG
media_image22.png
71
1029
media_image22.png
Greyscale
PNG
media_image23.png
100
1044
media_image23.png
Greyscale
Although Taware and King utilize exercise parameters and a count of calories burned as a workout is performed, as well as matching the training metrics to the target user as stated previously above, Taware and King do not explicitly teach determining the estimated calories of the target activity based on the target user.
Kadam does teach determining, by the AI model, an estimated target calories for the target target user [See section 1715 section V. of Kadam where an AI model takes in the details of the target user, as well as the exercise information to determine that estimated amount of calories burned].
PNG
media_image24.png
200
586
media_image24.png
Greyscale
and setting a target efficiency level corresponding to the target activity for the target user [See Figure 3 of Kadam below where metrics corresponding to efficiency levels such as the heart rate, and body temperature, which is compared to the target users].
PNG
media_image25.png
378
791
media_image25.png
Greyscale
Therefore it would have been obvious to one with ordinary skill in the art before the effective filing date to combine the method of obtaining video data of Taware with the method of comparing user profiles with a target user to obtain information related to the workout of King, along with the method of using the details from the user profile in comparison to benchmark users to determine the amount of calories burned during physical activities. All of the inventions fall under the same field of endeavor of analyzing data to determine fitness related information. The motivation to combine would be “to accurately predict the number of calories burned during exercise” as stated in section VI of Kadam.
Claim 10 is similarly analyzed to claim 1 with the benchmark user being read the same as the target user as previously mentioned in claim 1. Benchmark user can be read this way, as “benchmark” simply indicates that there are known values associated with that user. Therefore, here examiner addresses the additional limitation of adapting, by the AI model, the user efficiency level based on the set of user profile attributes of the user and a set of benchmark profile attributes of the benchmark user; and calculating, by the AI model, in real-time, contemporaneous to the user performing the activity, a count of calories burned by the user based on the adapted user efficiency level.
Although as stated above, Taware and King teach obtaining a benchmark profile, determining a user efficiency level and contemporaneously tracking a count of calories, Taware and King do not explicitly teach that the obtaining of the user profile adapts the user efficiency level or count of calories burned.
Kadam does teach adapting, by the AI model, the calculation of calories burned based on the set of user profile attributes of the user and a set of benchmark profile attributes of the benchmark user [see section VI system design on page 1714 where heart rate, body temp, gender, age and height are all used for analysis, as well as data from similar users. This data is input into a machine learning model which the system considers, indicating that the calculation for the amount of calories burned for the activity can be adapted];
PNG
media_image26.png
231
363
media_image26.png
Greyscale
and calculating, by the AI model, in real-time, contemporaneous to the user performing the activity, a count of calories burned by the user based on the adapted calculation algorithm. [See section VI of Kadam above where the adaptions made based on user related components, and comparisons to benchmark user is used to accurately predict the amount of calories burned during an activity].
Therefore it would have been obvious to one with ordinary skill in the art before the effective filing date to combine the method of using real time video data to for pose estimation and the count of calories of Taware, with the user efficiency level and comparison to a benchmark user of King, along with the adapting the calories burned based on user metrics of Kadam, as they are in the same field of endeavor of producing accurate health data information when it comes to physical activity. The motivation to combine is “to accurately predict the number of calories burned during exercise” as stated in section VI of Kadam.
Claim 11 is similarly analyzed to claim 2 with the additional limitations of claim 10
Claim 12 is similarly analyzed to claim 3 with the additional limitations of claim 10
Claim 13 is similarly analyzed to claim 4 with the additional limitations of claim 10
Claim 14 is similarly analyzed to claim 5 with the additional limitations of claim 10
Claim 15 is similarly analyzed to claim 6 with the additional limitations of claim 10
Claim 16 is similarly analyzed to claim 7 with the additional limitation of adjusting, by the AI model, the user efficiency level based on the computed deviation for each of the set of user profile attributes [see paragraph 0118 below of King where the deviation in the user profile attributes is used to adjust the weights, therefore adjusting the user efficiency level using the AI model].
PNG
media_image27.png
168
1048
media_image27.png
Greyscale
Claim 17 is similarly analyzed to claim 8 with the additional limitations of claim 10
Claim 18 is similarly analyzed to claim 10 with King teaching the additional limitations of a processer and memory [see King 0038].
PNG
media_image21.png
167
1128
media_image21.png
Greyscale
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANUSHA KASHYAPA whose telephone number is (571) 272-8766. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANUSHA KASHYAPA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights