DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 26, 2026 has been entered.
Response to Amendment
The amendment filed June 26, 2026 has been entered. Claims 1, 3-5, 7-9, 12-30, 33-49, and 52-65 remain pending in the application.
Response to Arguments
Applicant’s arguments, see Pages 18-21 of Remarks, filed June 26, 2026, with respect to the rejection(s) of claim(s) 1, 3-5, 7-9, 12-30, 33-49, and 52-65 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Halevy et al. (US 20220241642 A1).
Claim Objections
Claims 63-65 are objected to because of the following informalities:
In claims 63-65 lines 5-6, “wherein displaying the virtual avatar includes; and” should read “wherein displaying the virtual avatar includes:”.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 12-16, 18-20, and 22-24 (and corresponding claims 33-37, 39-41, 43, and 52-56, 58-60, 62) are rejected under 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) as being anticipated by Halevy et al. (US 20220241642 A1), hereinafter Halevy.
Regarding claim 1, Halevy teaches a computer system configured to communicate with a camera and a display generation component (Paragraph 0030, 0032, 0038 – “system 100 may function as a computer-based, or artificially intelligent, personal trainer. System 100 may provide general instruction regarding movement based activities and may utilize motion capture devices 114 to record the movement of a user 112… the system 100 may display (e.g., via display 116) a demonstration of an example movement pattern (e.g., a video or other visual representation) for a movement activity to be performed by the user 112…motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream (e.g., a temporally ordered sequence of 2D or 3D images). In order to capture the images, a motion capture device 114 may include one or more image sensors, e.g., light detection and ranging (LIDAR) sensors, two-dimensional (2D) cameras (e.g., RGB cameras)”; Note: the system is a computer-based system that communicates with a motion capture device/camera and a display), comprising:
one or more processors (Paragraph 0106 – “Servers and other computing devices of system 100 can each include one or more processors”);
and memory storing one or more programs configured to be executed by the one or more processors (Paragraph 0106, 0110 – “Servers and other computing devices of system 100 can each include one or more processors, computer-readable memory… Memory may store any suitable data or information utilized by computing devices, including software embedded in a computer readable medium, and/or encoded logic incorporated in hardware or otherwise stored (e.g., firmware). Memory may also store the results and/or intermediate results of the various calculations and determinations performed by processors”), the one or more programs including instructions for:
capturing, via the camera, image data of a physical environment that includes a user of the computer system performing physical movements (Paragraph 0031, 0038, 0041 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity…a motion capture device 114 may include one or more image sensors, e.g., light detection and ranging (LIDAR) sensors, two-dimensional (2D) cameras (e.g., RGB cameras)…motion capture and feedback system 102 includes any suitable number and types of motion capture devices placed at different poses relative to the user 112 to enable capture of sufficient data to allow position determination of a group of body parts (which in some embodiments may be arranged into a skeleton) of the user 112 in 3D space, where a pose refers to the position and orientation of a motion capture device with respect to a reference coordinate system”);
prior to assessing whether the physical movements meet a set of one or more movement criteria, displaying a first portion of a video representation of a first individual other than the user of the computer system (Paragraph 0050 – “display 116 may display a trainer performing an example movement pattern for an activity to the user 112…the display of the trainer may be video (or a derivation thereof) of one or more experts of expert network system…The trainer may be displayed at any suitable time, such as before the user 112 performs a repetition of the activity”; Note: a video of a trainer, which is equivalent to the first individual, is displayed. It is implied that the video is displayed prior to movement assessment since the user has not performed the activity/movement yet);
assessing, based on the image data, whether the physical movements meet the set of one or more movement criteria (Paragraph 0031, 0066-0067 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity, analyze the motion with respect to model movement patterns…system 102 may analyze movement form of the user 112 performing an activity and determine that the user has suboptimal movement form…The determination that the user has suboptimal movement form may be based on comparison of the movement of the user 112 with data representing a model movement pattern and/or data representing improper movement patterns”; Note: based on the image data from the motion capture, the system assesses the user movement and determines if it meets proper movement pattern, which is equivalent to movement criteria);
and after assessing whether the physical movements meet a set of one or more movement criteria, displaying, via the display generation component, visual feedback based on the image data (Paragraph 0031, 0073 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity, analyze the motion with respect to model movement patterns, and provide real-time feedback and encouragement to the user 112 to promote healthy and optimal movement patterns…Responsive to a determination that the user 112 has committed a movement error, the system 102 may provide instruction regarding how to improve the movement form. The instruction may be visual (e.g., displayed on display 116)”; Note: after assessing that the user movement from the motion capture, visual feedback is displayed),
wherein displaying the visual feedback includes displaying, without requiring further user input, a second portion of the video representation of the first individual, wherein the video representation includes the first individual performing a second set of target movements that should be performed by the user of the computer system (Paragraph 0085 – “FIG. 6 illustrates an example series of images that may be generated by system 102 and displayed (e.g., by display 116) to provide movement instruction to user 112. The images may be part of a video stream that is displayed by the display 116. At 602, the user begins a repetition of an exercise. The display 116 shows the example movement pattern in the upper left corner and the representation of the user 112 in the middle. Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view”; Note: a video of a new repetition of the movement (second portion of the video) being performed by the trainer (first individual) is displayed. The displaying is performed by the system, without user input), wherein:
in accordance with a determination that the physical movements meet the set of one or more movement criteria, the second portion of the video representation includes a first representation of the first individual (Paragraph 0085-0087 – “Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view… an animation is played wherein the guide skeleton disappears to indicate that the correct form has been attained and an encouraging message (e.g., “Excellent!”) is presented at 610…once all of the guide skeleton and user segments are successfully aligned (or it is otherwise determined that the movement error has been corrected), the point of view may transition back to the default view”; Note: it is implied that when the movement criteria is met (correct form, no errors), the default point-of-view/representation of the trainer (first individual) is maintained. The default point-of-view is equivalent to the first representation);
and in accordance with a determination that the physical movements do not meet the set of one or more movement criteria, the second portion of the video representation includes a second representation of the first individual that is different from the first representation (Paragraph 0074, 0077, 0085 – “responsive to a detection of a movement error, an example movement pattern of the trainer performing the activity is shown (e.g., from an optimal point of view) to illustrate how the movement error may be corrected…the specific movement and/or body parts associated with the movement error may be highlighted on the view of the trainer as the trainer moves through the particular activity…when a movement error is detected, the associated optimal point of view is determined, and the representation of the user 112 or trainer is then displayed from that optimal point of view …Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view”; Note: when the movement criteria is not met (movement errors or suboptimal movement pattern detected), an optimal point-of-view/representation of the trainer (first individual) is displayed. The optimal point-of-view is equivalent to the second representation).
Regarding claim 12, Halevy teaches the computer system of claim 1. Halevy further teaches the one or more programs further including instructions for: while displaying the video representation of the first individual, displaying a video representation of the user of the computer system based on the captured image data (Paragraph 0038, 0050-0051 – “motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream…display 116 may display a trainer performing an example movement pattern for an activity to the user 112. The example movement pattern may take any suitable form (such as any of the representation formats described below with respect to a trainer or user 112). In some embodiments, the display of the trainer may be video… the trainer may be displayed simultaneously with the user 112”; Note: a video representation of a trainer (first individual) and a video representation of the user are displayed at the same time). The rejection and explanation for claim 12 above also applies to its corresponding CRM claim 33 and method claim 52.
Regarding claim 13, Halevy teaches the computer system of claim 1. Halevy further teaches wherein: the physical movements correspond to a first repetition of a set of physical movements; and displaying the visual feedback includes displaying guidance on a second set of target movements that should be performed by the user of the computer system, wherein the guidance on the second set of target movements is based on an assessment of the first repetition of the set of physical movements (Paragraph 0085 – “Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view to allow the user to view her chest in association with the personalized feedback (“Keep your chest up”) displayed by the system 102”; Note: the previous repetition is equivalent to the first repetition. Guidance is displayed on the current/second repetition set of movements based on the detected movements from the previous repetition). The rejection and explanation for claim 13 above also applies to its corresponding CRM claim 34 and method claim 53.
Regarding claim 14, Halevy teaches the computer system of claim 13. Halevy further teaches wherein the guidance on the second set of target movements does not include a representation of a human (Paragraph 0079 – “Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the exercise), the system 102 displays a corrective message: ‘Keep your chest up’”; Note: the guidance in this case is a textual message, not a human representation). The rejection and explanation for claim 14 above also applies to its corresponding CRM claim 35 and method claim 54.
Regarding claim 15, Halevy teaches the computer system of claim 13. Halevy further teaches wherein the guidance on the second set of target movements includes graphical content and audio content (Paragraph 0073, 0079, 0088 – “Responsive to a determination that the user 112 has committed a movement error, the system 102 may provide instruction regarding how to improve the movement form. The instruction may be visual (e.g., displayed on display 116) and/or auditory…Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the exercise), the system 102 displays a corrective message: ‘Keep your chest up’…the view showing the user 112 at the optimal point of view may also include one or more visual targets for the user's body parts so that the user can align with the proper form. The visual targets may include one or more of a written message with movement instruction such as “keep your chest up” or “bend your knees more”, an auditory message with movement instruction, or the guide skeleton showing a target position”; Note: the guidance is a textual message (graphical content) and/or auditory message). The rejection and explanation for claim 15 above also applies to its corresponding CRM claim 36 and method claim 55.
Regarding claim 16, Halevy teaches the computer system of claim 13. Halevy further teaches wherein the guidance on the second set of target movements includes content that is responsive to ongoing physical movements of the user of the computer system that are captured by the camera (Paragraph 0038, 0085-0086 – “motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream…FIG. 6 illustrates an example series of images that may be generated by system 102 and displayed (e.g., by display 116) to provide movement instruction to user 112. The images may be part of a video stream that is displayed by the display 116. At 602, the user begins a repetition of an exercise. The display 116 shows the example movement pattern in the upper left corner and the representation of the user 112 in the middle. Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view to allow the user to view her chest in association with the personalized feedback (“Keep your chest up”) displayed by the system 102. At 604, the user 112 has squatted down and is at or near the target position to be corrected. While the lower half of the user is correctly aligned with the guide skeleton, the upper half is still misaligned (in various embodiments, the misaligned segments may be a different color from the aligned segments, illustrated here by different dashing in the segments). At 606, the user has corrected position and the entire guide skeleton has turned the same color (illustrated by each segment having the same dashing). At 608 an animation is played wherein the guide skeleton disappears to indicate that the correct form has been attained and an encouraging message (e.g., “Excellent!”) is presented at 610”; Note: guidance on the current repetition is displayed in response to user movement from the video stream, captured by the camera/motion capture device). The rejection and explanation for claim 16 above also applies to its corresponding CRM claim 37 and method claim 56.
Regarding claim 18, Halevy teaches the computer system of claim 1. Halevy further teaches wherein displaying the visual feedback occurs after completing a process of image data capture that includes the capture of the image data and the assessing of whether the physical movements meet a set of one or more movement criteria (Paragraph 0038, 0085 – “motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream …FIG. 6 illustrates an example series of images that may be generated by system 102 and displayed (e.g., by display 116) to provide movement instruction to user 112. The images may be part of a video stream that is displayed by the display 116. At 602, the user begins a repetition of an exercise. The display 116 shows the example movement pattern in the upper left corner and the representation of the user 112 in the middle. Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view to allow the user to view her chest in association with the personalized feedback (“Keep your chest up”) displayed by the system 102. At 604, the user 112 has squatted down and is at or near the target position to be corrected. While the lower half of the user is correctly aligned with the guide skeleton, the upper half is still misaligned”; Note: feedback, such as the text or guide skeleton, is displayed only after image capture and detecting/assessing user movement patterns from the previous repetition). The rejection and explanation for claim 18 above also applies to its corresponding CRM claim 39 and method claim 58.
Regarding claim 19, Halevy teaches the computer system of claim 18. Halevy further teaches wherein the visual feedback is displayed over a representation of the user performing a physical movement that does not meet the set of one or more movement criteria (Fig. 5, Paragraph 0080-0081 – “system 102 may display a guide skeleton 514 to preview the correct movement form…the guide skeleton may be selected based on the type of detected movement error as a fixed position depicted by the guide skeleton may be selected to illustrate a position that needs correction…At 604, the user 112 has squatted down and is at or near the target position to be corrected. While the lower half of the user is correctly aligned with the guide skeleton, the upper half is still misaligned (in various embodiments, the misaligned segments may be a different color from the aligned segments, illustrated here by different dashing in the segments). At 606, the user has corrected position and the entire guide skeleton has turned the same color (illustrated by each segment having the same dashing). At 608 an animation is played wherein the guide skeleton disappears to indicate that the correct form has been attained”; Note: the guide skeleton, which is visual feedback, is displayed over the user performing the movement that has an error (does not meet criteria). Fig. 5 shows examples of the guide skeleton; see screenshot of Fig. 5 below).
PNG
media_image1.png
416
462
media_image1.png
Greyscale
Screenshot of Fig. 5 (taken from Halevy)
The rejection and explanation for claim 19 above also applies to its corresponding CRM claim 40 and method claim 59.
Regarding claim 20, Halevy teaches the computer system of claim 1. Halevy further teaches wherein the camera is contained within a housing of a first electronic device and the display generation component is in a housing of a second electronic device that is different from and separate from the first electronic device (Fig. 12, Paragraph 0038, 0042, 0125 – “motion capture device 114 may include one or more image sensors, e.g., light detection and ranging (LIDAR) sensors, two-dimensional (2D) cameras (e.g., RGB cameras)… a motion capture device 114 may have its own housing, power supply, or its own connection (e.g., wired or wireless) to the computing device 118 to send data captured by its image sensor(s) (or data processed therefrom) to the computing device 118…System 1220 may include motion capture devices 1214 (e.g., 1214C and 1214D) or other suitable devices (e.g., devices that may communicate with wearable sensors) that capture positional data representative of the movement of the user 1226 (e.g., using any of the techniques described above for tracking the motion of user 112 or subject 1212 or other suitable techniques). The positional data may be processed by computing device 1222 and/or other computing devices in the network (e.g., backend system 1204) and feedback that is based on the user's movement and the movement based instruction included in the activity profile may be provided to the user (e.g., via display 1224)”; Note: Fig. 12 shows the display 1224 having a different housing from motion capture devices 1214; see screenshot below).
PNG
media_image2.png
294
366
media_image2.png
Greyscale
Screenshot of Fig. 12 (taken from Halevy)
The rejection and explanation for claim 20 above also applies to its corresponding CRM claim 41 and method claim 60.
Regarding claim 22, Halevy teaches the computer system of claim 1. Halevy further teaches the one or more programs further including instructions for: while capturing the image data, displaying an indication of a number of repetitions of physical movements that meet the set of one or more movement criteria (Paragraph 0049, 0057, 0162-0163 – “system 102 renders the dynamic posture of the user 112 on the display 116 in real-time during performance of an activity by the user…system 102 may display, via repetition tracker 306, a number of repetitions of an activity that have been performed by the user 112…the configured phases and associated captures may be utilized by the system 1200 to implement a counter that tracks the number of repetitions of the activity that have been performed…A repetition may be detected when a determination is made that the user has passed through each phase of the activity (e.g., as defined by the captures and corresponding positions and/or positional boundary areas of the phases). For example, the counter may increment for a Goblet Squat after a determination that a user or subject has completed the Top, Down, and Fin phases. In various embodiments, a determination that a user has completed a phase may be made based on a determination that the 3D position of body parts of a user have passed constraints set by one or more captures associated with the phase. For example, if a user's body part has reached the associated positional boundary area of a capture then it may be determined that the constraints of the capture have been met”; Note: the number of repetitions meeting the movement constraints/criteria is displayed). The rejection and explanation for claim 22 above also applies to its corresponding CRM claim 43 and method claim 62.
Regarding claim 23, Halevy teaches a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors (Paragraph 0227 – “The embodiments of methods, hardware, software, firmware or code set forth above may be implemented via instructions or code stored on a machine-accessible, machine readable, computer accessible, or computer readable medium which are executable by a processing element. A machine-accessible/readable medium includes any mechanism that provides (e.g., stores and/or transmits) information in a form readable by a machine, such as a computer or electronic system. For example, a non-transitory machine-accessible medium includes random-access memory (RAM)”) of a computer system that is in communication with a camera and a display generation component (Paragraph 0030, 0032, 0038 – “system 100 may function as a computer-based, or artificially intelligent, personal trainer. System 100 may provide general instruction regarding movement based activities and may utilize motion capture devices 114 to record the movement of a user 112… the system 100 may display (e.g., via display 116) a demonstration of an example movement pattern (e.g., a video or other visual representation) for a movement activity to be performed by the user 112…motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream (e.g., a temporally ordered sequence of 2D or 3D images). In order to capture the images, a motion capture device 114 may include one or more image sensors, e.g., light detection and ranging (LIDAR) sensors, two-dimensional (2D) cameras (e.g., RGB cameras)”; Note: the system is a computer-based system that communicates with a motion capture device/camera and a display), the one or more programs including instructions for:
capturing, via the camera, image data of a physical environment that includes a user of the computer system performing physical movements (Paragraph 0031, 0038, 0041 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity…a motion capture device 114 may include one or more image sensors, e.g., light detection and ranging (LIDAR) sensors, two-dimensional (2D) cameras (e.g., RGB cameras)…motion capture and feedback system 102 includes any suitable number and types of motion capture devices placed at different poses relative to the user 112 to enable capture of sufficient data to allow position determination of a group of body parts (which in some embodiments may be arranged into a skeleton) of the user 112 in 3D space, where a pose refers to the position and orientation of a motion capture device with respect to a reference coordinate system”);
prior to assessing whether the physical movements meet a set of one or more movement criteria, displaying a first portion of a video representation of a first individual other than the user of the computer system (Paragraph 0050 – “display 116 may display a trainer performing an example movement pattern for an activity to the user 112…the display of the trainer may be video (or a derivation thereof) of one or more experts of expert network system…The trainer may be displayed at any suitable time, such as before the user 112 performs a repetition of the activity”; Note: a video of a trainer, which is equivalent to the first individual, is displayed. It is implied that the video is displayed prior to movement assessment since the user has not performed the activity/movement yet);
assessing, based on the image data, whether the physical movements meet the set of one or more movement criteria (Paragraph 0031, 0066-0067 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity, analyze the motion with respect to model movement patterns…system 102 may analyze movement form of the user 112 performing an activity and determine that the user has suboptimal movement form…The determination that the user has suboptimal movement form may be based on comparison of the movement of the user 112 with data representing a model movement pattern and/or data representing improper movement patterns”; Note: based on the image data from the motion capture, the system assesses the user movement and determines if it meets proper movement pattern, which is equivalent to movement criteria);
and after assessing whether the physical movements meet a set of one or more movement criteria, displaying, via the display generation component, visual feedback based on the image data (Paragraph 0031, 0073 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity, analyze the motion with respect to model movement patterns, and provide real-time feedback and encouragement to the user 112 to promote healthy and optimal movement patterns…Responsive to a determination that the user 112 has committed a movement error, the system 102 may provide instruction regarding how to improve the movement form. The instruction may be visual (e.g., displayed on display 116)”; Note: after assessing that the user movement from the motion capture, visual feedback is displayed),
wherein displaying the visual feedback includes displaying, without requiring further user input, a second portion of the video representation of the first individual, wherein the video representation includes the first individual performing a second set of target movements that should be performed by the user of the computer system (Paragraph 0085 – “FIG. 6 illustrates an example series of images that may be generated by system 102 and displayed (e.g., by display 116) to provide movement instruction to user 112. The images may be part of a video stream that is displayed by the display 116. At 602, the user begins a repetition of an exercise. The display 116 shows the example movement pattern in the upper left corner and the representation of the user 112 in the middle. Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view”; Note: a video of a new repetition of the movement (second portion of the video) being performed by the trainer (first individual) is displayed. The displaying is performed by the system, without user input), wherein:
in accordance with a determination that the physical movements meet the set of one or more movement criteria, the second portion of the video representation includes a first representation of the first individual (Paragraph 0085-0087 – “Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view… an animation is played wherein the guide skeleton disappears to indicate that the correct form has been attained and an encouraging message (e.g., “Excellent!”) is presented at 610…once all of the guide skeleton and user segments are successfully aligned (or it is otherwise determined that the movement error has been corrected), the point of view may transition back to the default view”; Note: it is implied that when the movement criteria is met (correct form, no errors), the default point-of-view/representation of the trainer (first individual) is maintained. The default point-of-view is equivalent to the first representation);
and in accordance with a determination that the physical movements do not meet the set of one or more movement criteria, the second portion of the video representation includes a second representation of the first individual that is different from the first representation (Paragraph 0074, 0077, 0085 – “responsive to a detection of a movement error, an example movement pattern of the trainer performing the activity is shown (e.g., from an optimal point of view) to illustrate how the movement error may be corrected…the specific movement and/or body parts associated with the movement error may be highlighted on the view of the trainer as the trainer moves through the particular activity…when a movement error is detected, the associated optimal point of view is determined, and the representation of the user 112 or trainer is then displayed from that optimal point of view …Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view”; Note: when the movement criteria is not met (movement errors or suboptimal movement pattern detected), an optimal point-of-view/representation of the trainer (first individual) is displayed. The optimal point-of-view is equivalent to the second representation).
Regarding claim 24, Halevy teaches a method comprising: at a computer system that is in communication with a camera and a display generation component (Paragraph 0030, 0032, 0038 – “system 100 may function as a computer-based, or artificially intelligent, personal trainer. System 100 may provide general instruction regarding movement based activities and may utilize motion capture devices 114 to record the movement of a user 112… the system 100 may display (e.g., via display 116) a demonstration of an example movement pattern (e.g., a video or other visual representation) for a movement activity to be performed by the user 112…motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream (e.g., a temporally ordered sequence of 2D or 3D images). In order to capture the images, a motion capture device 114 may include one or more image sensors, e.g., light detection and ranging (LIDAR) sensors, two-dimensional (2D) cameras (e.g., RGB cameras)”; Note: the system is a computer-based system that communicates with a motion capture device/camera and a display. The instructions performed by the system is equivalent to the method):
capturing, via the camera, image data of a physical environment that includes a user of the computer system performing physical movements (Paragraph 0031, 0038, 0041 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity…a motion capture device 114 may include one or more image sensors, e.g., light detection and ranging (LIDAR) sensors, two-dimensional (2D) cameras (e.g., RGB cameras)…motion capture and feedback system 102 includes any suitable number and types of motion capture devices placed at different poses relative to the user 112 to enable capture of sufficient data to allow position determination of a group of body parts (which in some embodiments may be arranged into a skeleton) of the user 112 in 3D space, where a pose refers to the position and orientation of a motion capture device with respect to a reference coordinate system”);
prior to assessing whether the physical movements meet a set of one or more movement criteria, displaying a first portion of a video representation of a first individual other than the user of the computer system (Paragraph 0050 – “display 116 may display a trainer performing an example movement pattern for an activity to the user 112…the display of the trainer may be video (or a derivation thereof) of one or more experts of expert network system…The trainer may be displayed at any suitable time, such as before the user 112 performs a repetition of the activity”; Note: a video of a trainer, which is equivalent to the first individual, is displayed. It is implied that the video is displayed prior to movement assessment since the user has not performed the activity/movement yet);
assessing, based on the image data, whether the physical movements meet the set of one or more movement criteria (Paragraph 0031, 0066-0067 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity, analyze the motion with respect to model movement patterns…system 102 may analyze movement form of the user 112 performing an activity and determine that the user has suboptimal movement form…The determination that the user has suboptimal movement form may be based on comparison of the movement of the user 112 with data representing a model movement pattern and/or data representing improper movement patterns”; Note: based on the image data from the motion capture, the system assesses the user movement and determines if it meets proper movement pattern, which is equivalent to movement criteria);
and after assessing whether the physical movements meet a set of one or more movement criteria, displaying, via the display generation component, visual feedback based on the image data (Paragraph 0031, 0073 – “The system 100 includes a motion capture and feedback system 102 operable to track the motion of a user 112 performing a movement activity, analyze the motion with respect to model movement patterns, and provide real-time feedback and encouragement to the user 112 to promote healthy and optimal movement patterns…Responsive to a determination that the user 112 has committed a movement error, the system 102 may provide instruction regarding how to improve the movement form. The instruction may be visual (e.g., displayed on display 116)”; Note: after assessing that the user movement from the motion capture, visual feedback is displayed),
wherein displaying the visual feedback includes displaying, without requiring further user input, a second portion of the video representation of the first individual, wherein the video representation includes the first individual performing a second set of target movements that should be performed by the user of the computer system (Paragraph 0085 – “FIG. 6 illustrates an example series of images that may be generated by system 102 and displayed (e.g., by display 116) to provide movement instruction to user 112. The images may be part of a video stream that is displayed by the display 116. At 602, the user begins a repetition of an exercise. The display 116 shows the example movement pattern in the upper left corner and the representation of the user 112 in the middle. Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view”; Note: a video of a new repetition of the movement (second portion of the video) being performed by the trainer (first individual) is displayed. The displaying is performed by the system, without user input), wherein:
in accordance with a determination that the physical movements meet the set of one or more movement criteria, the second portion of the video representation includes a first representation of the first individual (Paragraph 0085-0087 – “Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view… an animation is played wherein the guide skeleton disappears to indicate that the correct form has been attained and an encouraging message (e.g., “Excellent!”) is presented at 610…once all of the guide skeleton and user segments are successfully aligned (or it is otherwise determined that the movement error has been corrected), the point of view may transition back to the default view”; Note: it is implied that when the movement criteria is met (correct form, no errors), the default point-of-view/representation of the trainer (first individual) is maintained. The default point-of-view is equivalent to the first representation);
and in accordance with a determination that the physical movements do not meet the set of one or more movement criteria, the second portion of the video representation includes a second representation of the first individual that is different from the first representation (Paragraph 0074, 0077, 0085 – “responsive to a detection of a movement error, an example movement pattern of the trainer performing the activity is shown (e.g., from an optimal point of view) to illustrate how the movement error may be corrected…the specific movement and/or body parts associated with the movement error may be highlighted on the view of the trainer as the trainer moves through the particular activity…when a movement error is detected, the associated optimal point of view is determined, and the representation of the user 112 or trainer is then displayed from that optimal point of view …Because a suboptimal movement pattern has been detected (e.g., in a previous repetition of the activity), the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view”; Note: when the movement criteria is not met (movement errors or suboptimal movement pattern detected), an optimal point-of-view/representation of the trainer (first individual) is displayed. The optimal point-of-view is equivalent to the second representation).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 63 and 3-5 (and corresponding claims 64-65, 25-27, 44-46) are rejected under 35 U.S.C. 103 as being unpatentable over Halevy in view of Aragones et al. (US 20210358591 A1), hereinafter Aragones.
Regarding claim 63, Halevy teaches the computer system of claim 1. Halevy further teaches wherein: data corresponding to the user of the computer system in the captured image data includes data of the user of the computer system captured at a first size within a field-of-view of the camera (Fig. 2A, Paragraph 0038, 0041 – “motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream… a motion capture device 114A is placed directly in front of the user 112 and a second motion capture device 114B is placed to the side of the user 112 (such that the angle formed between the first device, the user 112, and the second device is roughly 90 degrees in a horizontal plane). As another example, two motion capture devices may be placed at least a threshold distance apart (e.g., 5 feet) and may each be oriented towards the subject (e.g., one at a 45 degree angle and one at a −45 degree angle in a horizontal plane with respect to the user 112)”; Note: Fig. 2A shows the user captured at a first size; see screenshot of Fig. 2A below); displaying the visual feedback includes displaying a virtual avatar that is animated based on the physical movements of the user captured in the image data (Paragraph 0049, 0054-0055 – “system 102 renders the dynamic posture of the user 112 on the display 116 in real-time during performance of an activity by the user…FIG. 2A illustrates a display of a representation 202 of a user 112… a representation 202 may include a view of the entire user 112 as captured by the motion capture devices 114 and transformed (e.g., via a matrix) to the desired orientation, a view of a skeleton or other key points of the user 112, an avatar of the user 112 or superimposed on a representation of the user 112 (e.g., the representation 202 may be a simulated human or avatar with movements governed by the 3D positional data set), or an extrapolation of the images captured by motion capture devices 114”). Halvey does not teach wherein displaying the virtual avatar includes: in accordance with a determination that the physical movements meet the set of one or more movement criteria, displaying the virtual avatar in a first manner that includes displaying a portion of the virtual avatar at a third size, wherein the portion of the virtual avatar corresponds to a portion of the user of the computer system that is included in the captured image data; and in accordance with a determination that the physical movements do not meet the set of one or more movement criteria, displaying the virtual avatar in a second manner, different than the first manner, that includes displaying the portion of the virtual avatar at a fourth size, different from the first size, that is larger than the third size. However, Aragones teaches wherein displaying the virtual avatar includes: in accordance with a determination that the physical movements meet the set of one or more movement criteria, displaying the virtual avatar in a first manner that includes displaying a portion of the virtual avatar at a third size (Paragraph 0100, 0102 – “Computer 102 may define overlap zones that compare the amount of distance between a desired body part location and an actual location. For example, a good form zone may be within a first distance from a desired location (e.g., elbow is within 2 inches from desired location) or vary by no more than a certain percentage (e.g., 5%) from the desired location. An acceptable form zone may be within a second distance range of a desired location (e.g., elbow is within 2-4 inches from desired location) or where a body part differs by no more than a certain percentage (e.g., 15%) from the desired location. An unacceptable form zone may be more than a certain distance away from a desired location and/or where a body part differs by more than a certain percentage (e.g., more than 15%) from a desired location…As part of the overlap determinations and/or other criteria, system 100 may cause the display 136 to present a recommended correction to the user's form. This may be performed whether there is an indication of either an acceptable form zone or an unacceptable form zone. With reference to FIG. 10B, the displayed instruction 1006 may be provided to prompt the user to straighten their knees. Computer 102 may also cause the displayed video of the user avatar 802…to zoom in on or enlarge video of a body part or region of a user's body that has poor form…to audibly or visually inform the user of the problem”; Note: after determining that the movement is in good form zone (meets the movement criteria), it is implied that the size of the displayed body part of the user avatar is maintained (not enlarged). The body part is equivalent to the portion of the user, and the maintained size is equivalent to a third size), wherein the portion of the virtual avatar corresponds to a portion of the user of the computer system that is included in the captured image data (Paragraph 0098 – “computer 102 (or any other electronic device) may divide an image from captured video into subsections to identify body parts that may be performing an incorrect movement”); and in accordance with a determination that the physical movements do not meet the set of one or more movement criteria, displaying the virtual avatar in a second manner, different than the first manner, that includes displaying the portion of the virtual avatar at a fourth size, different from the first size, that is larger than the third size (Paragraph 0100, 0102 – “Computer 102 may define overlap zones that compare the amount of distance between a desired body part location and an actual location. For example, a good form zone may be within a first distance from a desired location (e.g., elbow is within 2 inches from desired location) or vary by no more than a certain percentage (e.g., 5%) from the desired location. An acceptable form zone may be within a second distance range of a desired location (e.g., elbow is within 2-4 inches from desired location) or where a body part differs by no more than a certain percentage (e.g., 15%) from the desired location. An unacceptable form zone may be more than a certain distance away from a desired location and/or where a body part differs by more than a certain percentage (e.g., more than 15%) from a desired location…As part of the overlap determinations and/or other criteria, system 100 may cause the display 136 to present a recommended correction to the user's form. This may be performed whether there is an indication of either an acceptable form zone or an unacceptable form zone. With reference to FIG. 10B, the displayed instruction 1006 may be provided to prompt the user to straighten their knees. Computer 102 may also cause the displayed video of the user avatar 802…to zoom in on or enlarge video of a body part or region of a user's body that has poor form…to audibly or visually inform the user of the problem”; Note: after determining that the movement is in acceptable or unacceptable form zone (does not meet the movement criteria), the user avatar is displayed with an enlarged body part, which is equivalent to a fourth size). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halevy to incorporate the teachings of Aragones to display a portion of the virtual avatar in a larger size if the movement criteria are not met for the benefit of “audibly or visually inform[ing] the user of the problem” (Aragones 2: Paragraph 0102). In other words, enlarging the body part associated with the movement error highlights and makes it clearer where the user should focus on to fix the issue.
PNG
media_image3.png
443
226
media_image3.png
Greyscale
Screenshot of Fig. 2A (taken from Halevy)
The rejection and explanation for claim 63 above also applies to its corresponding CRM claim 64 and method claim 65.
Regarding claim 3, Halevy in view of Aragones teaches the computer system of claim 63. Halevy further teaches wherein the virtual avatar is displayed without displaying a photorealistic representation of the user that is based on the captured image data (Fig. 2A, Paragraph 0054-0055– “FIG. 2A illustrates a display of a representation 202 of a user 112…a representation 202 may include a view of the entire user 112 as captured by the motion capture devices 114 and transformed (e.g., via a matrix) to the desired orientation, a view of a skeleton or other key points of the user 112, an avatar of the user 112 or superimposed on a representation of the user 112 (e.g., the representation 202 may be a simulated human or avatar with movements governed by the 3D positional data set)”; Note: Fig. 2A shows an avatar of the user that is not photorealistic; see screenshot of Fig. 2A above). The rejection and explanation for claim 3 above also applies to its corresponding CRM claim 25 and method claim 44.
Regarding claim 4, Halevy in view of Aragones teaches the computer system of claim 63. Halevy does not teach wherein: data corresponding to the user of the computer system in the captured image data includes data of the user of the computer system captured from a first perspective; and displaying the virtual avatar (Paragraph 0054-0055 – “FIG. 2A illustrates a display of a representation 202 of a user 112…a representation 202 may include a view of the entire user 112 as captured by the motion capture devices 114 and transformed (e.g., via a matrix) to the desired orientation, a view of a skeleton or other key points of the user 112, an avatar of the user 112”; Note: the displayed representation of the user is an avatar) includes displaying the virtual avatar from a second perspective, different from the first perspective (Paragraph 0038, 0041, 0085 – “motion capture devices 114A and 114B may capture multiple images (e.g., 2D or 3D images) of the user 112 over a time period to produce a video stream…a motion capture device 114A is placed directly in front of the user 112…the point of view of the display of the user 112 (and the trainer) has been changed from the default point of view (which is shown at 610) to a side view to allow the user to view her chest”; Note: there is data of the user from a front view, which is equivalent to the first perspective, and the user is displayed from a side view, which is equivalent to the second perspective). The rejection and explanation for claim 4 above also applies to its corresponding CRM claim 26 and method claim 45.
Regarding claim 5, Halevy in view of Aragones teaches the computer system of claim 63. Halevy further teaches wherein: displaying the virtual avatar includes displaying the virtual avatar at a second size, different from the first size (Paragraph 0135, 0137 – “a representation 1302 of the subject 1212 as well as a skeleton 1304 (or other grouping) of detected body parts of the subject 1212 is displayed…Editor system 1206 may allow an editor to rotate the point of view or the zoom level of the representation 1302 and/or skeleton 1304…As another example, a scroll of a mouse wheel or other input device manipulation could change the zoom level. The change in the point of view or zoom level may be performed at any suitable time, such as before the playback begins, during playback, or while the playback is paused”; Note: the representation is equivalent to the avatar in this case, and it is displayed with a changed zoom level, meaning it is at a second size different from the initial first size). The rejection and explanation for claim 5 above also applies to its corresponding CRM claim 27 and method claim 46.
Claim 7 (and corresponding claims 28 and 47) is rejected under 35 U.S.C. 103 as being unpatentable over Halevy in view of Wells et al. (AU 2020203489 A1), hereinafter Wells.
Regarding claim 7, Halevy teaches the computer system of claim 1. Halevy does not teach wherein displaying the visual feedback includes displaying an augmented reality environment that includes one or more virtual objects, wherein the one or more virtual objects are animated to provide guidance on a first set of target movements that should be performed by the user of the computer system. However, Wells teaches wherein displaying the visual feedback includes displaying an augmented reality environment that includes one or more virtual objects (Paragraph 0042 – “a movement capture apparatus may be used to capture video and the start bubble 308 and the end bubble 314 may be automatically added to the captured video. The start bubble 308 or the end bubble 314 may be color coordinated, such as a green start bubble 308 indicating the starting point for a movement and a red end bubble 314 indicating the ending point for the movement”; Note: the augmented reality environment comprises the captured video and augmented bubbles. The bubbles are virtual objects), wherein the one or more virtual objects are animated to provide guidance on a first set of target movements that should be performed by the user of the computer system (Paragraph 0042 – “the colors may change as the movement is in progress. For example, the start bubble 308 may be a first color and then change as a specific body part or object is placed within the start bubble 308, upon which the start bubble 308 may change to a second color to indicate the placement is correct. The edges 318 or 320 may similarly change color based on whether movement is within or outside the edges 318 or 320. The end bubble 314 may similarly change color when the body part or object is placed in the end bubble 314. In an example, the start bubble 308 or the end bubble 314 may be "popped" (e.g., animated to disappear), such as when a user places a body part or object (e.g., a predetermined specific body part or object) within the start bubble 308 or the end bubble 314”; Note: the bubbles are animated to indicate if the user’s movement is correct). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halevy to incorporate the teachings of Wells to display an augmented reality with target movement guidance for the benefit of helping the user visualize their movement and perform it correctly based on the given guidance. The rejection and explanation for claim 7 above also applies to its corresponding CRM claim 28 and method claim 47.
Claims 8-9 (and corresponding claims 29-30 and 48-49) are rejected under 35 U.S.C. 103 as being unpatentable over Halevy in view of Wells and Garofalo et al. (US 20200105041 A1), hereinafter Garofalo.
Regarding claim 8, Halevy in view of Wells teaches the computer system of claim 7. Halevy does not teach wherein displaying the augmented reality environment includes displaying a representation of at least a portion of the captured image data that corresponds to the user of the computer system overlaid on a virtual background. However, Garofalo teaches wherein displaying the augmented reality environment includes displaying a representation of at least a portion of the captured image data that corresponds to the user of the computer system overlaid on a virtual background (Paragraph 0081 – “Referring now to FIG. 3A, it shows an exemplary display a user 300 may see a live avatar 302 is inserted into a display video. During the playback of an instructional video by a chosen teacher 304 in the PIVOT Yoga App, there are some extra spaces on one side of a frame. Into the extra space, the live avatar 302 representing the user 300 may be inserted as a comparison to the chosen teacher 304”; Note: the avatar is a representation of the user. The avatar is overlaid on the background of an instructional video). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halevy to incorporate the teachings of Garofalo to overlay the user on a virtual background for the benefit of ridding of background distractions so that the user can focus on their own body and movement. Additionally, like in Garofalo, the user can be overlaid on a virtual background with an instructor, which can make the experience more immersive and resemble an exercise class. The rejection and explanation for claim 8 above also applies to its corresponding CRM claim 29 and method claim 48.
Regarding claim 9, Halevy in view of Wells teaches the computer system of claim 7. Halevy does not teach wherein: data corresponding to the user of the computer system in the captured image data includes data of the user of the computer system captured from a first perspective; and displaying the augmented reality environment includes displaying the augmented reality environment from a second perspective, different from the first perspective. However, Garofalo teaches wherein: data corresponding to the user of the computer system in the captured image data includes data of the user of the computer system captured from a first perspective (Paragraph 0081 – “Referring now to FIG. 3A, it shows an exemplary display a user 300 may see a live avatar 302 is inserted into a display video. During the playback of an instructional video by a chosen teacher 304 in the PIVOT Yoga App, there are some extra spaces on one side of a frame. Into the extra space, the live avatar 302 representing the user 300 may be inserted as a comparison to the chosen teacher 304. For example, as the user raises his left arm, the corresponding avatar on the screen raises its left arm as well…one or more cameras are used to capture the pose of the user, images from the cameras are analyzed to derive a pose of the user, where the derived pose is used to render the avatar”); and displaying the augmented reality environment includes displaying the augmented reality environment from a second perspective, different from the first perspective (Fig. 3B, 3C, Paragraph 0081-0082 – “As different camera angles come into view, the avatar, or appropriate portion of the avatar in the event of a close-up, will automatically move into correct position in view of the teacher… the view on the display may change”; Note: the display may show the avatar from a view different from the original one where the user pose was captured. Fig. 3B and 3C show the different possible perspectives). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halevy to incorporate the teachings of Garofalo to display the augmented reality environment in a perspective different from the originally captured perspective for the benefit of having more view options, which can give the user a better understanding of how they are positioned and help them improve their poses and movement. The rejection and explanation for claim 9 above also applies to its corresponding CRM claim 30 and method claim 49.
Claim 17 (and corresponding claims 38 and 57) is rejected under 35 U.S.C. 103 as being unpatentable over Halevy in view of Aragones et al. (US 20230343450 A1), hereinafter Aragones 2.
Regarding claim 17, Halevy teaches the computer system of claim 1. Halevy does not teach the one or more programs further including instructions for: in accordance with a determination that the physical movements meet a second set of one or more movement criteria, displaying, via the display generation component, guidance on a third set of target movements that should be performed by the user of the computer system; and in accordance with a determination that the physical movements do not meet the second set of one or more movement criteria, forgoing displaying the guidance on the third set of target movements that should be performed by the user of the computer system. However, Aragones 2 teaches in accordance with a determination that the physical movements meet a second set of one or more movement criteria, displaying, via the display generation component, guidance on a third set of target movements that should be performed by the user of the computer system (Fig. 39, Paragraph 0037, 0120 – “FIG. 39 illustrates an example graphical user interface providing a demonstration of a first exercise drill…Some users may adapt at different intervals and the program may be structure to account for these differences. Some programs may also include shorter or longer progressions and the progressions may be determined by analyzing the performance of a user. The computer 102 prompts the user to learn proper technique which leads to progression. Challenges are increased over time by adding repetitions and equipment. As the user conquers a workout, they are given a new routine to learn, providing an intrinsic reward”; Note: Challenges are movement criteria, in this case. When challenges are met, a new workout is displayed to the user. Fig. 39 shows an example of guidance for the new workout, which is equivalent to the third set of target movements); and in accordance with a determination that the physical movements do not meet the second set of one or more movement criteria, forgoing displaying the guidance on the third set of target movements that should be performed by the user of the computer system (Paragraph 0120 – “Some users may adapt at different intervals and the program may be structure to account for these differences. Some programs may also include shorter or longer progressions and the progressions may be determined by analyzing the performance of a user. The computer 102 prompts the user to learn proper technique which leads to progression. Challenges are increased over time by adding repetitions and equipment. As the user conquers a workout, they are given a new routine to learn, providing an intrinsic reward”; Note: Challenges are movement criteria, in this case. The new workout, which is equivalent to the third set of target movements, is only shown if the challenge or previous workout was completed). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halevy to incorporate the teachings of Aragones 2 to only display further target movements if the movement criteria were met for the benefit of pacing the user and ensuring proper progression. In other words, the user should not move on to different exercises if they are not ready for them yet, especially for safety and health reasons. The rejection and explanation for claim 17 above also applies to its corresponding CRM claim 38 and method claim 57.
Claim 21 (and corresponding claims 42 and 61) is rejected under 35 U.S.C. 103 as being unpatentable over Halevy in view of Dye et al. (KR 20190142397 A), hereinafter Dye.
Regarding claim 21, Halevy teaches the computer system of claim 1. Halevy does not teach the one or more programs further including instructions for: while capturing the image data: in accordance with a determination that a set of one or more image capture correction criteria are met, displaying, via the display generation component, guidance for adjusting image capture; and in accordance with a determination that the set of one or more image capture correction criteria are not met, forgoing displaying the guidance for adjusting image capture. However, Dye teaches while capturing the image data: in accordance with a determination that a set of one or more image capture correction criteria are met, displaying, via the display generation component, guidance for adjusting image capture (Paragraph 0436 – “the alignment guidance display criteria include a requirement that the orientation of the device does not change by more than a threshold amount over a threshold amount of time while the relative difference (e.g., 1412) between the orientation of the focal plane (e.g., 1406) of the camera (e.g., 1402) and the predetermined orientation (e.g., 1408) remains within their respective alignment thresholds so that the alignment guidance display criteria are satisfied (e.g., the alignment guidance display criteria require that the device remain in a substantially stationary orientation relative to the predetermined orientation (e.g., 1408) while within a range of orientations corresponding to displaying the alignment guidance (e.g., 1414A and 1414B) so that the alignment guidance (e.g., 1414A and 1414B) is displayed”; Note: the guidance is displayed if the alignment guidance display criteria is met, which is equivalent to the image capture correction criteria); and in accordance with a determination that the set of one or more image capture correction criteria are not met, forgoing displaying the guidance for adjusting image capture (Paragraph 0437 – “if it is determined that alignment guidance display criteria are not met (e.g., the angle between a predetermined plane and a focal plane (e.g., 1406) of a camera (e.g., 1402) is greater than a threshold) (e.g., the camera lens is not sufficiently parallel to the horizontal/vertical plane), the electronic device (e.g., 1400) withholds displaying the alignment guidance (e.g., 1414A and 1414B) within the camera viewfinder (e.g., 1410)”; Note: the alignment guidance display criteria is equivalent to the image capture correction criteria). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halevy to incorporate the teachings of Dye to determine that a set of image capture correction criteria are met for the benefit of being able to detect and troubleshoot alignment or orientation problems. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Halevy to incorporate the teachings of Dye to only display guidance for image capture adjustment if the criteria were met for the benefit of ensuring that any alignment or orientation problems are fixed prior to further moving the camera to different poses, which would help with optimal image capture. The rejection and explanation for claim 21 above also applies to its corresponding CRM claim 42 and method claim 61.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Calderon (US 20220203168 A1) teaches a method of training and using a predictive model to provide exercise feedback and evaluate a user’s exercise performance. Augustin et al. (US 20230356033 A1) teaches a method of dynamically tracking user movement and displaying machine-generated feedback.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE HAU MA whose telephone number is (571)272-2187. The examiner can normally be reached M-Th 7-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE HAU MA/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights