DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice of Reply
This communication is responsive to the amendment(s) and/or argument(s) filed 2/23/26. The previous ground(s) of objection and/or rejection is/are withdrawn. The following new and/or reiterated ground(s) of rejection is/are set forth hereinbelow.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 46 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 46 positively recites “wherein the at least one processor being configured to: identify, as the orientation of the body site, an orientation that can be represented with use of a line segment connecting two points of the body site, wherein the line segment is parallel to a horizontal plane if the posture is in a normal state; and/or an orientation that can be represented with use of a normal line of a line segment connecting two points of the body site, wherein the line segment is perpendicular to a horizontal plane if the posture is in a normal state.”. Claim 46 fails to further limit independent claim 28 from which it directly depends because in either required instance of “identify, as the orientation of the body site, an orientation that can be represented” or “orientation that can be represented”, since the claim merely requires that it can, it does not have to and is thus not necessarily required. Since both phrases are optional, claim 46 fails to further limit claim 28.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 28-33 and 46 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Flury et al. (US 8,702,485 B2, hereinafter Flury).
For claim 28, Flury discloses a posture assessment apparatus (Figs 1A-5) (Col 8-30), comprising:
at least one processor (125) (Cols 29-30) and a memory (135, 140, 145) (Cols 29-30) storing instructions executable by the at least one processor (Cols 29-30), the at least one processor being configured to:
identify, as an orientation identifier (105) (Fig 1A), an orientation of a body site (Figs 1B-5) (Col 8-30, especially Col 12-14 and Col 28), based on at least two points of a body site of an assessed person wherein the orientation is used to quantitatively assess a posture state of the assessed person (Figs 1B-5) (Col 8-30, especially Col 12-14 and comparison of the input performance points during postured dance moves to the target performance) (see also Col 28 for posture assessment variations of yoga or martial arts), and
cause display onto a display screen (Col 30), as an orientation displayer (130), information regarding the orientation that has been identified in association with at least one point of the body site of the assessed person (Figs 1B-5) (Col 8-30, especially Col 12-14 and output regarding target dance moves posture assessment versus input performance, and especially Col 15-17 and target dance moves posture assessment feedback compared to target performance),
wherein the point that has been associated with the information regarding the orientation is different from any of the points of identifying the orientation of the body site (Figs 1B-5) (Col 8-30, especially Cols 17-19 regarding overcoming occlusion limitations and Cols 28-29 regarding filters with hand held sensor input), and
wherein the identified orientation and displayed information are configured to allow evaluation of a balance of muscles of the assessed person and to support provision of an exercise menu for the assessed person (Figs 1B-5) (Col 8-30, especially Cols 19-21 detailing training and fitness modes)
For claim 29, Flury discloses the posture assessment apparatus according to claim 28, wherein the at least one processor identifies the orientation, which is identified by a normal line of aline segment connecting the two points of identifying the orientation of the body site as the orientation of the body site (Figs 1B-5) (Col 8-30, especially Cols 17-19 regarding overcoming occlusion limitations and Cols 28-29 regarding filters with hand held sensor input).
For claim 30, Flury discloses the posture assessment apparatus according to claim 28, wherein the points that have been associated with the information regarding the orientation of the body site are points to be aligned on a straight line, in a case of a the assessed person in a normal posture (Figs 1B-5) Figs 1B-5) (Col 8-30, especially Col 12-14 and comparison of the input performance points during postured dance moves to the target performance) (see also Col 28 for posture assessment variations of yoga or martial arts).
For claim 31, Flury discloses the posture assessment apparatus according to claim 28, wherein the orientation identifier and/or the orientation displayer is performed based on images that have been captured from the front direction, the lateral side direction, and the rear direction or images that have been captured from the front direction, the lateral side direction, and the top direction (Figs 1B-5) (Col 8-30); the points that have been associated with the information regarding the orientation of the body site are different from any of the points of identifying the orientation of the body site (Figs 1B-5) (Col 8-30); and the points that have been associated with the information regarding the orientation of the body site are points to be aligned on a straight line, in a case of the assessed person in a normal posture Figs 1B-5) (Col 8-30, especially Col 12-14 and comparison of the input performance points during rotational postured dance moves to the target performance) (see also Col 28 for posture assessment variations of yoga or martial arts).
For claim 32, Flury discloses the posture assessment apparatus according to claim 28, wherein the orientation identifier and/or the orientation displayer is performed based on images that have been captured from the front direction, the lateral side direction, and the rear direction or images that have been captured from the front direction, the lateral side direction, and the top direction Figs 1B-5) (Col 8-30, especially Col 12-14 and comparison of the input performance points during rotational postured dance moves to the target performance) (see also Col 28 for posture assessment variations of yoga or martial arts).
For claim 33, Flury discloses the posture assessment apparatus according to claim 28, wherein the at least one processor being further configured to
identify, as a position identifier (115) (Fig 1A), the position of aplurality of body sites (Figs 1B-5) (Col 8-30, especially Cols 17-19 regarding overcoming occlusion limitations and Cols 28-29 regarding filters with hand held sensor input);
change, as a virtual skeleton change part (digital calculated version of 530) (Fig 5), a virtual skeleton set in a virtual model in accordance with the orientation that has been identified by the orientation identifier and/or the position that has been identified by the position identifier (Figs 1B-5) (Col 8-30, especially Cols 17-19 regarding overcoming occlusion limitations and Cols 28-29 regarding filters with hand held sensor input); and
render and output, as a virtual model displayer (displayed version of 530) (Fig 5), a virtual model onto the display screen in accordance with the virtual skeleton that has been changed, and wherein the virtual model is displayed as a two-dimensional image or a three-dimensional image (Figs 1B-5) (Col 8-30, especially Cols 23).
For claim 46, Flury discloses the posture assessment apparatus according to claim 28, wherein the at least one processor being configured to: identify, as the orientation of the body site, an orientation that can be represented with use of a line segment connecting two points of the body site (Figs 1B-5) (Col 8-30, especially Cols 17-19 regarding overcoming occlusion limitations and Cols 28-29 regarding filters with hand held sensor input, especially Col 12-14 and comparison of the input performance points during postured dance moves to the target performance) (see also Col 28 for posture assessment variations of yoga or martial arts), wherein the line segment is parallel to a horizontal plane if the posture is in a normal state ((Figs 1B-5) (Col 8-30, especially Cols 17-19 regarding overcoming occlusion limitations and Cols 28-29 regarding filters with hand held sensor input); and/or an orientation that can be represented with use of a normal line of a line segment connecting two points of the body site, wherein the line segment is perpendicular to a horizontal plane if the posture is in a normal state (Figs 1B-5) (Col 8-30, especially Cols 17-19 regarding overcoming occlusion limitations and Cols 28-29 regarding filters with hand held sensor input).
Response to Arguments
Applicant’s arguments, see pages 9-13, filed 2/23/26, with respect to the amended claims obviating the previous 112 and 101 rejections have been fully considered and are persuasive. The previous 112 and 101 rejections of the claims have been withdrawn.
Applicant's arguments, see pages 13-15, filed 2/23/26 with respect to the amended calims overcoming the anticipatory rejection of Flury have been fully considered but they are not persuasive..
Applicant argues the following:
For example, paragraph [0056] of Flury does not disclose displaying the orientation (vector) of a body site on the screen. Paragraph [0097] does not display the orientation of the wrist (the vector from the shoulder joint to the wrist) determined in [0056]. Furthermore, even if [0097] of Flury were to display the orientation of the wrist determined in [0056], it would amount to displaying the orientation of the wrist using the two points (the shoulder joint and the wrist) used to identify the orientation of the wrist. Therefore, it would not satisfy the constituent feature in Claim 28 that "a point associated with information about the orientation of the body site is different from any of the plurality of points based on which the orientation of the body site is identified."
Moreover, Applicant submits that the newly amended features added to independent claim 28 further distinguish the claimed invention from Flury. For example, Flury is directed to conventional methods, apparatuses, programs, and systems for scoring a player performance comprising one or more poses in a dance-based video game. A performance 3D skeleton is received that indicates a pose of the player.
Flury's core purpose appears to be measurement and visualization, not posture-state grasping for exercise therapy. Flury is directed to: capturing body pose, computing skeletal orientation, and displaying a virtual skeleton. See, e.g., cols. 8-30 of Flury. By contrast, the amended claimed invention is directed to: quantitatively grasping a posture state, accurately assessing posture using a simple method, deriving muscle balance, and supporting provision of an appropriate exercise menu.
In response the Examiner respectfully disagrees and notes the following:
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “displaying the orientation”, “(vector)”, “posture-state grasping”, “exercise therapy”, “quantitatively grasping a posture state”, “accurately assessing posture using a simple method”, and/or “deriving muscle balance”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
There mere fact that Flury is directed scoring player performance on dance poses during a video game does not preclude it from fairly and reasonably being considered as a multi-modal posture assessment with muscle balance evaluation(s) and exercise component(s), particularly since dance/yoga/martial arts all include various postures being assessed whether during a performance-based scoring session, a training session, or a fitness (exercise) mode session (see citations set forth hereinabove for reiteration thereof).
Further, and in support of above, Flury explicitly states inter alia the following:
Cols 18-19
In some implementations, weighting is based on the "confidence" that the camera system may provide for detecting a joint's position. For example, in some versions of Project Natal, the camera system provides "tracked" positional information in the form of a position for a joint and a confidence level that the position is correct. When the joint is off-screen, Natal also provides an "inferred" position. When a joint's position is inferred, e.g., when the joint is clipped or occluded, neighboring joints can be examined to better assess where the inferred joint is. For example, if an elbow is raised above the user's ear, there are only a few possible locations of the user's wrist, e.g., straight up above the elbow, down near the user's chin, or somewhere in between. In these scenarios, because the object of the game is to be fun, the maximum positional window, e.g., 0 to d.sub.+outer, is widened so that the filtering is looser to allow for greater variation in positional differences. Additionally, the inner window of "perfect" position, zero to d.sub.+inner, may also be widened.
In some embodiments, the invention will suspend the game if too much of the skeleton is occluded or off-screen for more than a threshold amount of time, e.g., 10 second, or 6 beats, rather than continuing to reward the user for incorrect positioning.
To assist the user in completing moves correctly, per-limb feedback is given to the user when performing a move. In some embodiments, if the user is not satisfying a filter for a limb, the game renders a red outline around the on-screen dancer's corresponding limb to demonstrate to the user where they need to make an adjustment. In some embodiments, the per-limb feedback is on the mirror-image limb from the limb that is not satisfying the filter. For example, if the user is satisfying the filter for both feet, the hips, and the left arm, but not satisfying the filter for the right arm, the game renders a red outline around the on-screen dancer's left arm. This indicates to the user that his right arm is not correct, since the user is facing the on-screen dancer and mimicking the on-screen dancer in mirror image.
Other per-limb feedback is also possible. In some embodiments, an indicator such as a "phantom" limb is drawn in the target location. Alternatively or additionally, an indicator is anchored on the errant limb and its direction and length are based on the direction and degree of error in the user's limb position. For example, if the user's wrist is below the target location, the game draws an arrow starting from where the user's wrist is located in the input performance and ending where the on-screen dancer's wrist is in the target performance. Alternatively, in embodiments where a representation of what the user is doing is displayed on-screen, the arrow is drawn starting from the user representation's wrist. In some embodiments, the indicator persists until the user satisfies the filters for the target performance's arms. In some embodiments, the intensity, geometry, material, or color characteristic of the indicator may be changed based on the degree of error for that limb. For example, the color of the indicator may become a more saturated red if the error for a limb becomes greater. Other highlighting may also be used, as may verbal cues such as "get your <limbs> movin'" where <limbs> is any body zone that is not satisfying the filter.
Cols 28-29:
The examples given herein of a user satisfying a filter by completing a series of moves can be adapted to satisfy a "mirror mode" as well, where the user provides input that mirrors the target performance, e.g., providing input using a right hand when the target performance uses a left hand, providing right leg input when the target performance uses a left leg, and so forth.
Additionally, where a target performance skeleton is provided, it can be generated beforehand, or can be generated during execution of the game based on the motion capture data.
Any system that can detect movement can be used as long as positions of the scored joints can be determined in either two-dimensional space or three-dimensional space to create or simulate a skeleton. For two-dimensional implementations, scoring is typically adjusted to compare the projection of the target performance and the projection of the input performance onto a plane parallel to the screen. Although the system and technology has been described in terms of a camera input system like Natal, camera systems that utilizes sensors on the user's body, e.g., PLAYSTATION.RTM. Move, or systems that use sensors held in the user's hand, e.g., the NINTENDO.RTM. Wii, may also be utilized. In those implementations where only hand held sensors are utilized by the user, testing for leg input is ignored or not performed.
Although the embodiments described herein use dancing as an example, and the performance is typically accompanied by a song, the performance can also be movements that occur on a timeline with no musical accompaniment, e.g., a series of yoga poses, movements in a martial arts kata, or the like.
In some implementations, the mocap data is mapped to a skeleton similar to that used to reflect the user's input. Thus, the mocap data is used to generate an ideal skeleton that represents a performance of the dance routine in a format that is directly comparable to the skeleton representing the user's input. Then, during the game, as the user provides input, the user's skeleton is compared to the ideal skeleton, in effect normalizing the target input (the target performance) and actual inputs (the user's performance) to the same frame of reference, i.e., both performances are expressed in terms of the same skeleton-based technology.
In some embodiments, rather than matching position necessarily within a time window as described above, filter types are predefined and used to test user input. For example, proximity filters tests if a joint in a particular position, or close to a particular other joint, e.g., "are the left wrist and right wrist less than, greater than, or within a delta of a certain distance of one another. Another filter is a displacement filter which tests if a joint has moved a certain distance between times t.sub.0 and t.sub.n. Another example is the angle filter, which tests if a joint is at a particular angle from the origin. One or more of these filters is then hand-inserted (or "authored") into the timeline and bound to joints such that at a particular time, the condition is tested, e.g., "has the RIGHT WRIST moved from x.sub.0 to x.sub.n since I began tracking it?" would be a displacement filter. If the user's wrist had, the filter would be satisfied. Yet another filter is an acceleration filter which tests if a joint or bone has accelerated or decelerated between times t.sub.0 and t.sub.n. An acceleration filter can also test whether the magnitude of the acceleration matches a predetermined value.
In these embodiments, multiple filters can be overlaid on the timeline, and tested, in effect, simultaneously. An overall score for the frame is determined based on contributions from all of the active filters during a given frame. The filters can output a Boolean, and the score is computed from those. Or--in some implementations--the outputs are continuous, and the aggregate score is computed from those. Similar to the system described above, contributions from each active filter can be weighted differently in their contributions to the score. For Boolean filters, successfully completing 3 out of 5 filters gives the user a score of 0.6. In some implementations, each keyframe comparison gives a percentage credit for the move as a whole being correct. The user's score may be adjusted based on the aggregate score for a keyframe. Or the aggregate score for a keyframe may be quantized into groups, each group being compared to one or more thresholds, each group associated with a score that is added to the user's score. In any of these, if the user achieves a threshold score for a move, where if the user meets or exceeds the threshold, e.g., 80%, the user is considered to have successfully performed the move.
In some embodiments, execution of game software limits the game platform 120 to a particular purpose, e.g., playing the particular game. In these scenarios, the game platform 120 combined with the software, in effect, becomes a particular machine while the software is executing. In some embodiments, though other tasks may be performed while the software is running, execution of the software still limits the game platform 120 and may negatively impact performance of the other tasks. While the game software is executing, the game platform directs output related to the execution of the game software to a display, thereby controlling the operation of the display. The game platform 120 also can receive inputs provided by one or more users, perform operations and calculations on those inputs, and direct the display to depict a representation of the inputs received and other data such as results from the operations and calculations, thereby transforming the input received from the users into a visual representation of the input and/or the visual representation of an effect caused by the user.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jeffrey G. Hoekstra whose telephone number is (571)272-7232. The examiner can normally be reached Monday through Thursday from 5am-3pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles A. Marmor II can be reached at (571)272-4730. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Jeffrey G. Hoekstra
Primary Examiner
Art Unit 3791
/JEFFREY G. HOEKSTRA/ Primary Examiner, Art Unit 3791
Prosecution Insights