DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Application
Claims 9-16, 23, 24, 31, and 32 have been cancelled. Claims 1-8, 17-22, and 25-30 are currently pending in this application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/12/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 17, 18, 25, and 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (Hereafter, “Zhang”) [US 2022/0080260 A1] in view of GAO, Yi-chen et al. (Hereafter, “Gao”) [CN 111814772 A1].
In regards to claim 1, Zhang discloses a posture evaluation apparatus ([0005] methods and systems for enabling pose comparison [Fig. 2] user computing entity 200 [Fig. 3] management computing entity 300) comprising: a memory storing instructions ([0089] User computing entity 200 may also include volatile and/or non-volatile storage or memory 230. The volatile and non-volatile storage or memory may store an operating system 214, application software 216, data 218, databases, database instances, database management systems, programs, program modules, scripts, source code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like to implement the functions of user computing entity 200. [0103] instructions stored in volatile or non-volatile memory 330 and 340); and one or more processors configured to execute the instructions ([0103] processing unit 310 may be configured for a particular use or configured to execute instructions stored in volatile or non-volatile memory 330 and 340) to: extract, based on an image acquired by imaging a side surface of a body of a subject person [Fig. 12A] and position information about at least a cervical vertebrae, hip joints, and knee joints of the body on the image, a spine edge point cloud constituted of a predetermined number of points representing a spine shape on the image ([0142] At a step 1030, a frame of a user video is received on the user computing device, wherein the frame of the user video comprise a user. At step 1040, a user posture is extracted from the frame of the user video, by performing the machine learning-based computer vision algorithm on the frame of the user video, wherein the machine learning-based computer vision algorithm detects one or more body key points of the user in an image plane of the user video. At step 1050, a user feature is generated from the user posture.); calculate a feature value about at least a spine, based on the position information and the spine edge point cloud ([0144] For example, a posture feature vector may represent limb angles relative to the person's torso or relative to each other, as calculated based on estimated body key points. Another posture feature vector may indicate one or more alignment factors for the head, the spine (e.g., cervical, thoracic, and lumbar curvatures), the pelvis, lower body joints (e.g., hip, knee, ankle), as well as shoulder symmetry.); and estimate a state of at least the spine, based on the feature value ([0144] In some embodiments, for a given human posture or posture flow, a feature or posture feature is a numerical or quantitative characterization of the posture or posture flow, different from a simple scale-normalized version of the original posture or posture flow. Such posture features are suitable for characterizing static poses.).
Gao discloses a posture evaluation apparatus ([0007] a human posture assessment method, device, electronic device, and storage medium) comprising: a memory storing instructions ([0040] memory); and one or more processors configured to execute the instructions ([0040] an electronic device, including a processor, a communication interface, a memory, and a bus, wherein the processor, the communication interface, and the memory communicate with each other through the bus, and the processor can call logical commands in the memory to execute the steps of the method provided in the first aspect.) to: extract, based on an image acquired by imaging a side surface of a body of a subject person ([0009] Extract the human side skeleton and external contour from the human side image of the object to be evaluated. [0061] In order to assess their posture, it is necessary to first take a side view of them in a static standing position, that is, a side view of the subject to be evaluated.) and position information about at least a cervical vertebrae, hip joints, and knee joints of the body on the image ([0010] Determine the key points of the lateral skeleton on the side of the human body, and the key points of the lateral contour on the outer contour of the human body.), a spine edge point cloud constituted of a predetermined number of points representing a spine shape on the image ([Abstract] extracting the human body side framework and the human body outline from the human body side image of the object to be evaluated; determining the side framework key point on the human body side framework; and the side outline key point on the human body outer outline); calculate a feature value about at least a spine, based on the position information and the spine edge point cloud ([Abstract] based on the side frame key point and the side outline key point, determining the side bending angle of the object to be evaluated [0011] Based on the key points of the side skeleton and the key points of the side contour, the side curvature angle of the object to be evaluated is determined.); estimate a state of at least the spine, based on the feature value ([Abstract] based on the side bending angle, determining the attitude evaluation result of the object to be evaluated [0012] Based on the lateral bending angle, the posture evaluation result of the object to be evaluated is determined.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the determination of key points of the side skeleton of the human body as taught by Gao in order to determination the curvature of the skeleton and determination of the posture of the human body. The motivation behind this modification would have been to improve the comprehensiveness and accuracy of posture assessment [See Gao].
In regards to claim 2, the limitations of claim 1 have been addressed. Zhang fails to explicitly disclose the one or more processors configured to execute the instructions to: extract the position information from the image.
Gao discloses the one or more processors configured to execute the instructions to: extract the position information from the image ([0121] Specifically, the side view image of the human body is a depth image obtained using a depth camera, such as Kinect. After a depth camera captures a side view image of the human body, it can establish the coordinates of each joint of the human body based on the depth information in the side view image, thereby constructing the side view skeleton of the human body. Existing methods for human skeleton extraction are typically based on planar images of the human body, such as RGB images, and use trained neural network models to estimate key points. On the one hand, the above-mentioned joint estimation method can only predict the approximate position of each joint on the human skeleton, with a large error and low accuracy.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the teachings of Gao in order to improve the comprehensiveness and accuracy of posture assessment [See Gao].
Claim 17 lists all the same elements of claim 1, but in method form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 1 applies equally as well to claim 17.
Claim 18 lists all the same elements of claim 2, but in method form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 2 applies equally as well to claim 18.
Claim 25 lists all the same elements of claim 1, but in non-transitory computer-readable medium form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 1 applies equally as well to claim 25.
Claim 26 lists all the same elements of claim 2, but in non-transitory computer-readable medium form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 2 applies equally as well to claim 26.
Claim(s) 3-5, 7, 8, 19-21, and 27-29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gao in further view of TAKAGI et al. (Hereafter, “Takagi”) [US 2020/0155900 A1].
In regards to claim 3, the limitations of claim 1 have been addressed. Zhang discloses the one or more processors configured to execute the instructions to: ([0012] a reference feature generated from a frame of a reference video, wherein the frame of the reference video comprises a reference person, and wherein the reference feature is computed from a reference posture of the reference person in the frame of the reference video), wherein the one or more processors estimate a state of at least the spine, based on the type of the posture, the feature value calculated by the feature value calculating means, and the reference value ([0012] generating a first user feature from the first user posture; and determining an output score based on a first distance between the reference feature and the first user feature).
Takagi discloses the one or more processors configured to execute the instructions to: store, in association with each other, a type of a posture of the body and a reference value of the feature value for the type of the posture ([0120] Then, the trainer-side terminal 4 determines the posture pattern of the client from the plurality of posture patterns set in advance based on the assessment result of the objects acquired in step S33 (step S34).), wherein the one or more processors estimate a state of at least the spine, based on the type of the posture, the feature value calculated by the feature value calculating means, and the reference value ([0120] In step S34, it is preferable to select a feature exhibited in the posture of the client from a plurality of feature candidates set in advance based on the assessment result of the objects acquired in step S33, and to determine the posture pattern based on the selected feature, for example. [0121] Note that the posture pattern is classified based on types of the postures often observed in general in a prescribed steady state or moving state, for example. The posture pattern is preferable to be set for each type of assessment targets, such as front standing posture, side standing posture, front overhead squat posture, side overhead squat posture, and the like. Note that the feature and/or the posture pattern determined in step S34 are preferable to be used for determining the exercise menu in step S6 to be described later, for example.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the teachings of Takagi in order to improve the assessment of the posture [See Takagi].
In regards to claim 4, the limitations of claim 3 have been addressed. Zhang fails to explicitly disclose the one or more processors configured to execute the instructions to: determine a type of the posture, based on the position information.
Takagi discloses the one or more processors configured to execute the instructions to: determine a type of the posture, based on the position information ([0121] Note that the posture pattern is classified based on types of the postures often observed in general in a prescribed steady state or moving state, for example. The posture pattern is preferable to be set for each type of assessment targets, such as front standing posture, side standing posture, front overhead squat posture, side overhead squat posture, and the like. Note that the feature and/or the posture pattern determined in step S34 are preferable to be used for determining the exercise menu in step S6 to be described later, for example.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the determination of the posture type from the posture pattern as taught by Takagi in order to improve the assessment of the posture [See Takagi].
In regards to claim 5, the limitations of claim 1 have been addressed. Zhang fails to explicitly disclose the one or more processors configured to execute the instructions to: display the image and the state estimated.
Takagi discloses the one or more processors configured to execute the instructions to: display the image and the state estimated ([0131] Returning to the flowchart of FIG. 6, the trainer-side terminal 4 displays the image regarding the posture assessment made in step S3 (step S4). FIG. 11 is an example of the display screen displaying the image regarding the posture assessment that corresponds to at least one of the embodiments of the present invention. On the display screen of the trainer-side terminal 4, displayed are a posture type display section 401, a current image display section 402, a feature display section 403, a past image display section 404, an ideal target image/unideal target image display section 405, and a comment section 406. [0132] The posture type display section 401 is a section that displays the type of the posture as the assessment subject. In the example of FIG. 11, it is displayed as “front standing posture” and an image acquired by capturing a standing position from the front is the assessment subject.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the displaying of the image and posture type as taught by Takagi in order to improve the assessment of the posture [See Takagi].
In regards to claim 7, the limitations of claim 5 have been addressed. Zhang fails to explicitly disclose wherein, in a case where the one or more processors extract the spine edge point cloud, the one or more processors display, together with the image, the spine edge point cloud in such a way that the spine edge point cloud can be modified by a user.
Takagi discloses wherein, in a case where the one or more processors extract the spine edge point cloud, the one or more processors display, together with the image, the spine edge point cloud in such a way that the spine edge point cloud can be modified by a user ([0132] The posture type display section 401 is a section that displays the type of the posture as the assessment subject. In the example of FIG. 11, it is displayed as “front standing posture” and an image acquired by capturing a standing position from the front is the assessment subject. It is preferable for the posture type display section 401 to be able to change to other posture types by receiving input of touch operations, for example. [0133] The current image display section 402 is a section that displays a current image regarding the current posture of the client assessed in step S3. As the current image, it is preferable to be an image captured in step S2, an image of prescribed objects generated based on the image captured in step S2, or an image that integrally shows the image captured in step S2 and the prescribed objects generated based on the image, for example. In the example of FIG. 11, displayed in the current image display section 402 is the image that integrally shows the image captured in step S2 and the prescribed objects generated based on the image, and further displayed in the vicinity of each of the objects are inclination angles of each of the objects. With such configuration, the trainer and the client can easily grasp the current condition of the body of the client. Further, in the bottom of the current image display section 402, the posture pattern of the client determined in step 34 is displayed.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the displaying of the image and posture type and the user being able to modify the display and images as taught by Takagi in order to improve the assessment of the posture [See Takagi].
In regards to claim 8, the limitations of claim 5 have been addressed. Zhang fails to explicitly disclose the one or more processors configured to execute the instructions to: capture an image of the side surface of the body; and capture an image of another surface of the body, simultaneously with capturing the image of the side surface of the body, wherein the one or more processors display the image of the side surface of the body and the image of another surface of the body.
Gao discloses the one or more processors configured to execute the instructions to: capture an image of the side surface of the body ([0126] obtain side view of the human body of the object to be evaluated); and capture an image of another surface of the body, simultaneously with capturing the image of the side surface of the body ([0126] obtain frontal view of the human body of the object to be evaluated),
Takagi discloses that multiple images can be displayed on the display [Fig. 11].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the determination of key points of the side skeleton of the human body as taught by Gao in order to determination the curvature of the skeleton and determination of the posture of the human body. The motivation behind this modification would have been to improve the comprehensiveness and accuracy of posture assessment [See Gao]. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang and Gao with the displaying of different images of the subject and/or user as taught by Takagi in order to be able to compare the images [See Takagi].
Claim 19 lists all the same elements of claim 3, but in method form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 3 applies equally as well to claim 19.
Claim 20 lists all the same elements of claim 4, but in method form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 4 applies equally as well to claim 20.
Claim 21 lists all the same elements of claim 5, but in method form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 5 applies equally as well to claim 21.
Claim 27 lists all the same elements of claim 3, but in non-transitory computer-readable medium form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 3 applies equally as well to claim 27.
Claim 28 lists all the same elements of claim 4, but in non-transitory computer-readable medium form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 4 applies equally as well to claim 28.
Claim 29 lists all the same elements of claim 5, but in non-transitory computer-readable medium form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 5 applies equally as well to claim 29.
Claim(s) 6, 22, and 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Gao in further view of Takagi in even further view of NODA et al. (Hereafter, “Noda”) [US 2018/0153445 A1].
In regards to claim 6, the limitations of claim 5 have been addressed. Zhang fails to explicitly disclose wherein the one or more processors display the state by color-coding the spine edge point cloud.
Noda discloses wherein the one or more processors display the state by color-coding the spine edge point cloud ([0069] The output processing unit 14 outputs an angle calculated by the calculation unit 12 in a state corresponding to a determination result regarding whether or not the posture of the test subject is proper. For example, the output processing unit 14 outputs the angle calculated by the calculation unit 12 with a color corresponding to the determination result of the determination unit 13. In a case where the determination unit 13 determines that the posture of the test subject is improper, the output processing unit 14 outputs an angle with a black color, the angle being calculated from framework data acquired at the same time as framework data used for the determination. In a case where it is determined that the posture is improper, the output processing unit 14 outputs an angle at the time with a red color. As another example, in a case where it is determined that the posture is improper, the output processing unit 14 may output an angle by flickering the angle, may change a background color of the angle, or may output a predetermined voice together with the output of a display of the angle.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Zhang with the display of the determination of the posture using colors as taught by Noda in order to improve the level of accuracy and display of the posture determination [See Noda].
Claim 22 lists all the same elements of claim 6, but in method form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 6 applies equally as well to claim 22.
Claim 30 lists all the same elements of claim 6, but in non-transitory computer-readable medium form rather than apparatus form. Therefore, the supporting rationale of the rejection to claim 6 applies equally as well to claim 30.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kaitlin A Retallick whose telephone number is (571)270-3841. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Kelley can be reached at (571) 272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KAITLIN A RETALLICK/Primary Examiner, Art Unit 2482
Prosecution Insights