DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
This application claims benefit to the U.S. Provisional Application Serial No.
63/424,431, filed on 11/10/2022, which is hereby incorporated by reference herein in its entirety.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e. a law of nature, a natural phenomenon, or an abstract idea) without significantly more.Step 1 – Statutory Categories of Invention:
Claims 1-18 are drawn to a method, which are statutory categories of invention.Step 2A – Judicial Exception Analysis, Prong 1:
Independent claim 1 recites a method comprising providing one or more images of the infant; selecting at least two geometric symmetry measures of the infant indicative of the disease or condition; analyzing, using both an infant face landmark estimation model and an infant body landmark estimation model, the one or more images to produce a plurality of landmark values corresponding to the geometric symmetry measures; determining a geometry of one or more facial structures of the infant and/or a 3D body pose of the infant based on the plurality of landmark values; and quantifying, based on the geometry of the one or more facial structures of the infant and/or the 3D body pose of the infant, a symmetry of the geometric symmetry measures.
These steps amount to certain methods of organizing human activity which includes functions relating to managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) (MPEP § 2106.04(a)(2)(II)(C) citing the abstract idea grouping for methods of organizing human activity for managing personal behavior or relationships or interactions between people – also note MPEP § 2106.04(a)(2)(II) stating certain activity between a person and a computer may fall within the “certain methods of organizing human activity” grouping).
Step 2A – Judicial Exception Analysis, Prong 2:
This judicial exception is not integrated into a practical application because the additional elements within the claims only amount to instructions to implement the judicial exception using a computer [MPEP 2106.05(f)].
Claim 1 recites a computer vision system.
These elements are recited at a high-level of generality such that it amounts to mere instructions to apply the exception because this is an example of applying the
abstract idea by use of general-purpose computer which does not integrate the abstract
idea into a practical application.
The above claims, as a whole, are therefore directed to an abstract idea.Step 2B – Additional Elements that Amount to Significantly More: The present claims do not include additional elements that are sufficient to amount to more than the abstract idea because the additional elements or combination of elements amount to no more than a recitation of instructions to implement the abstract idea on a computer.
As discussed above with the respect to integration of the abstract idea into a
practical application, the additional element of a computer vision system to perform the method of the invention amounts to no more than mere instructions to apply the exception using a generic computing component (MPEP 2016.05(f) see for additional guidance on the “mere instructions to apply an exception”).
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. Their collective functions merely provide conventional computer implementation.
For the reasons stated, these claims are consequently rejected under 35 U.S.C. § 101.
Analysis of the Dependent Claims
Dependent claim 2 recites wherein the disease or condition is torticollis, such as congenital muscular torticollis (CMT), autism spectrum disorder (ASD), or cerebral palsy (CP).
Dependent claim 3 recites wherein the step of quantifying comprises determining bilateral postural asymmetry.
Dependent claim 4 recites wherein the step of quantifying further comprises using a symmetry classifier to assess, based on body joint angles obtained from the 3D pose determination, one or more symmetry ratings corresponding to the geometric symmetry measures.
Dependent claim 10 recites wherein the geometric symmetry measures include one or more of orbit slopes angle, relative face size, face angle, gaze angle, translational deformity, habitual head deviation, or combinations thereof.
Dependent claim 11 recites wherein the step of analyzing further comprises producing, by the infant face landmark estimation model, produce 68 landmark values.
Dependent claim 12 recites wherein the step of analyzing further comprises producing, by the infant body landmark estimation model, a plurality of landmark values corresponding to one or more of an upper arm, a lower arm, an upper leg, a lower leg, or combinations thereof of the infant.
Dependent claim 13 recites wherein the plurality of landmark values produced by the infant body landmark estimation model correspond to one or more pairs of limbs of the infant, the one or more pairs of limbs of the infant including one or more of an upper right arm and upper left arm pair, a lower right arm and lower left arm pair, an upper right leg and upper left leg pair, a lower right leg and lower left leg pair, or combinations thereof.
Dependent claim 14 recites wherein the step of quantifying a symmetry of the of the geometric symmetry measures further comprises determining an angle difference across each of the one or more pairs of limbs.
Dependent claim 15 recites further comprising comparing the angle difference for each of the one or more pairs of limbs to a corresponding threshold angle.
Dependent claim 16 recites further comprising classifying each of the pairs of limbs as symmetrical or asymmetrical, wherein; a symmetrical classification indicates that a corresponding one of the pairs of limbs has a determined angle difference within the corresponding angle threshold; and an asymmetrical classification indicates that the corresponding one of the pairs of limbs has a determined angle difference exceeding the corresponding angle threshold.
Dependent claim 17 recites further comprising assigning each of the pairs of limbs to an angle class.
Dependent claim 18 recites wherein the angle classes include one or more of <30°, ≥ 30°, <60°, ≥60°, 30°-59°, or combinations thereof.
Each of these steps of the preceding dependent claims 2-4, 10-18 only serve to further limit or specify the features of independent claims 1 accordingly, and hence are nonetheless directed towards fundamentally the same abstract idea as the independent claim. None of the preceding claims recite any further additional elements to be further considered in Step 2A Prong 2 and Step 2B.
Dependent claim 5 recites further comprising aggregating, using a Bayesian estimator, a plurality of annotated symmetry ratings corresponding to the one or more symmetry ratings to establish one or more aggregated Bayesian ground truths.
Dependent claim 6 recites further comprising comparing each of the symmetry ratings to a corresponding one of the aggregated Bayesian ground truths to calibrate the computer vision system.
Dependent claim 7 recites determining, by a maximum-a-posteriori (MAP) estimator, a performance standard applicable to the annotated symmetry ratings; and evaluating, by applying an expectation maximization algorithm, a performance relative to the performance standard of a rater associated with each of the annotated symmetry ratings.
Dependent claim 8 recites wherein the step of aggregating, by the Bayesian estimator, further comprises weighting each of the annotated symmetry ratings according to the evaluated rater performance.
Dependent claim 9 recites wherein the evaluated performance is at least partially determined according to an average Cohen's x agreement between each rater and the other raters.
Claims 5-9 recite steps that amount to a mathematical concept which includes mathematical relationships, mathematical formulas or equations, and mathematical calculations. The mathematical concept need not be expressed in mathematical symbols but not merely limitations that are based on or involve a mathematical concept (MPEP § 2106.04(a)(2)(I)(C) citing the abstract idea grouping for mathematical concepts for mathematical formulas or equations).
Claims 4, 5, 8 and 9 recite further additional elements beyond those disclosed in the independent claim.
Dependent claim 4 recites using a symmetry classifier.
Dependent claim 5 and 8 recite using a Bayesian estimator.
Dependent claim 7 recites a maximum-a-posteriori (MAP) estimator and applying an expectation maximization algorithm.
The system classifier, Bayesian estimator, a maximum-a-posteriori (MAP) estimator, and an expectation maximization algorithm are recited as a tools to apply data to an algorithm and report the results (MPEP § 2106.05(f)(2) see case involving a commonplace business method or mathematical algorithm being applied on a general-purpose computer within the “Other examples.. i.”) amounting to instruction to implement the abstract idea using a general-purpose computer. Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 134 S. Ct. 2347, 1357 (2014).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4-9 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recites the limitation "from the 3D pose determination" in Line 3-4. There is insufficient antecedent basis for this limitation in the claim. Due to Claim 4 being dependent on Claim 1 which states that determining… a geometry of one or more facial structures of the infant and/ or a 3D body pose. Claim 4 lack antecedent basis since Claim 1 does not require the 3D body pose to satisfy the determining the geometry limitation, however to satisfy Claim 4 the results of determining the geometry in Claim 1 would need to be the 3D body pose.
As per Claims 5-9, the claims depend on Claim 4 and do not remedy the written description requirement issues of Claim 4. As dependent claims inherit the deficiencies of the claims they depend on, they are also rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, and 10-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Passalis ("Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition," Oct. 2011) in view of Zhou (US 20130182008 A1) in view of Sarmiento (Behavioral Phenotyping for Autism Spectrum Disorder Biomarkers Using Computer Vision (2020)).
REGARDING CLAIM 1
Passalis teaches a method of screening, diagnosing, or monitoring a disease or condition in a human infant,
determining, by the computer vision system, a geometry of one or more facial structures of the infant and/or a 3D body pose of the infant based on the plurality of landmark values; ([Pg. 1 Introduction] Initial registration with the Annotated Face Model (AFM) [2] using the detected landmarks allows the pose estimation of each facial scan. Then, the AFM is fitted to the facial scan using a subdivision-based deformable model framework that is extended to allow symmetric fitting. The symmetric fitting alleviates the missing data problem allowing the creation of geometry images that are pose invariant. [Pg. 3 3 UR3D-S: A Symmetric Face Recognition Method] Step4. Symmetric Deformable Model Fitting: The AFM is fitted to the data using facial symmetry. The fitted model is then converted to a geometry image and a normal image. For the purposes of examination, the prior art is made in view of determining, by the computer vision system, a geometry of one or more facial structures of the infant.)
and quantifying, based on the geometry of the one or more facial structures of the infant and/or the 3D body pose of the infant, a symmetry of the geometric symmetry measures.( [Pg. 1 Introduction] Initial registration with the Annotated Face Model (AFM) [2] using the detected landmarks allows the pose estimation of each facial scan. Then, the AFM is fitted to the facial scan using a subdivision-based deformable model framework that is extended to allow symmetric fitting. The symmetric fitting alleviates the missing data problem allowing the creation of geometry images that are pose invariant. [Pg. 4 3 UR3D-S: A Symmetric Face Recognition Method] Step 5. Wavelet Analysis: A wavelet transform is applied on the geometry and normal images and the wavelet coefficients are stored as a biometric signature. For the purposes of examination, the prior art is made in view of determining, by the computer vision system, a geometry of one or more facial structures of the infant.)
Passalis does not explicitly teach, however Zhou teaches
the method comprising providing one or more images of the infant; ([Para. 0009] one or more medical images are received. The medical images include first and second regions, wherein the first region is substantially symmetric to the second region.)
selecting at least two geometric symmetry measures of the infant indicative of the disease or condition ([Para. 0005] Digital medical images are constructed using raw image data…. and are processed using medical image recognition techniques to determine the presence of anatomical structures such as cysts, tumors, polyps, etc…. An automatic technique should point out anatomical features in the selected regions of an image to a doctor for further diagnosis of any disease or condition. [Para. 0025] Gross asymmetry is found only in some internal anatomical features, such as the heart, liver, spleen, colon, etc. The human brain, for instance, exhibits bilateral (or reflection) symmetry and can be divided by a mid-sagittal plane into two halves that are mirror images of each other. Other anatomical features, such as the femur head, may exhibit radial (or rotational) symmetry in which components are arranged regularly around a central axis. Other types of symmetry may also be exhibited. In addition, it should be noted that the anatomical feature may exhibit more than one type of symmetry (e.g., both bilateral and rotational symmetry). Such symmetry may be exhibited by the anatomical structure itself or with respect to another anatomical structure. [Para. 0034] The one or more medical images may include at least first and second substantially symmetric regions about a point, line or plane of symmetry. FIG. 3a shows exemplary images 302a-c of anatomical structures that exhibit marked bilateral symmetry across the plane of symmetry 340a-c (e.g., mid-sagittal plane). [Para. 0036] FIG. 3b shows exemplary images (330 and 302d) of a femur head 350 that exhibits marked rotational symmetry.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis and incorporate providing medical images for symmetry-based visualization as taught by Zhou, with the motivation of enhancing detection of anomalies in digital or digitized medical images (Zhou Para. 0002).
Passalis/ Zhou do not explicitly teach, however Sarmiento teaches
analyzing, using both an infant face landmark estimation model and an infant body landmark estimation model of a computer vision system, the one or more images to produce a plurality of landmark values corresponding to the geometric symmetry measures; ([Pg. 2 II. Methods A. Attention Tracking 1) Head pose estimation] We collect head movement data using yaw (lateral movement), roll (lateral movement), and pitch (vertical movement). For each video frame (i.e. images), the head-pose estimation is predicted by the detection of 2D facial landmarks to estimate the 3D pose as seen in Image 1. We use dlib, a toolkit containing machine learning algorithms and tools (i.e. infant face landmark estimation model) for finding the face and 68 facial landmarks necessary for predicting the head pose using OpenCV. For each frame, we detect the 2D facial landmarks, identify image locations predict 3D pose, and gather roll, pitch, and yaw measurements as a time-series data. [Pg. 2 II. Methods A. Attention Tracking 1) Body pose estimation] Estimate 2D body pose and estimate arm and shoulder angles. In order to measure these angles, the keypoints or landmarks of the body needed to be identified first. We use a pre-trained model created by [12] trained using the COCO Human Pose Dataset [13]. Cao et al. [12] created a real-time multi-person 2D pose estimation method (i.e. an infant body landmark estimation model) by training a very deep Neural networks that takes a colored image of size w x h and produces the 2D location of keypoints for the person in the image. Using this model, we get the landmarks with multiple keypoints on frames on every specified time interval (e.g. every 0.1s). We aim to collect angles from left and right shoulders, and left and right elbows as illustrated in Fig. 3.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, and incorporate computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, with the motivation of improving early detection of developmental disorders (Pg.1 Sarmiento Abstract).
REGARDING CLAIM 2
Passalis/ Zhou/ Sarmiento teach the method of claim 1, Sarmiento teaches wherein the disease or condition is torticollis, such as congenital muscular torticollis (CMT), autism spectrum disorder (ASD), or cerebral palsy (CP). ([Pg. 1 Abstract] a scalable, baseline application using computer vision algorithms and methodologies to capture four simple biomarkers data reliably for visual attention tracking using head pose estimation, specific observable behavioral patterns measuring blink rate and body posture, and morphological anomalies examining open mouth appearance from children
with autism spectrum disorder (ASD).
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, and incorporate computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, with the motivation of improving early detection of developmental disorders (Pg.1 Sarmiento Abstract).
REGARDING CLAIM 3
Passalis/ Zhou/ Sarmiento teach the method of claim 1, Passalis further teaches wherein the step of quantifying comprises determining bilateral postural asymmetry. ([Pg. 3 2.2 Facial Asymmetry] Our main assumption is that the difference (caused by facial asymmetry) between the left and the right region of a subject’s face is less than the difference between these regions and the regions of another subject’s face.)
REGARDING CLAIM 10
Passalis/ Zhou/ Sarmiento teach the method of claim 1, Sarmiento further teaches wherein the geometric symmetry measures include one or more of orbit slopes angle, relative face size, face angle, gaze angle, translational deformity, habitual head deviation, or combinations thereof. ([Pg. 1 Abstract] A scalable, baseline application using computer vision algorithms and methodologies to capture four simple biomarkers data reliably for visual attention tracking using head pose estimation (i.e. face angle), specific observable behavioral patterns measuring blink rate and body posture, and morphological anomalies examining open mouth appearance from children with autism spectrum disorder (ASD).)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, and incorporate computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, with the motivation of improving early detection of developmental disorders (Pg.1 Sarmiento Abstract).
REGARDING CLAIM 11
Passalis/ Zhou/ Sarmiento teach the method of claim 1, Sarmiento further teaches wherein the step of analyzing further comprises producing, by the infant face landmark estimation model, produce 68 landmark values. ([Pg. 2 II. Methods] We collect head movement data using yaw, roll, and pitch. For each video frame, the head-pose
estimation is predicted by the detection of 2D facial landmarks to estimate the 3D pose as seen in Image 1. We use dlib [7], a toolkit containing machine learning
algorithms and tools for finding the face and 68 facial landmarks necessary for predicting the head pose using OpenCV.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, and incorporate computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, with the motivation of improving early detection of developmental disorders (Pg.1 Sarmiento Abstract).
REGARDING CLAIM 12
Passalis/ Zhou/ Sarmiento teach the method of claim 1, Sarmiento teaches wherein the step of analyzing further comprises producing, by the infant body landmark estimation model, a plurality of landmark values corresponding to one or more of an upper arm, a lower arm, an upper leg, a lower leg, or combinations thereof of the infant. ([Pg. 2 2) Body pose estimation] Body motion is recognized to be an important behavioral measurement for people with ASD which includes head turning, postural control, and asymmetric arm positions. Observing patterns in body movements measured by angles in a time-series fashion in body movements measured by angles in a time-series fashion could provide insight for assessment and monitoring for motor movements of children with autism. Using this model, we get the landmarks with multiple keypoints on frames on every specified time interval (e.g. every 0.1s). We aim to collect angles from left and right shoulders (i.e. upper arms), and left and right elbows (i.e. lower arms) as illustrated in Fig. 3.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, and incorporate computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, with the motivation of improving early detection of developmental disorders (Pg.1 Sarmiento Abstract).
REGARDING CLAIM 13
Passalis/ Zhou/ Sarmiento teach the method of claim 12, Sarmiento further teaches wherein the plurality of landmark values produced by the infant body landmark estimation model correspond to one or more pairs of limbs of the infant, the one or more pairs of limbs of the infant including one or more of an upper right arm and upper left arm pair, a lower right arm and lower left arm pair, an upper right leg and upper left leg pair, a lower right leg and lower left leg pair, or combinations thereof. ([Pg. 2 2) Body pose estimation] Body motion is recognized to be an important behavioral measurement for people with ASD which includes head turning, postural control, and asymmetric arm positions. Observing patterns in body movements measured by angles in a time-series fashion in body movements measured by angles in a time-series fashion could provide insight for assessment and monitoring for motor movements of children with autism. Using this model, we get the landmarks with multiple keypoints on frames on every specified time interval (e.g. every 0.1s). We aim to collect angles from left and right shoulders (i.e. upper arms), and left and right elbows (i.e. lower arms) as illustrated in Fig. 3.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, and incorporate computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, with the motivation of improving early detection of developmental disorders (Pg.1 Sarmiento Abstract).
Claim(s) 14-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Passalis ("Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition," Oct. 2011) in view of Zhou (US 20130182008 A1) in view of Sarmiento (Behavioral Phenotyping for Autism Spectrum Disorder Biomarkers Using Computer Vision (2020)) in view of Goswami (US 7503900 B2).
REGARDING CLAIM 14
Passalis/ Zhou/ Sarmiento teach the method of claim 13, however Goswami teaches wherein the step of quantifying a symmetry of the of the geometric symmetry measures further comprises determining an angle difference across each of the one or more pairs of limbs. ([Abstract] Kinematic quantification of gait asymmetry is achieved by plotting the values of the angles of corresponding joints and then comparing the resulting figure to the figure that would have been produced based on a perfectly symmetrical gait. The comparison is based on geometric characteristics that are calculated based on the figures. [Col 3 Lines 65-67 and Col. 4 Lines 1-2] FIG. 3d illustrates a graph of the left knee angle versus the right knee angle (i.e. pair of lower limbs), based on the experimental data. This type of graph is commonly known as a cyclogram (or angle-angle plot). A cyclogram simultaneously plots the angular positions of two joints. [Col. 4 Lines 27-30] This invention uses bilateral cyclograms to describe gait. Bilateral cyclograms plot the angle of one joint on one leg versus the angle of the corresponding joint on the other leg. [Col. 5 Lines 16-23] A synchronized bilateral cyclogram generated based on a perfectly symmetrical gait is a line having a slope of 1 that crosses through the origin (0,0). This is because the joint angle on the left leg behaves identically to the corresponding joint angle on the right leg once the data has been synchronized. The characteristics of such a synchronized bilateral cyclogram are as follows: the area within it is zero; its orientation (i.e. angle difference) is 45.degree.; and its minimum moment is zero. [Col. 5 Lines 36-38] As illustrated in FIG. 3e, the area within the cyclogram is greater than zero, its orientation is less than 45.degree., and its minimum moment is greater than zero.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, and incorporate kinematic quantification of gait asymmetry as taught by Goswami, with the motivation of quantifying gait asymmetry that overcomes the limitations of algebraic indices while also being less computationally involved and easier to interpret than statistical techniques and parameters (Goswami Col 1 Lines 66-67 and Col. 2 Lines 1-2).
REGARDING CLAIM 15
Passalis/ Zhou/ Sarmiento/ Goswami teach the method of claim 14, Goswami teaches further comprising comparing the angle difference for each of the one or more pairs of limbs to a corresponding threshold angle. ([Col. 5 Lines 44-60] FIGS. 4a-l illustrate synchronized bilateral hip cyclograms of two sets of experimental subjects: "normal" subjects and subjects with gait pathologies. FIGS. 4a, 4c, 4e, 4g, 4i, and 4k concern normal subjects (experimental subjects #1, 3, etc.), while FIGS. 4b, 4d, 4f, 4h, 4j, and 4l concern subjects with gait pathologies (experimental subjects #2, 4, etc.). In this embodiment, the gait pathologies are caused by strokes. FIGS. 5a-c illustrate graphs of particular characteristics of the synchronized bilateral hip cyclograms in FIGS. 4a-l. The "X"s represent subjects with gait pathologies, while the dots represent normal subjects. FIG. 5a illustrates a graph of the area of the cyclogram for each experimental subject. [Col. 5 Lines 61-67] FIG. 5b illustrates a graph of the orientation of the cyclogram for each experimental subject. As mentioned above, the synchronized bilateral cyclogram of a perfectly symmetrical gait has an orientation of 45.degree (i.e. angle threshold). FIG. 5b shows that while the group of normal subjects had cyclograms with orientations at or close to 45.degree., the group of subjects with gait pathologies had many cyclograms with orientations nowhere near 45.degree.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, and incorporate kinematic quantification of gait asymmetry as taught by Goswami, with the motivation of quantifying gait asymmetry that overcomes the limitations of algebraic indices while also being less computationally involved and easier to interpret than statistical techniques and parameters (Goswami Col 1 Lines 66-67 and Col. 2 Lines 1-2).
REGARDING CLAIM 16
Passalis/ Zhou/ Sarmiento/ Goswami teach the method of claim 15, Goswami teaches further comprising classifying each of the pairs of limbs as symmetrical or asymmetrical, wherein;
a symmetrical classification indicates that a corresponding one of the pairs of limbs has a determined angle difference within the corresponding angle threshold; ([Col. 5 Lines 61-67] FIG. 5b illustrates a graph of the orientation of the cyclogram for each experimental subject. As mentioned above, the synchronized bilateral cyclogram of a perfectly symmetrical gait has an orientation of 45.degree (i.e. angle threshold). FIG. 5b shows that while the group of normal subjects had cyclograms with orientations at or close to 45.degree, the group of subjects with gait pathologies had many cyclograms with orientations nowhere near 45.degree. Examiner interprets that orientations of cyclograms that are at or close to 45 degrees is indicative of a symmetrical classification.)
and an asymmetrical classification indicates that the corresponding one of the pairs of limbs has a determined angle difference exceeding the corresponding angle threshold. ([Col. 5 Lines 20-35] The characteristics of such a synchronized bilateral cyclogram are as follows: the area within it is zero; its orientation is 45.degree (i.e. angle difference).; and its minimum moment is zero. In one embodiment, after generating the (synchronized) bilateral cyclogram 230, the characteristics of the cyclogram are measured 240 using characteristic computation module 122. [Col. 5 Lines 61-67] FIG. 5b illustrates a graph of the orientation of the cyclogram for each experimental subject. As mentioned above, the synchronized bilateral cyclogram of a perfectly symmetrical gait has an orientation of 45.degree (i.e. angle threshold). FIG. 5b shows that while the group of normal subjects had cyclograms with orientations at or close to 45.degree, the group of subjects with gait pathologies had many cyclograms with orientations nowhere near 45.degree. Examiner interprets that orientations of cyclograms that nowhere near to 45 degrees is indicative of an asymmetrical classification.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, and incorporate kinematic quantification of gait asymmetry as taught by Goswami, with the motivation of quantifying gait asymmetry that overcomes the limitations of algebraic indices while also being less computationally involved and easier to interpret than statistical techniques and parameters (Goswami Col 1 Lines 66-67 and Col. 2 Lines 1-2).
REGARDING CLAIM 17
Passalis/ Zhou/ Sarmiento/ Goswami teach the method of claim 14, Goswami further teaches further comprising assigning each of the pairs of limbs to an angle class. ([Col. 5 Lines 61-67] FIG. 5b illustrates a graph of the orientation of the cyclogram for each experimental subject. As mentioned above, the synchronized bilateral cyclogram of a perfectly symmetrical gait has an orientation of 45.degree (i.e. angle class). FIG. 5b shows that while the group of normal subjects had cyclograms with orientations at or close to 45.degree., the group of subjects with gait pathologies had many cyclograms with orientations nowhere near 45.degree.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, and incorporate kinematic quantification of gait asymmetry as taught by Goswami, with the motivation of quantifying gait asymmetry that overcomes the limitations of algebraic indices while also being less computationally involved and easier to interpret than statistical techniques and parameters (Goswami Col 1 Lines 66-67 and Col. 2 Lines 1-2).
REGARDING CLAIM 18
Passalis/ Zhou/ Sarmiento/ Goswami teach the method of claim 17, Goswami further teaches wherein the angle classes include one or more of <30°, ≥ 30°, <60°, ≥60°, 30°-59°, or combinations thereof. ([Col. 6 Lines 10-20] FIG. 6 illustrates a three-dimensional plot of the information contained in FIGS. 5a, 5b, and 5c. Each of the three axes represents a different characteristic of a cyclogram: area, orientation, and minimum moment magnitude. The locations of synchronized bilateral cyclograms based on pathological gaits are shown by stars, while the locations of cyclograms based on normal gaits are shown by dots. The location of a cyclogram formed from a perfectly symmetrical gait is shown by a diamond shape and is located at <0, 45, 0> (encompassing ≥ 30°, <60°, and 30°-59°);. As shown in FIG. 6, the normal gaits are located close to the perfectly symmetrical gait, while the pathological gaits are not.)
Therefore, it would be prima facie obvious to one of ordinary skill in the art, at the time of filing, to modify the method of using facial symmetry to handle pose variations as taught by Passalis, providing medical images for symmetry-based visualization as taught by Zhou, computer vision algorithms and methods for capturing head pose and body pose estimation biomarkers as taught by Sarmiento, and incorporate kinematic quantification of gait asymmetry as taught by Goswami, with the motivation of quantifying gait asymmetry that overcomes the limitations of algebraic indices while also being less computationally involved and easier to interpret than statistical techniques and parameters (Goswami Col 1 Lines 66-67 and Col. 2 Lines 1-2).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Aalamifar (US 20210004957 A1), which discloses a mobile application to allow users, such as parents and care providers, to measure and monitor, for example, a patient's body including an infant's head shape, at the point of care. The point of care can be, for instance, the home environment, a doctor's office, or a hospital setting. After acquiring 2D and/or 3D images of the body part, parameters reflecting potential deformity can be calculated. If abnormal measurements are determined, the user can be guided through therapeutic options to improve the condition. Based on the severity of the condition, different recommendations can be provided. Moreover, longitudinal monitoring and evaluation of the parameters can be performed. Monitoring of the normal child development can also be performed through longitudinal determination of parameters and comparison to normative values. Data can be shared with clinician's office.
Xu et al (US 20190259493 A1), which discloses medical image data may be applied to a machine-learned network learned on training image data and associated image segmentations, landmarks, and view classifications to classify a view of the medical image data, detect a location of one or more landmarks in the medical image data, and segment a region in the medical image data based on the application of the medical image data to the machine-learned network. The classified view, the segmented region, or the location of the one or more landmarks may be output.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Patricia K Edouard whose telephone number is (571)272-6084. The examiner can normally be reached Monday - Friday 7:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter H Choi can be reached at 469-295-9171. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.K.E./Examiner, Art Unit 3681
/PETER H CHOI/Supervisory Patent Examiner, Art Unit 3681
Prosecution Insights