DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
The office action is in response to the claim amendments filed on July 2, 2024 for the application filed December 7, 2023 which claims priority to a foreign application filed on June 7, 2021. Claims 3-9, 13-14 and 19 have been amended and claims 21 and 23 have been cancelled. Claims 1-20 and 22 are currently pending and have been examined.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Eligibility Step 1:
Under step 1 of the 2019 Revised Patent Subject Matter Eligibility Guidance, claims 1-19 are directed towards a computer system (i.e. a machine), which is a statutory category. Claim 20 is directed towards a method (i.e. a process), which is a statutory category. Since the claims are directed toward statutory categories, it must be determined if the claims are directed towards a judicial exception (i.e. a law of nature, a natural phenomenon, or an abstract idea). In the instant application, the claims are directed towards an abstract idea.
Eligibility Step 2A, Prong One:
Under step 2A, prong one of the 2019 Revised Patent Subject Matter Eligibility Guidance, independent claims 1 and 20 are determined to be directed to an judicial exception because an abstract idea is recited in the claims which fall within the subject matter groupings of abstract ideas.
The abstract idea (identified in bold) recited in the representative claim 1 is identified as:
A computer system for estimating a condition of a subject, wherein the computer system comprises:
one or more processors configured to:
receive a plurality of images photographed of the subject walking,
generate at least one silhouette image of the subject by extracting silhouette regions from the plurality of image frames, normalizing the extracted silhouette regions, and averaging the normalized silhouette regions, and
estimate a health-related condition of the subject by inputting the at least one silhouette image into a trained machine learning model.
The identified limitations of the abstract idea of claims 1 and 20 fall within the subject matter grouping of certain methods of organizing human activity related and the sub grouping of managing personal behavior or relationships or interactions between people, (including social activities, teaching, and following rules or instructions). The claims are directed to estimating a health-related condition of the subject, which is a human activity routinely performed by healthcare professionals. Limiting the estimation to be based on a silhouette image is merely a rule or instruction a healthcare professional should follow when estimating a health-related condition of the subject.
The identified limitations of the abstract idea of claims 1 and 20 fall within the subject matter grouping of mental processes. If a claim recites a limitation that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper, the limitation falls within the mental processes grouping, and the claim recites an abstract idea. The limitation “estimating a health-related condition of the subject at least based on the at least one silhouette image” can be performed mentally using observations, evaluations, judgments and opinions.
Accordingly, claims 1 and 20 recite an abstract idea under step 2A, prong one.
Eligibility Step 2A, Prong Two:
Under step 2A, prong two of the 2019 Revised Patent Subject Matter Eligibility Guidance, it must be determined whether the identified abstract ideas are integrated into a practical application. After evaluation, there is no indication that any additional elements or combination of elements integrate the abstract idea into a practical application, such as through: an additional element that reflects an improvement to the functioning of a computer, or an improvements to any other technology or technical field; an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition; an additional element that implements the judicial exception with, or uses the judicial exception in connection with, a particular machine or manufacture that is integral to the claim; an additional element that effects a transformation or reduction of a particular article to a different state or thing; or an additional element that applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. As shown below, the additional elements of a computer system comprising a receiving means…; and an estimation means, when considered both individually and as an ordered combination, amount to no more than a recitation of: generally linking the abstract idea to a particular technological environment or field of use; insignificant extra-solution activity to the judicial exception; and/or adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea as evidenced below.
The additional elements recited in representative claim 1 are identified in italics as:
A computer system for estimating a condition of a subject, wherein the computer system comprises:
one or more processors configured to:
receive a plurality of images photographed of the subject walking,
generate at least one silhouette image of the subject by extracting silhouette regions from the plurality of image frames, normalizing the extracted silhouette regions, and averaging the normalized silhouette regions, and
estimate a health-related condition of the subject by inputting the at least one silhouette image into a trained machine learning model.
The additional limitations of “A computer system, wherein the computer system comprises: one or more processors configured to …” are determined to be mere instructions to apply an abstract idea under MPEP §2106.05(f). The computer system and processors are recited at a high level of generality and merely used to perform the abstract idea. Similarly the “trained machine learning model” is used to generally apply the abstract idea without placing any limits on how the trained model functions. Rather, this limitation only recite the outcome of “estimate a health-related condition” and does not include any details about how the “estimating” us accomplished. Therefore, these additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or no more than mere instructions to implement an abstract idea or other exception on a computer or no more than merely using a computer as a tool to perform an abstract idea.
The additional limitation of “receive a plurality of images photographed of the subject walking” is determined to be no more than insignificant extra-solution activity to the judicial exception under MPEP §2106.05(g). Receiving a plurality of images photographed of the subject walking is determined to be mere necessary data gathering which does not imposes meaningful limits on the claim.
The additional limitation “generate at least one silhouette image of the subject by extracting silhouette regions from the plurality of image frames, normalizing the extracted silhouette regions, and averaging the normalized silhouette regions” does not provide any details as to how the generating, extracting, normalizing or averaging are accomplished such that this limitation is determined to not provide: an additional element that reflects an improvement to the functioning of a computer, or an improvements to any other technology or technical field; an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition; an additional element that implements the judicial exception with, or uses the judicial exception in connection with, a particular machine or manufacture that is integral to the claim; an additional element that effects a transformation or reduction of a particular article to a different state or thing; or an additional element that applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
Accordingly, claims 1 and 20 do not recite additional elements which integrate the abstract idea into a practical application.
Eligibility Step 2B:
Under step 2B of the 2019 Revised Patent Subject Matter Eligibility Guidance, it must be determined whether provide an inventive concept by determining if the claims include additional elements or a combination of elements that are sufficient to amount to significantly more than the judicial exception. After evaluation, there is no indication that an additional element or combination of elements are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional limitations of “a computer system… wherein the computer system comprises: one or more processors configured to”, “by inputting into a trained machine learning model” and “receive…” are determined to be mere instructions to apply an abstract idea under MPEP §2106.05(f) and insignificant extra-solution activity to the judicial exception under MPEP §2106.05(g), which is do not amount to significantly more than the abstract idea. Evidence that using a processor to receive data (i.e. images of a subject) is well-understood, routine and conventional is provided by MPEP §2106.05(d), subsection II. The additional element of a processor configure to “generate at least one silhouette image of the subject by extracting silhouette regions from the plurality of image frames, normalizing the extracted silhouette regions, and averaging the normalized silhouette regions” is determined to be well-understood routine and conventional as evidenced Wang et al. (Silhouette analysis-based gait recognition for human identification) published in 2003 which discloses that using a processor and specialized algorithm for generating at least one silhouette image of the subject from the plurality of images by extracting regions, normalizing and averaging is well-understood, routine and conventional.
Furthermore, looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements amounts to an inventive concept.
Dependent Claims:
The dependent claims merely present additional abstract information in tandem with further details regarding the elements from the independent claims and are, therefore, directed to an abstract idea for similar reasons as given above. None of these limitations are deemed to integrate the claims into a practical application or to amount to significantly more than the abstract idea as detailed below:
Regarding claim 2, estimating a condition including a condition related to at least one disease of the subject is considered both a mental process and certain method of organizing human activity for similar reasons as detailed above with respect to claim 1.
Regarding claim 3, the use of the learned model that has learned the relationships between a learning silhouette image and the condition related to at least one disease of the object shown in the learning silhouette image provide nothing more than mere instructions to implement an abstract idea on a generic computer. The learned model is used to generally apply the abstract idea without placing any limits on how the algorithm functions. Rather, these limitations only recite the outcome of “estimating a health-related condition of the subject” and do not include any details about how the “estimating” is accomplished. That is, the limitations only recited a generic learned “relationships” of inputs to outputs without the details of the “relationships” used to estimate a health-related condition.
Regarding claim 4, extracting skeletal feature of the subject from the plurality of images and estimating the condition based on the skeletal feature is determined to recite abstract idea falling within the subject matter grouping of mental processes as this can be practically performed in the human mind using observations, evaluations, judgments and opinions. Furthermore, the extraction is disclosed in the specification as utilizing techniques known in the art (see paragraph [0222] of the printed publication), and therefore well-understood, routine and conventional under step 2B.
Regarding claim 5, the obtaining the first score, second score and estimating the condition based on the first and second scores is determined to recite abstract idea falling within the subject matter grouping of mental processes as this can be practically performed in the human mind using observations, evaluations, judgments and opinions.
Regarding claim 6, the additional limitations of extracting regions, normalizing and averaging are determined to be well-understood, routine and conventional techniques in the art under step 2B, as evidenced by Wang et al. (Silhouette analysis-based gait recognition for human identification) published in 2003.
Regarding claim 7 and 16, defining the received images still results in mere necessary data gathering which is insignificant extra-solution activity to the judicial exception under MPEP §2106.05(g) and well-understood, routine and conventional under step 2B and does not impose meaningful limits on the claims.
Regarding claim 8, analyzing results of the estimation to identify a region of interest that contributes largely to the result of the estimation and modify the algorithm of the estimation means based on the region of interest is determined to recite abstract idea falling within the subject matter grouping of mental processes as this can be practically performed in the human mind using observations, evaluations, judgments and opinions. Furthermore, the analysis is disclosed in the specification as utilizing techniques known in the art (see paragraph [0258] of the printed publication), and therefore well-understood, routine and conventional.
Regarding claims 9-15, 17 and 18, the limitations merely generally define the estimation and what it is related to and are encompassed by the abstract idea recited in claim 1.
Regarding claim 19, providing information based on the condition is considered to be directed to a mental process and human activity as this can be performed mentally using observations, evaluations, judgments an opinions and is an activity routinely performed by healthcare providers in order to inform patients of treatment or intervention information based on an estimated condition/diagnosis. Using a processor to provide information is determined to be no more than insignificant extra-solution activity to the judicial exception under MPEP §2106.05(g) as providing information is mere data output and is well-understood, routine and conventional in a computer environment as evidence by MPEP §2106.05(d), subsection II.
Therefore, whether taken individually or as an ordered combination, 1-20 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 6-7, 9-10, 13, 20 and 22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Verlekar et al. (Using transfer learning for classification of gait pathologies).
Regarding claim 1, Verlekar discloses a computer system for estimating a condition of a subject (Page 2376, Abstract, and Page 2379, section III A, graphic card), wherein the computer system comprises:
one or more processors configured to (Page 2376, Abstract, and Page 2379, section III A, graphic card. Also see fig. 1 which is construed as being implemented via a processor.):
receive a plurality of images photographed of the subject walking (Page 2376, Section I, The system operates on videos acquired by a single 2D camera. Page 2376, Abstract, computes the walking individual's silhouettes, which are computed from a 2D video sequence.),
generate at least one silhouette image of the subject (Page 2377, Section II A, The first step of the proposed system involves transforming a 2D video sequence into a GEI, see Fig. 2.) by extracting silhouette regions from the plurality of image frames, normalizing the extracted silhouette regions, and averaging the normalized silhouette regions (Page 2377, Section II, the proposed system extracts binary silhouettes from a given video sequence. Page 2377, Section II A, The first step of the proposed system involves transforming a 2D video sequence into a GEI, see Fig. 2. It begins by using background subtraction to convert an input video, into a sequence of binary silhouettes. The silhouettes are then normalized to a common height. Next, the silhouettes belonging to a gait cycle (1, N) are cropped, Ic(x,y,n), and averaged to obtain the GEI, GEI(x,y).), and
estimate health-related condition of the subject by inputting on the at least one silhouette image into a trained machine learning model (Page 2377, Section II, Next, the resulting GEI is used as an input to the feature extraction step, using the VGG-19 network. Although VGG-19 can be used as a classifier, in this paper the output from its fully connected layer is used as a feature vector [19]. The quality of the feature vectors resulting from VGG-19 can be further improved by transfer learning, where the final layers of the CNN are fine-tuned to better represent the different gait pathologies. The final step of classification is performed using LDA, which classifies each feature vector across different pathology groups. Also see Page 2380, Table IV.).
Regarding claim 2, Verlekar further discloses wherein the one or more processors are further configured to estimate a condition including a condition related to at least one disease of the subject (Page 2380, Section IV, This paper presents a novel system to perform classification of gait across different pathologies. These pathologies vary from restrictions in leg movement to alterations in gait caused by neurological or systemic disorders such as diplegia, hemiplegia, neuropathy and Parkinson’s diseases . Also see Page 2380,Table IV.).
Regarding claim 3, Verlekar further discloses wherein the learned model has learned a relationship between the silhouette image and the condition related to at least one disease of the subject shown in the silhouette image (Page 2376, Abstract, The system computes the walking individual's silhouettes, which are computed from a 2D video sequence, and combines them into a representation known as the gait energy image (GEI), which provides robustness against silhouette segmentation errors. In this work, instead of using a set of hand-crafted gait features, feature extraction is done using the VGG-19 convolutional neural network. The network is fine-tuned to automatically extract the features that best represent gait pathologies, using transfer learning. The use of transfer learning improves the classification accuracy while avoiding the need of a very large training set, as the network is pre-trained for generic image description, which also contributes to a better generalization when tested across different datasets. The proposed system performs the final classification using linear discriminant analysis (LDA). Page 2379, Section III B, LDA classifier must be trained separately. Page 2377, Section II, VGG-19 can be used as a classifier. Also see Table IV.).
Regarding claim 6, Verlekar further discloses wherein
the one or more processors are further configured to generate the at least one silhouette image by
extracting a plurality of silhouette regions from the plurality of images (Page 2377, Section II, the proposed system extracts binary silhouettes from a given video sequence. Page 2377, Section II A, The first step of the proposed system involves transforming a 2D video sequence into a GEI, see Fig. 2. It begins by using background subtraction to convert an input video, into a sequence of binary silhouettes.),
normalizing each of the plurality of extracted silhouette regions (Page 2377, Section II A, The silhouettes are then normalized to a common height.), and
averaging the plurality of normalized silhouette regions (Page 2377, Section II A, Next, the silhouettes belonging to a gait cycle (1, N) are cropped, Ic(x,y,n), and averaged to obtain the GEI, GEI(x,y).).
Regarding claim 7, Verlekar further discloses wherein the plurality of images are a plurality of frames in a video of the subject walking photographed from a direction approximately perpendicular to the direction in which the subject walks (Page 2376, Section I, The system operates on videos acquired by a single 2D camera. Page 2376, Abstract, computes the walking individual's silhouettes, which are computed from a 2D video sequence. Page 2377, Section I A, a significant amount of work has also been done in capturing and analysing gait from a single 2D camera. Since the major articulations during a gait cycle occur in the sagittal plane [12], some vision based systems rely on a single side view video sequence of an individual to perform gait analysis. Such systems typically acquire several biomechanical features, such as step length, leg angles, gait cycle time [13], cadence, speed, and stride length [14], or the fraction of the stance and swing phases during a gait cycle [15], using the available side view body silhouettes. Also see Page 2377, Fig. 2).
Regarding claim 9, Verlekar further discloses wherein the health-related condition includes a condition related to at least one disease of the subject, and the at least one disease includes a disease that causes a walking disorder (Page 2376, Abstract).
Regarding claim 10, Verlekar further discloses wherein the at least one disease includes at least one selected from the group consisting of locomotor diseases that cause a walking disorder, neuromuscular diseases that cause a walking disorder, cardiovascular disease that cause a walking disorder, and respiratory diseases that cause a walking disorder (Page 2376, Abstract).
Regarding claim 13, Verlekar further discloses wherein the at least one disease includes at least one selected from the group consisting of cervical spondylotic myelopathy (CSM), lumbar canal stenosis (LCS), osteoarthritis (OA), neuropathy, intervertebral disc herniation, ossification of the posterior longitudinal ligament (OPLL), rheumatoid arthritis (RA), heart failure, hydrocephalus, peripheral artery disease (PAD), myositis, myopathy, Parkinson's disease, amyotrophic lateral sclerosis (ALS), spinocerebellar degeneration, multiple system atrophy, brain tumor, Lewy body dementia, subclinical fracture, drug addiction, meniscal injury, ligament injury, spinal cord infarction, myelitis, myelopathy, pyogenic spondylitis, discitis, bunion, chronic obstructive pulmonary disease (COPD), obesity, cerebral infarction, locomotive syndrome, frailty, and hereditary spastic paraplegia (Page 2376, Abstract).
Regarding claim 20, Verlekar discloses a method for estimating a condition of a subject (Page 2376, Abstract), wherein the method comprises:
receiving a plurality of images photographed of the subject walking (Page 2376, Section I, The system operates on videos acquired by a single 2D camera. Page 2376, Abstract, computes the walking individual's silhouettes, which are computed from a 2D video sequence.),
generating at least one silhouette image of the subject from the plurality of images (Page 2377, Section II A, The first step of the proposed system involves transforming a 2D video sequence into a GEI, see Fig. 2. It begins by using background subtraction to convert an input video, into a sequence of binary silhouettes.), and
estimating a health-related condition of the subject at least based on the at least one silhouette image (Page 2377, Section II, Next, the resulting GEI is used as an input to the feature extraction step, using the VGG-19 network. Although VGG-19 can be used as a classifier, in this paper the output from its fully connected layer is used as a feature vector [19]. The quality of the feature vectors resulting from VGG-19 can be further improved by transfer learning, where the final layers of the CNN are fine-tuned to better represent the different gait pathologies. The final step of classification is performed using LDA, which classifies each feature vector across different pathology groups. Also see Page 2380, Table IV.).
Regarding claim 22, Verlekar discloses a method that creates a model for estimating a condition of a subject (Page 2376, Abstract), wherein the method comprises:
receiving a plurality of images photographed of the subject walking (Page 2376, Section I, The system operates on videos acquired by a single 2D camera. Page 2376, Abstract, computes the walking individual's silhouettes, which are computed from a 2D video sequence.),
generating at least one silhouette image of the subject from the plurality of image (Page 2377, Section II A, The first step of the proposed system involves transforming a 2D video sequence into a GEI, see Fig. 2. It begins by using background subtraction to convert an input video, into a sequence of binary silhouettes.)s, and
causing a machine learning model to learn the at least one silhouette image as input training data and the health-related condition of the subject as output training data, for each subject among a plurality of subjects (Page 2376, Abstract, The system computes the walking individual's silhouettes, which are computed from a 2D video sequence, and combines them into a representation known as the gait energy image (GEI), which provides robustness against silhouette segmentation errors. In this work, instead of using a set of hand-crafted gait features, feature extraction is done using the VGG-19 convolutional neural network. The network is fine-tuned to automatically extract the features that best represent gait pathologies, using transfer learning. The use of transfer learning improves the classification accuracy while avoiding the need of a very large training set, as the network is pre-trained for generic image description, which also contributes to a better generalization when tested across different datasets. The proposed system performs the final classification using linear discriminant analysis (LDA). Page 2379, Section III B, LDA classifier must be trained separately. Page 2377, Section II, VGG-19 can be used as a classifier. Also see pages 2378-2379, Sections III A and B. Also see Table IV.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Verlekar et al. (Using transfer learning for classification of gait pathologies) in view of Loureiro et al. (Using a Skeleton Gait Energy Image for Pathological Gait Classification).
Regarding claim 4, Verlekar does not appear to explicitly disclose, but Loureiro teaches that it was old and well known in the art of gait classification at the time of the filing wherein the one or more processors are further configured to extract a skeletal feature of the subject from the plurality of images (Loureiro, Page 505, Section III, first step to compute the SEI is obtaining a skeleton for each image of the walking person. By applying the OpenPose [13] algorithm a set of 24 pairs of 2D coordinates are obtained, corresponding to different parts of the human body: nose, neck, shoulders, elbows, wrists, middle hip, right and left hip, knees, ankles, eyes, ears, big toes, small toes, and heels. Notice that the SGEI model [17] uses only 13 keypoints, located in the limbs and the body torso, not including the tip of the feet or the head. As a consequence, the proposed SEI can better model impaired gait postures. The SEI skeleton image is obtained by drawing lines connecting the 24 estimated coordinates using OpenCV [22], obtaining representations similar to the one illustrated in Fig. 1 (right). Then a resizing and averaging process is applied, as described for the GEI computation, to obtain the SEI.), and
estimate the condition further based on the skeletal feature (Loureiro, Page 505-506, Section V B, Since the computation of silhouettes and skeletons uses different algorithms, combining the results using gait representations based on GEI and SEI could further improve the performance of the system. As such, an experiment merging the classification results of the two representations was conducted. To do this, for each gait sequence, the CNN was used to perform classification, using both the GEI and the corresponding SEI, as the full CNN classifier obtained best results with the GAIT-IST dataset. The resulting scores for each category, obtained before the softmax layer, represent the probability that the input observed belongs to that category, and the maximum between the values observed for the GEI and SEI was kept. A maxscore fusion was therefore applied, selecting as the predicted category the one with the highest score corresponding to the usage of GEI or SEI. This can be interpreted as choosing the prediction of the model that performed the classification with higher certainty.) to provide an improvement of the overall classification accuracy (Loureiro, Page 506, Section V B).
Therefore, it would have been obvious to one of ordinary skill in the art of gait classification at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Loureiro, in order to provide an improvement of the overall classification accuracy.
Regarding claim 5, Verlekar does not appear to explicitly disclose, but Loureiro teaches that it was old and well known in the art of gait classification at the time of the filing wherein r he one or more processors are further configured to obtain a first score indicating the condition based on the at least one silhouette image (Loureiro, Page 505-506, Section V B, the CNN was used to perform classification, using both the GEI and the corresponding SEI. The resulting scores for each category, obtained before the softmax layer, represent the probability that the input observed belongs to that category.),
obtain a second score indicating the condition based on the skeletal feature (Loureiro, Page 505-506, Section V B, the CNN was used to perform classification, using both the GEI and the corresponding SEI. The resulting scores for each category, obtained before the softmax layer, represent the probability that the input observed belongs to that category.), and
estimate the condition based on the first score and the second score (Loureiro, Page 506, Section V B, The resulting scores for each category, obtained before the softmax layer, represent the probability that the input observed belongs to that category, and the maximum between the values observed for the GEI and SEI was kept. A maxscore fusion was therefore applied, selecting as the predicted category the one with the highest score corresponding to the usage of GEI or SEI. This can be interpreted as choosing the prediction of the model that performed the classification with higher certainty.) to provide an improvement of the overall classification accuracy (Loureiro, Page 506, Section V B).
Therefore, it would have been obvious to one of ordinary skill in the art of gait classification at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Loureiro, in order to provide an improvement of the overall classification accuracy.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Verlekar et al. (Using transfer learning for classification of gait pathologies) in view of Zhang et al. (Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks).
Regarding claim 8, Verlekar does not appear to explicitly disclose, but Zhang teaches that it was old and well known in the art of image classification at the time of the filing wherein the one or more processors are further configured to
identify, in the at least one silhouette image, a region of interest that contributes to the estimation result, and modify a parameter or structure of an algorithm based on the identified region of interest (Zhang, Page 2, section 1, (1) we introduce Group-CAM, an efficient explaining approach for deep convolutional networks by estimating the importance of input image pixels for the model’s prediction; (4) we extend the application of saliency methods and apply Group-CAM as an effective data augment trick for fine-tuning classification networks, extensive experimental results suggest that Group-CAM can boost the networks’ performance by a large margin. Page 7, Section 5, we extend the application of Group-CAM and apply it as an effective data augment strategy to finetune/train the classification models. Page 8, Section 5, The fine-tuning process is defined as follows: (1) generate saliency map M for I0 with G = 16 and the ground-truth target class c; (2) binarize M with threshold θ, where θ is the mean value of M. (3) apply Eq. 5 to get the blurred input ˜I0. (4) adopt ˜I0 to fine-tune the classification model. Since ˜I0 are generated during the training process, which means that when the performance of the classification model is improved, Group-CAM will generate a better ˜I0, which in turn will promote the performance of the classification model.).
Therefore, it would have been obvious to one of ordinary skill in the art of image classification at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Zhang, as Verlekar contemplates further fine-tuning of the models to obtain more significant results (Page 2380, Section IV) and Group-CAM can be applied to fine-tuning of classification models in order to boost the networks’ performance by a large margin.
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Verlekar et al. (Using transfer learning for classification of gait pathologies) in view of Sankai et al. (U.S. Pub. No. 2021/0204836).
Regarding claim 11, Verlekar does not appear to explicitly disclose, but Sankai teaches that it was old and well known in the art of gait detection at the time of the filing wherein estimating the condition related to at least one disease includes determining which organ the disease causing a walking disorder relates to (Sankai, paragraph [0054], gaits which are likely caused by a disorder in the brain are spastic hemiplegic gait (compass gait), spastic paraplegia gait (scissors gait), Parkinson gait (tiny step gait), and ataxic gait. Paragraph [0064], determination result recognizes that it is caused by manifestation in the brain.) to discover the symptom of dementia at an early stage before such symptom is discovered by others (Sankai, paragraph [0003]).
Therefore, it would have been obvious to one of ordinary skill in the art of gait detection at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Sankai, in order to discover the symptom of dementia at an early stage before such symptom is discovered by others.
Regarding claim 12, Verlekar further discloses wherein the determination includes determining whether the disease causing a walking disorder is a locomotor disease, a neuromuscular disease, a cardiovascular disease, or a respiratory disease (Page 2376, Abstract).
Claims 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Verlekar et al. (Using transfer learning for classification of gait pathologies) in view of Kalsi-Ryan et al. (Quantitative Assessment of Gait Characteristics in Degenerative Cervical Myelopathy: A Prospective Clinical Study).
Regarding claim 14, Verlekar does not appear to explicitly disclose, but Kalsi-Ryan teaches that it was old and well known in the art of gait assessment at the time of the filing wherein the health-related condition of the subject is represented by the severity of at least one disease, and the estimate is the severity (Kalsi-Ryan, Page 1, Abstract, Quantitative gait assessments show promise as an accurate and objective tool to diagnose and classify DCM. Page 2, Section 1, characterize mild, moderate, and severe DCM, as defined by the mJOA classification system, using quantitative spatiotemporal measurements of gait.) to enable early identification of the disease and monitoring disease progression (Kalsi-Ryan, Page 8, Section 4).
Therefore, it would have been obvious to one of ordinary skill in the art of gait detection at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Kalsi-Ryan, in order to enable early identification of the disease and monitoring disease progression, as Verlekar contemplates further fine-tuning of the models on different datasets to obtain more significant results (Page 2380, Section IV).
Regarding claim 15, Verlekar does not appear to explicitly disclose, but Kalsi-Ryan teaches that it was old and well known in the art of gait assessment at the time of the filing wherein the disease is cervical spondylotic myelopathy, and the estimate is a cervical spine JOA score as the severity (Kalsi-Ryan, Page 1, Abstract, Quantitative gait assessments show promise as an accurate and objective tool to diagnose and classify DCM. Page 2, Section 1, characterize mild, moderate, and severe DCM (Degenerative Cervical Myelopathy), as defined by the mJOA classification system, using quantitative spatiotemporal measurements of gait.) to enable early identification of the disease and monitoring disease progression (Kalsi-Ryan, Page 8, Section 4).
Therefore, it would have been obvious to one of ordinary skill in the art of gait detection at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Kalsi-Ryan, in order to enable early identification of the disease and monitoring disease progression, as Verlekar contemplates further fine-tuning of the models on different datasets to obtain more significant results (Page 2380, Section IV).
Regarding claim 16, Verlekar does not appear to explicitly disclose, but Kalsi-Ryan teaches that it was old and well known in the art of gait assessment at the time of the filing wherein a plurality of images photographed is received of walking of a subject whose cervical JOA score is determined to be 10 or more (Kalsi-Ryan, Page 3, Section 2.2, Mild DCM was defined by mJOA values between 15 and 17, moderate DCM by mJOA values from 12 to 14, and severe DCM by a mJOA score <12. Page 6, Section 3.1 and Table 3, The sample of DCM patients consisted of 83 male and 70 female participants, with a mean age of 56.81 ± 10.92 years. The mean duration of symptoms was 44.19 ± 56.06 months prior to assessment. Table 3 defines the sample stratified by mJOA into mild, moderate, and severe groups.) to enable early identification of the disease and monitoring disease progression (Kalsi-Ryan, Page 8, Section 4).
Therefore, it would have been obvious to one of ordinary skill in the art of gait detection at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Kalsi-Ryan, in order to enable early identification of the disease and monitoring disease progression, as Verlekar contemplates further fine-tuning of the models on different datasets to obtain more significant results (Page 2380, Section IV).
Claims 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Verlekar et al. (Using transfer learning for classification of gait pathologies) in view of Ichikawa et al. (JP2018069035A).
Regarding claim 17, Verlekar does not appear to explicitly disclose, but Ichikawa teaches that it was old and well known in the art of gait analysis at the time of the filing wherein the one or more processors are further configured to estimate the walking ability of the subject (Ichikawa, Abstract, A walking analysis system capable of calculating a walking age that matches walking ability is provided. A gait analysis system 100 includes a three-dimensional measuring apparatus 1 that sequentially measures three-dimensional coordinates of a plurality of predetermined body feature points of a subject H as the subject walks, and a plurality of body feature points of the subject. The velocity age, balance age, and posture age of the subject are calculated using the three-dimensional coordinates and the first correspondence relationship stored in advance, and each age of the calculated subject and the second correspondence relationship stored in advance are calculated.) to know an objective evaluation standard as to at which age's average level (walking age) their walking ability is (Ichikawa, paragraph [0002]).
Therefore, it would have been obvious to one of ordinary skill in the art of gait analysis at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Ichikawa, in order to know an objective evaluation standard as to at which age's average level (walking age) their walking ability is, as Verlekar contemplates further fine-tuning of the models on different datasets to obtain more significant results (Page 2380, Section IV).
Regarding claim 18, Verlekar does not appear to explicitly disclose, but Ichikawa teaches that it was old and well known in the art of gait analysis at the time of the filing wherein the walking ability of the subject is expressed by a numerical value indicating which age level the subject is at (Ichikawa, Abstract, A walking analysis system capable of calculating a walking age that matches walking ability is provided. A gait analysis system 100 includes a three-dimensional measuring apparatus 1 that sequentially measures three-dimensional coordinates of a plurality of predetermined body feature points of a subject H as the subject walks, and a plurality of body feature points of the subject. The velocity age, balance age, and posture age of the subject are calculated using the three-dimensional coordinates and the first correspondence relationship stored in advance, and each age of the calculated subject and the second correspondence relationship stored in advance are calculated.) to know an objective evaluation standard as to at which age's average level (walking age) their walking ability is (Ichikawa, paragraph [0002]).
Therefore, it would have been obvious to one of ordinary skill in the art of gait analysis at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Ichikawa, in order to know an objective evaluation standard as to at which age's average level (walking age) their walking ability is, as Verlekar contemplates further fine-tuning of the models on different datasets to obtain more significant results (Page 2380, Section IV).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Verlekar et al. (Using transfer learning for classification of gait pathologies) in view of Delp et al. (U.S. Pub. No. 2021/0315486).
Regarding claim 19, Verlekar does not appear to explicitly disclose, but Delp teaches that it was old and well known in the art of gait evaluation at the time of the filing wherein the one or more processors are further configured to provide treatment or intervention or information according to the estimated condition (Delp, Abstract, a motion evaluation system that trains a model to evaluate motion (such as, but not limited to, gait) through images (or video) captured by a single image capture device. Paragraph [0052], Output engines in accordance with several embodiments of the invention can provide a variety of outputs to a user, including (but not limited to) treatment interventions for correcting anomalies in a user's motion, physical therapy exercises, as well as displaying, recording, and/or transmitting scores for tracking progress of the individual. Paragraph [0084], outputs from a process can include a diagnosis for a disease.) to allow a user to quickly and efficiently evaluate motion of an individual (Delp, Paragraph [0047]).
Therefore, it would have been obvious to one of ordinary skill in the art of gait evaluation at the time of the filing to modify the system of Verlekar to include the limitations above, as taught by Delp, in order to allow a user to quickly and efficiently evaluate motion of an individual.
Response to Arguments
Applicant's arguments filed February 25, 2025 regarding claims 1-20 being rejected under 35 U.S.C. §101 have been fully considered but they are not persuasive.
Applicant argues that the analysis under step 2A, prong 1 fails to account for the technical operations and a healthcare professional cannot, without a computer, extract binary silhouettes from video frames, normalize those silhouettes and average the silhouettes and input the results into a trained machine learning model.
In response, these limitations are considered additional elements and considered under step 2A, prong 2 and step 2B. The abstract idea is identified as “estimate a health-related condition of the subject by at least one silhouette image” and not to how the silhouette image in generated or the generic use of processors and machine learning models to perform the estimation.
Applicant argues under step 2A, prong 2 that the claims imposed structural and functional constraints on how the processor operates and cannot be mere instructions to implement the abstract idea o a generic computer as this ignores the specific algorithmic constraints imposed by the claims, such as specific extraction, normalization, averaging steps followed by the trained-model inference and the dependent claims 4, 5, 8, 14 and 15, which reflect technical improvement to the health-condition estimation and collectively amount to significantly more.
In response, the additional elements recite the desired outcome and do not include an details as to how the outcomes are accomplished. For example, the claims and specification provide no details as to how the silhouette is extracted, normalized or averaged, how the train-model estimates the condition or how any of the features of claims 4, 5, 8, 14 and 15 are accomplished which could be considered an improvement to any of these technologies. As detailed in the previous rejection, the extraction, normalization, averaging steps are determined to be well-understood, routine and conventional steps in art of Silhouette analysis-based gait recognition. Applicant provides no argument or evidence that these represent improvements to technology or even that they go beyond mere well-understood, routine and conventional image pre-processing to perform the abstract idea of estimating a condition of a subject based on a silhouette image using a generic “trained machine learning model”.
Applicant's arguments filed February 25, 2025 regarding claims 1-20 and 22 being rejected under 35 U.S.C. §102/103 have been fully considered but they are not persuasive.
Applicant argues that Verlekar does not disclose how the silhouette image is generated as it relies on a preprocessed dataset-proceed gait representation to perform the extraction, normalization and averaging operations on extracted silhouette images within the system and does not disclose the end-end solution which generates the silhouette image and then used the image as input for the model to the estimate the condition.
In response, Verlekar relies on a preprocessed dataset-proceed gait representations for model training data, not for generating the silhouette image and estimating the condition of a subject based on the image. Figure 1 of Verlekar clearly shows that the proposes system receives video, performs pre-processing (i.e. extraction, normalization and averaging), performs feature extraction and classification (estimating a condition using a machine learning model) and provides the results.
Applicant argues that Verlekar and Loureiro do not teach claims 4 and 5, fails to provide a proper rationale for the combination and fails to identify where Loureiro teaches the limitations.
In response to the Applicant’s assertions, it is maintained that the rejection clearly indicates where and how Loureiro teaches the identified limitations and the rationale to provide an improvement of the overall classification accuracy is proper as Loureiro teaches that combining GEI and SEI analysis and scoring improves classification accuracy and Verlekar is concerned with improving classification accuracy.
Applicant argues that Verlekar and Zhang do not teach claim 8 as the discloses Group-CAM method of Zhang does not teach modifying a parameter or structure of the estimation algorithm in a close-loop fashion and because the asserted motivation to “boost performance” is conclusory and not sufficient for a motivation to combine.
In response, Zhang clearly teaches that the Group-CAM method can be integrated into fine-tuning of classification models and Verlekar contemplate further fine-tuning of the estimation model to improve classification strength. Performing additional fine-tuning of the classification algorithm based on the Group-CAM results is known and therefore, one of ordinary skill in the art of would be when looking to the improve the model of Verlekar would look to fine-tuning methods, such as the Group-CAM method of Zhang. Furthermore, Group-CAM is an improvement on Grad-CAM which is disclosed as algorithm used by the claimed invention in paragraph [0258] of the published specification.
Applicant argues that Verlekar and Sankai do not teach claims 11 and 12 because it is not explained why a person would retrofit Verlekar’s classifier with a system that determines which organ/category the disease relates to.
In response, a person of ordinary skill in the art would modify the classifier of Verlekar to determine which organ the disease relates to in order to discover the symptom of dementia at an early stage before such symptom is discovered by others, as Verlekar is concerned with classification across different pathologies and contemplates further fine-tuning to identify different types of gate pathologies (i.e. such as pathologies relating to the brain).
Applicant argues that Verlekar and Kalsi-Ryan do not teach claims 14-16 because the Office does not cite any disclosure that a person of ordinary skill would have (i) the requisite training labels and dataset, (ii) a suitable model architecture fore predicting JOA scores, or (III) a reason to expect predicable success in doing so.
In response, Kalsi-Ryan is used to teach that conditions may be classified by JOA score. Verlekar contemplates further fine-tuning of additional datasets and any datasets which classify condition by JOA score could be used as training data in order to estimate a JOA score associated with a gait rather than a binary indicator. One of ordinary skill would be motivated to use JOA scores rather than a binary indicator in order to enable early identification of the disease and monitoring disease progression.
Applicant argues that Verlekar and Ichikawa do not teach claims 17-18 as the Office does not show that Ichikawa teaches or suggest integrating the walking age numerical score into Verlekar’s pathology classifier and because the stated rationale is an outcome and not a technical rationale explaining why a PHOSITA would combine the references.
In response, Ichikawa is used to teach that conditions may be classified by a walking age numerical value. Verlekar contemplates further fine-tuning of additional datasets and any datasets which classify condition by a walking age numerical value could be used as training data in order to estimate a walking age numerical rather than or in addition to a binary indicator. One of ordinary skill would be motivated to use waling age numerical values rather than/in addition to a binary indicator in order tor to know an objective evaluation standard as to at which age's average level (walking age) their walking ability is, which is a need or problem known in the art as taught by Ichikawa.
Applicant argues that Verlekar and Delp does not teach claim 19 because the Office does not explain why a person of skill would merge Delp’s output engine into Verlekar’s system without redesign, where the combined system would produce an estimated condition and then provide a treatment/intervention, information according to the intervention, or why a person of ordinary skill would combine the output engine with Verlekar’s system in wat that preserves the required pipeline rather than Delp’s end-to-end approach.
In response, claim 19 recites “wherein the one or more processors are further configured to provide treatment or intervention or information according to the estimated condition”. Delp teaches to provide information relating to gate anomalies, such as treatment interventions. Gate anomalies can be a condition of the subject therefore providing treatment or intervention or information according to the estimated condition is known. Simply providing additional information on treatment or interventions after the classification of Verlekar would have been obvious to one of ordinary skill in the art in order to allow a user to quickly and efficiently evaluate motion of an individual. Furthermore, this is merely combining prior art elements according to known methods to yield predictable results.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Devin C. Hein whose telephone number is (303)297-4305. The examiner can normally be reached 9:00 AM - 5:00 PM M-F MDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason B. Dunham can be reached at (571) 272-8109. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEVIN C HEIN/Examiner, Art Unit 3686
Prosecution Insights