DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
The restriction requirement, as set forth in the Office action mailed on July 13th, 2026, has been reconsidered in view of the preliminary amendment to the claims, filed August 11th, 2026. The restriction requirement is hereby withdrawn. In view of the above noted withdrawal of the restriction requirement, applicant is advised that if any claim presented in a continuation or divisional application is anticipated by, or includes all the limitations of, a claim that is allowable in the present application, such claim may be subject to provisional statutory and/or nonstatutory double patenting rejections over the claims of the instant application. Once a restriction requirement is withdrawn, the provisions of 35 U.S.C. 121 are no longer applicable. See In re Ziegler, 443 F.2d 1211, 1215, 170 USPQ 129, 131- 32 (CCPA 1971). See also MPEP § 804.01.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: “12” in fig. 2(a). The first notification screen in fig. 2(a) should be the reference character “11”. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“A motion detector that captures images of a subject ...” in claim 1.
“An answer detector that detects answers to questions on a predetermined cognitive evaluation by the subject performing the predetermined task” in claim 1.
“An evaluator that outputs motion features based on the frames and evaluates a cognitive function of the subject ...” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The motion detector is defined, in para. [0046-0047, 0049] of the published specification (see US 20250049382 A1), as a CCD image sensor, a CMOS image sensor, a range sensor and a processor and storage.
The answer detector is defined, in para. [0051-0052, 0147-0150] of the published specification (see US 20250049382 A1), as answer switches, a left-hand answer switch and a right-hand answer switch, line-of-sight detector, a sound collector.
The evaluator is defined, in para. [0069, 0073, 0075] of the published specification (see US 20250049382 A1), as a storage and a processor.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION. —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 10 recites the limitation "the neural network being built by a trained model used in the cognitive function evaluation system according to claim 1" in lines 2-3. It is unclear what the trained model is or how the trained model is used in the cognitive function evaluation system because claim 1 does not recite the use of a trained model or a neural network. It is suggested to amend claim 1 to recite the that the evaluator comprises a trained model (or neural network) to clarify how the trained model is used in cognitive function evaluation system according to claim 1.
Claim 11 recites the limitation “a learning method that determines parameter values for a neural network for use in training a trained model used in the cognitive function evaluation system according to claim 1” in lines 1-3. It is unclear what the trained model is and how the trained model is used in the cognitive function evaluation system because claim 1 does not recite the use of a trained model or a neural network. It is suggested to amend claim 1 to recite that the evaluator comprises a trained model (or neural network) to clarify how the trained model is used in cognitive function evaluation system according to claim 1.
Claim Rejections - 35 USC § 101
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statuto0ry subject matter. The claim(s) as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea. A streamlined analysis of claims 1 follows.
STEP 1
Regarding claim 1, the claim recites a series of structural elements, including a system. Thus, the claim is directed to a machine, which is one of the statutory categories of invention.
STEP 2A, PRONG ONE
The claim is then analyzed to determine whether it is directed to any judicial exception. The steps of:
an evaluator that outputs motion features based on the frames and evaluates a cognitive function of the subject based on the motion features and the answers detected by the answer detector;
the motion features representing a feature of a spatial positional relationship of joints included in the three-dimensional human skeleton model and features of respective temporal variations of each of the joints;
wherein the predetermined task includes a physical task that requires the subject to perform a predetermined behavior, and a cognitive task that requires the subject to answer the questions on the predetermined cognitive examination, and the motion detector captures the images of the subject performing the physical task to generate the frames.
set forth a judicial exception. These steps describe a concept performed in the human mind (including an observation, evaluation, judgment, opinion) (evaluate a cognitive function) and/or mathematical concepts (motion features). Thus, the claim is drawn to Mental Process and/or Mathematical Concepts, which is an Abstract Idea. Furthermore, the step of performing the predetermined task is directed to managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions). Thus, the claim is drawn to a Mental Process and Certain methods of organizing human activity, which is an Abstract Idea.
STEP 2A, PRONG TWO
Next, the claim as a whole is analyzed to determine whether the claim recites additional elements that integrate the judicial exception into a practical application. The claim fails to recite an additional element or a combination of additional elements to apply, rely on, or use the judicial exception in a manner that imposes a meaningful limitation on the judicial exception. Claim 1 recites a motion detector that captures images of a subject performing a predetermined task to generate frames representing a three-dimensional human skeleton model that moves according to motion of the subject whose images have been captured and an answer detector that detects answers to questions on a predetermined cognitive examination by the subject performing the predetermined task, which is merely adding insignificant extra-solution activity to the judicial exception (MPEP 2106.05(g)). The captured images and detected answers does not provide an improvement to the technological field, the method does not effect a particular treatment or effect a particular change based on the captured images and detected answers, nor does the method use a particular machine to perform the Abstract Idea.
Regarding claim 1, the system recited in the claim is a generic system comprising generic components configured to perform the abstract idea. The recited motion detector and answer detector are generic sensors configured to perform pre-solutional data gathering activity and the evaluator is configured to perform the Abstract Idea. According to section 2106.05(f) of the MPEP, merely using a computer as a tool to perform an abstract idea does not integrate the Abstract Idea into a practical application.
STEP 2B
Next, the claim as a whole is analyzed to determine whether any element, or combination of elements, is sufficient to ensure that the claim amounts to significantly more than the exception. Besides the Abstract Idea, the claim recites additional steps of:
a motion detector that captures images of a subject performing a predetermined task to generate frames representing a three-dimensional human skeleton model that moves according to motion of the subject whose images have been captured;
an answer detector that detects answers to questions on a predetermined cognitive examination by the subject performing the predetermined task;
an evaluator
The capturing and detecting steps are well-understood, routine and conventional activities for those in the field of medical diagnostics. Further, the capturing and detecting steps are each recited at a high level of generality such that it amounts to insignificant pre-solution activity, e.g., mere data gathering step necessary to perform the Abstract Idea. When recited at this high level of generality, there is no meaningful limitation, such as a particular or unconventional step that distinguishes it from well-understood, routine, and conventional data gathering and comparing activity engaged in by medical professionals prior to Applicant's invention. Furthermore, it is well established that the mere physical or tangible nature of additional elements such as the obtaining and comparing steps do not automatically confer eligibility on a claim directed to an abstract idea (see, e.g., Alice Corp. v. CLS Bank Int'l, 134 S.Ct. 2347, 2358-59 (2014)).
Consideration of the additional elements as a combination also adds no other meaningful limitations to the exception not already present when the elements are considered separately. Unlike the eligible claim in Diehr in which the elements limiting the exception are individually conventional, but taken together act in concert to improve a technical field, the claim here does not provide an improvement to the technical field. Even when viewed as a combination, the additional elements fail to transform the exception into a patent-eligible application of that exception. Thus, the claim as a whole does not amount to significantly more than the exception itself. The claim is therefore drawn to non-statutory subject matter.
Regarding claim 1, the system recited in the claim is a generic system comprising generic components configured to perform the abstract idea (as evidenced by - the Non-patent literature of record; Okura et al., [Paper] Automatic Collection of Dual-task Human Behavior for Analysis of Cognitive Function, ITE Transactions on Media Technology and Applications, 2018, Volume 6, Issue 2, Pages 138-150, Released on J-STAGE April 01, 2018, Online ISSN 2186-7364, https://doi.org/10.3169/mta.6.138; Knapp (US 20030196357 A1) which discloses in para. [0050, 0064] conventional remotely actuated sensors for operation, e.g. an infrared or other type of motion detector 44 and/or a sound detector 46 and conventional audio and video systems with integral and/or detachable speakers, clocks, timers and alarms, remote actuation, lighting, communications, weather stations, computer terminals and displays). The recited motion detector and answer detector are generic sensors configured to perform pre-solutional data gathering activity, and the evaluator is configured to perform the Abstract Idea. According to section 2106.05(f) of the MPEP, merely using a computer as a tool to perform an abstract idea does not integrate the Abstract Idea into a practical application.
The dependent claims also fail to add something more to the abstract independent claims. Claims 2-8 and 10-11 are directed to more abstract ideas (mental processes and/or mathematical concepts) and claim 9 is directed to certain methods of organizing human activity, which does not add anything significantly more. The steps recited in the independent claim maintain a high level of generality even when considered in combination with the dependent claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4, and 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Yagi (US 20180078184 A1) in view of Li (20220138536).
Regarding claim 1, Yagi discloses a cognitive function evaluation system (dual-task performing ability evaluation system 1c/1d, figs. 11 & 14, Abstract, para. [0074, 0317]), comprising: a motion detector (motion detecting section 5 & 5a & 5b, figs. 11 & 14, para. [0121]) that captures images of a subject performing a predetermined task to generate frames representing a three-dimensional human skeleton model that moves according to motion of the subject whose images have been captured (“movement task”; “images the subject ... human body skeleton model”, para. [0070, 0120-0122, 0227, 0291], figs. 5-6 & 8), the frames being a series of frames generated in time order (“specific period of time ... third ... model ... fourth ... fifth”, para. [0141, 0158-0166, 0227]); an answer detector (answer detecting section 4 & 4a & 4b & 4c, figs. 11 & 14, para. [0086, 0108]) that detects answers (“switches ... generates ... a signal”; “microphone ... converts voice (an answer) uttered by the subject 2 performing an intelligence task to an electrical signal”, para. [0086, 0108, 0227], fig. 15) to questions on a predetermined cognitive examination by the subject performing the predetermined task (“intelligence task ... serial subtraction ... recitation of expressions”, para. [0070-0071, 0078], fig. 15); and an evaluator (information processing device 3 & system controller 8, figs. 11 & 14) that outputs motion features based on the frames (“processing section 31 generates data indicating a human body skeleton model that moves along with the motion of the subject 2 ... display”, para. [0100, 0121-0122]) and evaluates a cognitive function of the subject based on the motion features and the answers detected by the answer detector (“evaluation step ... cognitive ability of the subject”; “at least one of the motion and the answers of the subject while performing the dual task step S207 is analyzed ... evaluation step S607 ... current cognitive ability of the subject or a current health degree of the subject's brain is determined”, para. [0074, 0314-0317], figs. 13 & 17), wherein the predetermined task (fig. 15) includes a physical task that requires the subject to perform a predetermined behavior (““walking in place”, “skipping”, or “running””, para. [0070, 0078], fig. 15), and a cognitive task that requires the subject to answer the questions on the predetermined cognitive examination (“intelligence task ... serial subtraction ... recitation of expressions”, para. [0070-0071, 0078], fig. 15), and the motion detector captures the images of the subject performing the physical task to generate the frames (“movement task”; “images the subject ... human body skeleton model”, para. [0070, 0120-0122, 0227, 0291], figs. 5-6 & 8).
Yagi does not expressly disclose the motion features representing a feature of a spatial positional relationship of joints included in the three-dimensional human skeleton model and features of respective temporal variations of each of the joints.
However Li directed to methods, devices, and non-transitory computer readable storage medium for recognizing a human action using a graph convolutional network (GCN) discloses motion features representing a feature of a spatial positional relationship of joints included in the three-dimensional human skeleton model and features of respective temporal variations of each of the joints (“joint poses ... coordinates ... spatial and temporal patterns from data”; “skeleton sequence 380 in a time lapse fashion”, para. [0039-0040, 0057], figs. 3B & 6A).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yago such that the motion features representing a feature of a spatial positional relationship of joints included in the three-dimensional human skeleton model and features of respective temporal variations of each of the joints, in view of the teachings of Li, as this would aid in monitoring elderly care patients by recognizing human actions based on the spatial and temporal patterns of joint poses (Li, para. [0098]).
Regarding claim 4, Yagi, as modified by Li hereinabove, discloses the cognitive function evaluation system according to claim 1, wherein the evaluator (information processing device 3, fig. 2) determines a cognitive function score indicating a cognitive ability of the subject (“evaluation step ... cognitive ability ... Mini Mental State Examination (MMSE) score or a Hasegawa's Dementia Scale”, para. [0074, 0116-0117], Table II, figs. 28-29).
Regarding claim 7, Yagi, as modified by Li hereinabove, discloses the cognitive function evaluation system according to claim 1. Yagi, as modified by Li hereinabove, does not expressly disclose wherein the evaluator includes a motion feature extractor that extracts the motion features by: generating respective spatial graphs for the frames, each of the respective spatial graphs indicating respective spatial positional relationships of each of the joints included in the three- dimensional human skeleton model; convolving the respective spatial graphs; generating time graphs across the frames, each of the time graphs representing respective variations in an identical joint between each adjacent frames; and convolving the time graphs.
However, Li directed to methods, devices, and non-transitory computer readable storage medium for recognizing a human action using a graph convolutional network (GCN) discloses wherein the evaluator (computer system 200, para. [0049]) includes a motion feature extractor (feature extractor 600, fig. 6) that extracts the motion features (“spatial-temporal GCN (ST-GCN)”; “extracts”, para. [0068-0071]) by: generating respective spatial graphs for the frames (“generate a skeleton sequence 380 in a time lapse fashion”; “feature extractor 600 ... graph network for skeleton data”, para. [0057-0058, 0069], fig. 6), each of the respective spatial graphs indicating respective spatial positional relationships of each of the joints included in the three- dimensional human skeleton model (“time points in sequence ... the plurality of joint poses may include joint coordinates in a form of three-dimension coordinates, for example (x, y, z)”; “spatial-temporal graph with the joints as graph nodes”, para. [0060-0062, 0069-0074]); convolving the respective spatial graphs (“620 may include one or more convolution layer”; “spatial graph convolution”, para. [0069-0071]); generating time graphs across the frames (“construct a spatial-temporal graph with the joints as graph nodes and natural connectivities in both human body structures and time as graph edges”, para. [0069-0070, 0074]), each of the time graphs representing respective variations in an identical joint between each adjacent frames (“weighted average of neighboring features for each joint”; “spatial-temporal graph with the joints as graph nodes and natural connectivities ... N joints and T frames”, para. [0071, 0074-77], fig. 6); and convolving the time graphs (“Each ST-GCN block contains a spatial graph convolution followed by a temporal graph convolution, which alternatingly extracts spatial and temporal features”, para. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li hereinabove, such that the evaluator includes a motion feature extractor that extracts the motion features by: generating respective spatial graphs for the frames, each of the respective spatial graphs indicating respective spatial positional relationships of each of the joints included in the three- dimensional human skeleton model; convolving the respective spatial graphs; generating time graphs across the frames, each of the time graphs representing respective variations in an identical joint between each adjacent frames; and convolving the time graphs, in view of the teachings of Li, as this would aid in recognizing a human action by alternatingly extracts spatial and temporal features.
Regarding claim 8, Yagi, as modified by Li hereinabove, discloses the cognitive function evaluation system according to claim 7. Yagi, as modified by Li hereinabove, does not expressly disclose wherein: the evaluator includes a plurality of motion feature extractors each of which corresponds to the motion feature extractor; each of the plurality of motion feature extractors is supplied with corresponding frames for each time the predetermined task is performed a plurality of times continuously.
However, Li discloses wherein: the evaluator (computer system 200, para. [0049])
includes a plurality of motion feature extractors (“or more GCN block”; “number of ST-GCN blocks in a feature extractor ... 13”, para. [0069, 0071-0072]) each of which corresponds to the motion feature extractor (feature extractor 600, fig. 6); each of the plurality of motion feature extractors is supplied with corresponding frames for each time the predetermined task is performed a plurality of times continuously (as seen in fig. 6A, “frames over a period of time”; “T frames ... frame No. from 1 to T”, para. [0059, 0072-0078]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li hereinabove, such that the evaluator includes a plurality of motion feature extractors each of which corresponds to the motion feature extractor; each of the plurality of motion feature extractors is supplied with corresponding frames for each time the predetermined task is performed a plurality of times continuously, in view of the teachings of Li, as this would aid in recognizing a human action by alternatingly extracts spatial and temporal features.
Yagi, as modified by Li hereinabove, discloses the evaluator evaluates the cognitive function of the subject based on the motion features acquired from each of the plurality of motion feature extractors and the answers detected by the answer detector (“evaluation step ... cognitive ability of the subject”; “at least one of the motion and the answers of the subject while performing the dual task step S207 is analyzed ... evaluation step S607 ... current cognitive ability of the subject or a current health degree of the subject's brain is determined”, para. [0074, 0314-0317], figs. 13 & 17).
Regarding claim 9, Yagi, as modified by Li hereinabove, discloses the cognitive function evaluation system according to claim 1, wherein: the predetermined task includes a dual task that requires the subject to perform the physical task and the cognitive task simultaneously (“dual task ... simultaneously performs the movement task and the intelligence task”, para. [0069]); the motion detector captures images of the subject performing the dual task (“movement task”; “images the subject ... human body skeleton model”; “motion and answers of the subject performing the dual task is detected”, para. [0070, 0120-0122, 0227, 0291, 0314], figs. 5-6 & 8 & 17); and the answer detector detects answers by the subject performing the dual task (“switches ... generates ... a signal”; “microphone ... converts voice (an answer) uttered by the subject 2 performing an intelligence task to an electrical signal”; “motion and answers of the subject performing the dual task is detected”, para. [0086, 0108, 0227, 0314], fig. 17).
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Yagi in view of Li, as applied to claim 1 above, and further in view of Sano (US 20200042323 A1).
Regarding claim 2, Yagi, as modified by Li hereinabove, discloses the cognitive function evaluation system according to claim 1. Yagi further discloses a cognitive function score indicating a cognitive ability of the subject (“MMSE score or a Hasegawa's Dementia Scale”, para. [0116-0117], figs. 20 & 28).
Yagi, as modified by Li hereinabove, does not expressly disclose wherein the evaluator classifies the cognitive function of the subject into a class in which a cognitive function score indicating a cognitive ability of the subject is less than or equal to a threshold or a class in which the cognitive function score is greater than the threshold.
However, Sano directed to a technique for evaluating human cognitive and motor functions by a plurality of hand movement tasks discloses an evaluator (generation device 1 including a control unit 101, a storage unit 102, fig. 2), wherein the evaluator (generation device 1 including a control unit 101, a storage unit 102, fig. 2) classifies the cognitive function of the subject into a class in which a cognitive function score indicating a cognitive ability of the subject is less than or equal to a threshold or a class in which the cognitive function score is greater than the threshold (“score of MMSE ... normal control ... cognitive impairment ... discriminate a group of subjects into two groups by the threshold N of MMSE with the scores of MMSE”; “threshold N max shown here is MMSE that can perform two-group discrimination ... normal control and mild cognitive impairment”, para. [0099-0101, 0118]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li hereinabove, such that the evaluator classifies the cognitive function of the subject into a class in which a cognitive function score indicating a cognitive ability of the subject is less than or equal to a threshold or a class in which the cognitive function score is greater than the threshold, in view of the teachings of Sano, as this would aid in discriminating the subject as one of normal control and mild cognitive impairment based on the MMSE score.
Regarding claim 3, Yagi, as modified by Li and Sano hereinabove, discloses the cognitive function evaluation system according to claim 2. Yagi, as modified by Li and Sano hereinabove, does not expressly disclose wherein according to the threshold that is set in advance, the evaluator classifies the subject into a class of dementia or a class of mild cognitive impairment and non-dementia, or into a class of dementia and mild cognitive impairment or a class of non-dementia.
However, Sano discloses wherein according to the threshold that is set in advance (“predetermined threshold”, Abstract, claim 1, para. [0099-0101]), the evaluator (generation device 1 including a control unit 101, a storage unit 102, fig. 2) classifies the subject into a class of dementia or a class of mild cognitive impairment and non-dementia (“threshold N max shown here is MMSE that can perform two-group discrimination ... mild cognitive impairment and cognitive impairment ... severe patients”, para. [0118-0119]), or into a class of dementia and mild cognitive impairment or a class of non-dementia (“threshold N max shown here is MMSE that can perform two-group discrimination ... normal control and mild cognitive impairment”, para. [0099-0101, 0118]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li and Sano hereinabove, such that according to the threshold that is set in advance, the evaluator classifies the subject into a class of dementia or a class of mild cognitive impairment and non-dementia, or into a class of dementia and mild cognitive impairment or a class of non-dementia, in view of the teachings of Sano, as this would aid in discriminating the subject as one of a normal control and a mild cognitive impairment or as one of a very severe cognitive patient and a milder cognitive patient based on the MMSE score.
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Yagi in view of Li, as applied to claim 1 above, and further in view of Arzy (US 20190167179 A1).
Regarding claim 5, Yagi, as modified by Li hereinabove, discloses the cognitive function evaluation system according to claim 1. Yagi, as modified by Li hereinabove, does not expressly disclose wherein the evaluator classifies the subject into a class of dementia, a class of mild cognitive impairment, or a class of non-dementia.
However, Arzy directed to a method of neuropsychological analysis discloses an evaluator (client computer 30 & server computer 50, fig. 2), wherein the evaluator (client computer 30 & server computer 50, fig. 2) classifies the subject into a class of dementia, a class of mild cognitive impairment, or a class of non-dementia (“classification groups ... Mild Cognitive Impairment (MCI) classification group ... a classification group encompassing one or more other dementias, and an age related cognitive decline classification group”, para. [0098, 0117]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li hereinabove, such that the evaluator classifies the subject into a class of dementia, a class of mild cognitive impairment, or a class of non-dementia, in view of the teachings of Arzy, as this would aid in assessing the cognitive function of a subject by classifying subjects into one of a plurality cognitive function classification groups characterized by a cognitive function or dysfunction.
Regarding claim 6, Yagi, as modified by Li and Arzy hereinabove, discloses the cognitive function evaluation system according to claim 5. Yagi, as modified by Li and Arzy hereinabove, does not expressly disclose wherein the evaluator classifies the subject into any one of at least two types of the dementia.
However, Arzy directed to a method of neuropsychological analysis discloses an evaluator (client computer 30 & server computer 50, fig. 2), wherein the evaluator (client computer 30 & server computer 50, fig. 2) classifies the subject into any one of at least two types of the dementia (“an Alzheimer's disease (AD) classification group ... classification group encompassing one or more other dementias ...two or more AD or MCI classification groups can be defined, for different severities of the AD or MCI”, para. [0098, 0117]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li and Arzy hereinabove, such that the evaluator classifies the subject into any one of at least two types of the dementia, in view of the teachings of Arzy, as this would aid in defining the severity of the cognitive dysfunction.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Yagi in view of Li, as applied to claim 1 above, and further in view of Diamond (US 20070092888 A1).
Regarding claim 10, Yagi, as modified by Li hereinabove, discloses the cognitive function evaluation system according to claim 1. Yagi, as modified by Li hereinabove, does not disclose a learning method that determines parameter values for a neural network that classifies a subject as positive or negative, the neural network being built by a trained model, wherein the learning method comprises determining the parameter values through a loss function that optimizes a sum of sensitivity and specificity, the sensitivity describing a rate of the subject being identified as true positive, the specificity describing a rate of the subject being identified as true negative.
However, Diamond directed to algorithmic techniques for relating markers (para. [0043]) discloses a learning method (“learning or optimization techniques”, para. [0043, 0135]) that determines parameter values for a neural network that classifies a subject as positive or negative (“parameters associated with the learning or optimization technique ... positive or negative predictive accuracy”, para. [0043, 0134-0135]), the neural network being built by a trained model (“learning or optimization techniques”; “learning algorithm used to construct the classifier”, para. [0043, 0116, 0134-0135]), wherein the learning method (“learning or optimization techniques”, para. [0043, 0135]) comprises determining the parameter values through a loss function that optimizes a sum of sensitivity and specificity (“optimize these selected markers ... maximizing the product of sensitivity and specificity of the selected markers, or positive or negative predictive accuracy”; “sum ... specificity ... sensitivity”, para. [0135, 0190, 0210]), the sensitivity describing a rate of the subject being identified as true positive, the specificity describing a rate of the subject being identified as true negative (“sensitivity ... with the disease ... correctly ... specificity ... without the disease ... correctly”; “positive or negative predictive accuracy”, para. [0096, 0190]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li hereinabove, to comprise a learning method that determines parameter values for a neural network that classifies a subject as positive or negative, the neural network being built by a trained model used in the cognitive function evaluation system, wherein the learning method comprises determining the parameter values through a loss function that optimizes a sum of sensitivity and specificity, the sensitivity describing a rate of the subject being identified as true positive, the specificity describing a rate of the subject being identified as true negative, in view of the teachings of Diamond, as this would aid in providing an algorithmic technique for correlating a diagnostic or prognostic indicator/relating markers (dual-task performing ability of a subject) to a condition or disease (degree of dementia) (Diamond, para. [0043]).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yagi in view of Li, as applied to claim 1 above, and further in view of Chabanne (US 20190294864 A1).
Regarding claim 11, Yagi discloses the cognitive function evaluation system according to claim 1.
Yagi, as modified by Li, hereinabove does not expressly disclose wherein the neural network includes a first network and a second network that convolve spatial graphs and convolve time graphs, the spatial graphs representing respective spatial positional relationships of joints included in a three-dimensional human skeleton model that moves according to motion of a subject whose images have been captured, the time graphs representing respective temporal variations of the joints included in the three-dimensional human skeleton model.
However, Li directed to methods, devices, and non-transitory computer readable storage medium for recognizing a human action using a graph convolutional network (GCN) discloses a neural network (“neural network”, para. [0038, 0069-0071], 600, fig. 6A) wherein the neural network includes a first network (“graph network”, para. [0038, 0069-0071], 610, fig. 6A) and a second network (“convolution layer” para. [0038, 0069-0071], 320, fig. 6A) that convolve spatial graphs and convolve time graphs (“convolution”, para. [0069-0071]), the spatial graphs representing respective spatial positional relationships of joints included in a three-dimensional human skeleton model that moves according to motion of a subject whose images have been captured (“time points in sequence ... the plurality of joint poses may include joint coordinates in a form of three-dimension coordinates, for example (x, y, z)”; “spatial-temporal graph with the joints as graph nodes”, para. [0060-0062, 0069-0074]), the time graphs representing respective temporal variations of the joints included in the three-dimensional human skeleton model (“weighted average of neighboring features for each joint”; “spatial-temporal graph with the joints as graph nodes and natural connectivities ... N joints and T frames”, para. [0071, 0074-77], fig. 6).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li hereinabove, such that the neural network includes a first network and a second network that convolve spatial graphs and convolve time graphs, the spatial graphs representing respective spatial positional relationships of joints included in a three-dimensional human skeleton model that moves according to motion of a subject whose images have been captured, the time graphs representing respective temporal variations of the joints included in the three-dimensional human skeleton model, in view of the teachings of Li, as this would aid in recognizing a human action by alternatingly extracting spatial and temporal features (Li, para. [0069-0071]) for evaluating the dual-task performing ability of a subject (Yagi, para. [0090-0091])
Yagi, as modified by Li hereinabove, does not disclose a learning method that determines parameter values for a neural network for use in training a trained model, and the learning method includes determining parameter values of the first network by learning from data entered into the first network, and determining parameter values of the second network by learning from data entered into the second network after setting the determined parameter values of the first network as initial values of parameter values of the second network.
However, Chabanne discloses a learning method that determines parameter values for a neural network for use in training a trained model (Abstract, fig. 2), wherein the neural network includes a first network (first CNN, para. [0059-0061], fig. 2) and a second network (second CNN, para. [0062-0064], fig. 2), and the learning method (Abstract, fig. 2) includes determining parameter values of the first network by learning from data entered into the first network (“already-classified ... learning data ... learns the parameters of a first CNN”, para. [0059], fig. 2), and determining parameter values of the second network by learning from data entered into the second network after setting the determined parameter values of the first network as initial values of parameter values of the second network (“converted to second CNNS ... learns the parameters of a last fully-connected layer (FC) of a second CNN ... parameters already learnt for the first CNNs are kept”, para. [0062-0066], fig. 2). Chabanne further discloses a trained model (third CNN, fig. 2, para. [0075, 0077]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yagi, as modified by Li hereinabove, to comprise a learning method that determines parameter values for a neural network for use in training a trained model used in the cognitive function evaluation system and the learning method includes determining parameter values of the first network by learning from data entered into the first network, and determining parameter values of the second network by learning from data entered into the second network after setting the determined parameter values of the first network as initial values of parameter values of the second network, in view of the teachings of Chabanne, as this would aid in and classifying/evaluating a dual-task performing ability of a subject corresponding to a degree of dementia.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Okura et al., [Paper] Automatic Collection of Dual-task Human Behavior for Analysis of Cognitive Function, ITE Transactions on Media Technology and Applications, 2018, Volume 6, Issue 2, Pages 138-150, Released on J-STAGE April 01, 2018, Online ISSN 2186-7364, https://doi.org/10.3169/mta.6.138;
Yan, S., Xiong, Y., & Lin, D. (2018). Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.12328;
T. Ahmad, L. Jin, X. Zhang, S. Lai, G. Tang and L. Lin, "Graph Convolutional Neural Network for Human Action Recognition: A Comprehensive Survey," in IEEE Transactions on Artificial Intelligence, vol. 2, no. 2, pp. 128-145, April 2021, doi: 10.1109/TAI.2021.3076974.
Bae (KR20210086881A English Translation) directed to a system and device for motor-cognitive dual task training and speed-accuracy relationship evaluation.
Adeli-Mosabbeb (US 11918370 A1) directed to estimation of Parkinson's Disease severity from videos using MDS-UPDRS.
Chen (US 20200381083 A1) discloses an optimizer that works in tandem with an AI program to update the model in response to the output of the loss function by combining the loss function and model parameters (para. [0078]).
El-Zehiry (US 20230259820 A1) directed to machine learning applications such as used in medical imaging.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW ELI HOFFPAUIR whose telephone number is (571)272-4522. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Marmor II can be reached at (571) 272-4730. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.E.H./Examiner, Art Unit 3791
/AURELIE H TU/Primary Examiner, Art Unit 3791
Prosecution Insights