DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This communication is in Response to the Applicant’s submission filed 13 March 2025 [hereinafter Response], which has been entered, where:
Claims 1, 6, 8, 10, 19, and 21 have been amended.
Claims 3 and 12 have been cancelled.
Claims 1, 2, 4-11, and 13-21 are pending.
Claims 1, 2, 4-11, and 13-21 are rejected.
Claim Rejections - 35 U.S.C. § 101
3. 35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
4. Claims 1, 2, 4-11, and 13-21 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a “machine learning system,” which is a machine, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites “[(a)1 a spatio-temporal action recognition module configured] to [(a.1)] process key-point coordinates over time obtained from video frames . . . to recognize an action taken by a subject,” “[(a) the spatio-temporal action recognition module] being configured to [(a.2)] generate a plurality of attention weights,” “[(a) the spatio-temporal action recognition module] having a first attention mechanism over time frames and a second attention mechanism over human key-points that are configured to [(a.5)] identify an engagement of a human key-point coordinate in a respective time frame for the recognized action,” and “[(b) a fuzzy intensity index calculation module] . . . to [(b.2)] produce an intensity index associated with the recognized action.” The limitations of “process,” “recognize,” “generate,” “identify,” and “produce,” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly are a mental process, (MPEP § 2106.04(a)(2) sub III), and is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)). The claim recites more details or specifics to the abstract idea of “[(a.5)] to identify an engagement of a human key-point coordinate,” where “[(a.6)] the human key-point coordinate being associated with at least one of the plurality of attention weights,” and accordingly, is merely more specific to the abstract idea. The claim also recites more details or specifics to the abstract idea of “produce an intensity index,” where the “[(b.2.1) intensity index being produced based at least in part on] inputting the plurality of attention weights to a first fuzzier and inputting an initial intensity score into a second fuzzifier,” in which a fuzzifier operates to convert “crisp values” to “fuzzy values” to regulate the degree of membership, the activity of “converting” being a mental process, (MPEP § 2106.04(a)(2) sub III). Thus claim 1 recites an abstract idea.
Under Step 2A Prong Two, the abstract idea of the claim is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include “a processor configured to perform an integrated model,” “a memory device in communication with the processor,” “[(a)] a spatio-temporal action recognition module,” “[(b)] a fuzzy intensity index calculation module,” and a “video source connected via a video connection.” These are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), which does not serve to integrate the abstract idea into a practical application. The claim also recites a “machine learning system” and an “integrated model,” which are recited at a high-level of generality and are thus generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not integrate the abstract idea into a practical application. Also, the claim recites more details or specifics to the additional element of the “[(a)] spatio-temporal action recognition module,” comprising “[(a.3)] a spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence,” which is recited at a high-level of generality, and is a generic computer component used to implement the abstract idea into a practical application, (MPEP § 2106.05(f)), that does not integrate the abstract idea into a practical application. The claim also recites additional details or specifics to the additional element of the “[(a)] spatio-temporal action recognition module] . . . [(a.4)] the video frames input being obtained from a video source connected via a video connection or from a memory device,” and accordingly, is merely more specific to the additional element. The claim also recites additional elements of “[(b) a fuzzy intensity calculation model] . . . [(b.1)] to receive attention weights output,” which is a pre-solution, insignificant extra-solution activities of mere data reception, (MPEP § 2106.05(g)), that does not integrate the abstract idea into a practical application. Therefore, claim 1 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements include “a processor configured to perform an integrated model,” “a memory device in communication with the processor,” “[(a)] a spatio-temporal action recognition module,” “[(b)] a fuzzy intensity index calculation module,” and a “video source connected via a video connection.” These are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), which does not amount to significantly more than the abstract idea. The claim also recites a “machine learning system” and an “integrated model,” which are recited at a high-level of generality, and thus, are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not amount to significantly more than the abstract idea. Also, the claim recites more details or specifics to the additional element of the “[(a)] spatio-temporal action recognition module,” comprising “[(a.3)] a spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence,” which is recited at a high-level of generality, and is a generic computer component used to implement the abstract idea into a practical application, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites additional details or specifics to the additional element of the “[(a)] spatio-temporal action recognition module] . . . [(a.4)] the video frames input being obtained from a video source connected via a video connection or from a memory device,” and accordingly, is merely more specific to the additional element. The claim also recites additional elements of “[(b) a fuzzy intensity calculation model] . . . [(b.1)] to receive attention weights output,” which is a pre-solution well-understood and routine activity of storing and retrieving data from memory, (MPEP § 2106.05(d) sub II.iv), that does not amount to significantly more than the abstract idea. Therefore, claim 1 is subject-matter ineligible.
Claim 10 recites a “machine learning method,” which is a process, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101). However, under Step 2A Prong One, the claim recites “(a) performing a spatio-temporal action recognition algorithm [(a.1)] that processes key-point coordinates over time obtained from video frames . . . to recognize an action taken by a subject,” “[(a) the spatio-temporal action recognition algorithm] being configured to [(a.2)] generate a plurality of attention weights,” “[(a) the spatio-temporal action recognition algorithm] having a first attention mechanism over time frames and a second attention mechanism over human key-points that are configured to [(a.4)] identify an engagement of a human key-point coordinate in a respective time frame for the recognized action,” and “[(b) a fuzzy intensity index calculation algorithm] . . . [(b.2)] to produces an intensity index associated with the recognized action.” The limitations of “performing,” “process,” “recognize,” “generate,” “identify,” and “produce,” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly are a mental process, (MPEP § 2106.04(a)(2) sub III), and is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)). The claim recites more details or specifics to the abstract idea of “[(a.4)] to identify an engagement of a human key-point coordinate,” where “[(a.5)] the human key-point coordinate being associated with at least one of the plurality of attention weights,” and accordingly, is merely more specific to the abstract idea. The claim also recites more details or specifics to the abstract idea of “[(b.1)] produce an intensity index,” where the “[(b.2.1) intensity index being produced based at least in part on] inputting the plurality of attention weights to a first fuzzier and inputting an initial intensity score into a second fuzzifier,” in which a fuzzifier operates to convert “crisp values” to “fuzzy values” to regulate the degree of membership, the activity of “converting” being a mental process, (MPEP § 2106.04(a)(2) sub III). Thus claim 10 recites an abstract idea.
Under Step 2A Prong Two, the abstract idea of the claim is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include “one or more processors” These are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), which does not serve to integrate the abstract idea into a practical application. Also, the claim recites more details or specifics to the additional element of the “[(a)] spatio-temporal action recognition algorithm,” comprising “[(a.3)] a trained spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence,” which is recited at a high-level of generality, and thus is a generic computer component used to implement the abstract idea into a practical application, (MPEP § 2106.05(f)), that does not integrate the abstract idea into a practical application. The claim also recites additional elements of “[(b) a fuzzy intensity calculation algorithm] . . . [(b.1)] that receives attention weights output [(a) by the spatio-temporal action recognition algorithm],” which is a pre-solution, insignificant extra-solution activities of mere data reception, (MPEP § 2106.05(g)), that does not integrate the abstract idea into a practical application. Therefore, claim 10 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements include “one or more processors.” These are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), which does not amount to significantly more than the abstract idea. Also, the claim recites more details or specifics to the additional element of the “[(a)] spatio-temporal action recognition algorithm,” comprising “[(a.3)] a trained spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence,” which is recited at a high-level of generality, and thus is a generic computer component used to implement the abstract idea into a practical application, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites additional elements of “[(b) a fuzzy intensity index calculation algorithm] . . . [(b.1)] that receives the plurality of attention weights output [(a) by the spatio-temporal action recognition algorithm],” which is a pre-solution well-understood and routine activity of storing and retrieving data from memory, (MPEP § 2106.05(d) sub II.iv), that does not amount to significantly more than the abstract idea. Therefore, claim 10 is subject-matter ineligible.
Claim 19 recites a “machine learning computer program embodied on a non-transitory computer-readable medium,” which is a product, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites “(a) performing a spatio-temporal action recognition algorithm [(a.1)] that processes key-point coordinates over time obtained from video frames . . . to recognize an action taken by a subject,” “[(a) the spatio-temporal action recognition algorithm] being configured to [(a.2)] generate a plurality of attention weights,” “[(a) the spatio-temporal action recognition algorithm] having a first attention mechanism over time frames and a second attention mechanism over human key-points that are configured to [(a.4)] identify an engagement of a human key-point coordinate in a respective time frame for the recognized action,” and “[(b) a fuzzy intensity index calculation algorithm] . . . to [(b.2)] produce an intensity index associated with the recognized action.” The limitations of “performing,” “process,” “recognize,” “generate,” “identify,” and “produce,” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly are a mental process, (MPEP § 2106.04(a)(2) sub III), and is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)). The claim recites more details or specifics to the abstract idea of “[(a.4)] to identify an engagement of a human key-point coordinate,” where “[(a.5)] the human key-point coordinate being associated with at least one of the plurality of attention weights,” and accordingly, is merely more specific to the abstract idea. The claim also recites more details or specifics to the abstract idea of “[(b.1)] produce an intensity index,” where the “[(b.1.2) intensity index being produced based at least in part on] inputting the plurality of attention weights to a first fuzzier and inputting an initial intensity score into a second fuzzifier,” in which a fuzzifier operates to convert “crisp values” to “fuzzy values” to regulate the degree of membership, the activity of “converting” being a mental process, (MPEP § 2106.04(a)(2) sub III). Thus claim 19 recites an abstract idea.
Under Step 2A Prong Two, the abstract idea of the claim is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include “one or more processors” These are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), which does not serve to integrate the abstract idea into a practical application. Also, the claim recites more details or specifics to the additional element of the “[(a)] spatio-temporal action recognition algorithm,” comprising “[(a.3)] a trained spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence,” which is recited at a high-level of generality, and thus is a generic computer component used to implement the abstract idea into a practical application, (MPEP § 2106.05(f)), that does not integrate the abstract idea into a practical application. The claim also recites additional elements of “[(b) a fuzzy intensity index calculation algorithm] . . . [(b.1)] to receive attention weights output [(a) by the spatio-temporal action recognition algorithm],” which is a pre-solution, insignificant extra-solution activities of mere data reception, (MPEP § 2106.05(g)), that does not integrate the abstract idea into a practical application. Therefore, claim 19 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements include a “non-transitory computer-readable medium.” This is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), which does not amount to significantly more than the abstract idea. Also, the claim recites more details or specifics to the additional element of the “[(a)] spatio-temporal action recognition algorithm,” comprising “[(a.3)] a trained spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence,” which is recited at a high-level of generality, and thus is a generic computer component used to implement the abstract idea into a practical application, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites additional elements of “[(b) a fuzzy intensity index calculation algorithm] . . . [(b.1)] that receives attention weights output [(a) by the spatio-temporal action recognition algorithm],” which is a pre-solution well-understood and routine activity of storing and retrieving data from memory, (MPEP § 2106.05(d) sub II.iv), that does not amount to significantly more than the abstract idea. Therefore, claim 19 is subject-matter ineligible.
Claim 21 recites a “machine learning-based method,” which is a process, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101). However, under Step 2A Prong One, the claim recites limitations of “b) extracting the pose of at least one person,” “c) recognizing the performed action using . . . an LSTM module,” “[(c.1)] . . . to generate a plurality of attention weights,” “[(c.2)] . . . having a first attention mechanism and a second attention mechanism that are configured to identify an engagement of a human key-point coordinate in a respective time frame for the recognized action,” and “d) recognizing an action intensity using the spatio-temporal distribution of the attention weights, fuzzy entropy measures and dynamically learned fuzzy logic rules.” The limitations of “extracting” and “recognizing” can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly are a mental process, ,” (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP ¶ 2106.04(a)(2)). The claim recites more details or specifics to the abstract idea of “[(c.2)] . . . identify an engagement,” where “[(c.3)] the human key-point coordinate being associated with at least one of the plurality of attention weights,” and the abstract idea of “d) recognizing an action intensity,” where “[(d.1)] the action intensity being recognized based at least in part on an intensity index,” and accordingly, are merely more specific to the respective abstract idea. The claim also recites more details or specifics to the abstract idea of “[d) recognizing an action intensity” where “[(d.2)] the intensity index being generated based at least in part on inputting the plurality of attention weights to a first fuzzifier and inputting an initial intensity score into a second fuzzifier,” in which a fuzzifier operates to convert “crisp values” to “fuzzy values” to regulate the degree of membership, the activity of “converting” being a mental process, (MPEP § 2106.04(a)(2) sub III). Thus, claim 21 is directed to an abstract idea.
Under Step 2A Prong Two, the abstract idea of claim 21 is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include “a spatio-temporal action recognition module that comprises an LSTM module,” which is an additional element of a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. The claim also recites “a) preparing a streaming video of at least one person in the group,” which such “preparing” is a pre-process insignificant extra-solution activity of data processing preparation, (MPEP § 2106.05(g)), that does not integrate the abstract idea into a practical application. The claim also recites “e) dynamically updating the spatio-temporal action recognition module as well as the fuzzy logic rules for further adaptation to a unique way an action intensity is performed,” which are post-processing insignificant extra-solution activities of data updating, (MPEP § 2106.05(g)), that does not integrate the abstract idea into a practical application. Thus, claim 21 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself, which include “a spatio-temporal action recognition module that comprises an LSTM module,” which is an additional element of a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites “a) preparing a streaming video of at least one person in the group,” which such “preparing” is a pre-processing, well-understood and conventional activity , (MPEP § 2106.05(g)), that does not integrate the abstract idea into a practical application. The claim also recites “e) dynamically updating the spatio-temporal action recognition module as well as the fuzzy logic rules for further adaptation to a unique way an action intensity is performed,” which are post-processing insignificant extra-solution activities of data updating, (MPEP § 2106.05(g)), that does not amount to significantly more than the abstract idea. The claim also recites “a) preparing a streaming video of at least one person in the group,” which such “preparing” is a pre-processing well-understood, routine, and conventional activity of selecting information based on types of information for analysis, (MPEP § 2106.05(d); see Electric Power Group LLC v. Alstom S.A., 830 F.3d 1350, 1354-55, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016)), that does not amount to significantly more than the abstract idea. The claim also recites “e) dynamically updating the spatio-temporal action recognition module as well as the fuzzy logic rules for further adaptation to a unique way an action intensity is performed,” which are post-processing, well-understood, routine, and conventional activities of updating fields in memory, (MPEP § 2106.05(d) sub II.iv), which does not amount to significantly more than the abstract idea. Thus, claim 21 is subject-matter ineligible.
Claim 2 depends from claim 1. Claim 11 depends from claim 10. Claim 20 depends from claim 19. The claims recite a “mental process” (claims 2, 11, and 20: “[a pre-processing module] . . . to transform the received video frames into the key-point coordinates over time”), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2) sub III). Also, the claim recites an additional element, (claims 2, 11, and 20: [a pre-processing module] . . . to receive video frames input to the machine learning system . . .”), which is an insignificant extra-solution activity of mere data gathering, (MPEP § 2106.05(f)), which does not integrate the abstract idea into a practical application, and also, is a well-understood, routine, and conventional activity of receiving and transmitting data over a network, (MPEP § 2106.05(d) sub II.i), that does not amount to significantly more than the abstract idea. Accordingly, claims 2, 11, and 20, are subject-matter ineligible.
Claim 4 depends directly or indirectly from claim 1. Claim 13 depends directly or indirectly from claim 10. The claims recite a mathematical concept, (claims 4 and 13: [wherein] . . . performs a kinetic fuzzy intensity analysis that processes the attention weights to calculate a fuzzy entropy associated with the recognized action”), which is one of the groupings of abstract idea. (MPEP § 2106.04(a)(2) sub I). The claims also recite more details or specifics of the additional element of the “fuzzy intensity calculation module,” (claims 4 and 13: [wherein] . . . includes a kinetic fuzzy intensity analysis module that performs . . .”), and accordingly, is merely more specific to the additional element, and is a generic computer component used to implement the abstract idea of “calculate a fuzzy entropy” (MPEP § 2106.05(g)), that does not serve to integrate the abstract idea into a practical application, nor does it amount to significantly more than the abstract idea. Thus, claims 4 and 13 are subject-matter ineligible.
Claims 5 and 6 depend directly or indirectly from claim 1. Claims 14 and 15 depend directly or indirectly from claim 10. The claims recite limitations directed to a “mathematical concept,” (claims 5 and 14: calculates the intensity index based at least in part on the calculated fuzzy entropy”; claims 6 and 15: comprises the first attention mechanism over time frames that calculates attention over time of the video frames and a second attention mechanism over human key-points that calculates attention over at least some of the key-point coordinates to produce first and second sets of the attention weights, respectively”), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2) sub I). The claims also recite more details or specifics of the additional element of the “fuzzy intensity index calculation module,” (claims 5 and 14: fuzzy intensity index calculation module includes a fuzzy inference module”), and the “spatio-temporal action recognition module,” (claims 6 and 14: “wherein the spatio-temporal action recognition module comprises a first attention mechanism . . . and a second attention mechanism . . . .”), and accordingly, are merely more specific to the additional element, and are generic computer components used to implement the abstract idea of “calculates the intensity index” and “calculates attention,” respectively, (MPEP § 2106.05(g)), that does not serve to integrate the abstract idea into a practical application, nor does it amount to significantly more than the abstract idea. Thus, claims 5, 6, 14, and 15 are subject-matter ineligible.
Claims 7 and 8 depend directly or indirectly from claim 1. Claims 16 and 17 depend directly or indirectly from claim 10. The claims recite more details or specifics of the abstract idea of “calculating the fuzzy entropy,” (claims 7 and 16: “the fuzzy entropy associated with the recognized action is calculated using the first and second sets of attention weights”; claims 8 and 17: “wherein the kinetic fuzzy intensity analysis module computes the initial intensity score based on the fuzzy entropy, and wherein the fuzzy inference module converts the initial intensity score and the first and second sets of attention weights into fuzzy sets using an adaptive membership function”), and thus, are merely more specific to the abstract idea in which generic computer components are used to implement. (MPEP § 2106.05(g)). Thus, claims 7, 8, 16, and 17 are subject-matter ineligible.
Claim 9 depends directly or indirectly from claim 1. Claim 18 depends directly or indirectly from claim 10. The claims recite limitations directed to a “mental process,” (claims 9 and 18: wherein the kinetic fuzzy intensity index calculation module uses truth values of the fuzzy sets to define fuzzy rules through which a final intensity index is determined by the fuzzy inference module), which is a grouping of abstract ideas. (MPEP § 2106.04(a)(2) sub III). The abstract idea of these claim is not integrated into a practical application, (see MPEP § 2106.04(d)), nor do they amount to significantly more than the abstract idea, (MPEP § 2106.05), because the claims recites no more than the abstract idea. Thus, claims 9 and 18 are subject-matter ineligible.
Response to Arguments
5. Examiner has fully considered Applicant’s arguments and/or amendments, and responds below, accordingly.
Claim Rejections – 35 U.S.C. § 101
6. Applicant submits that, under Step 2A Prong Two, that “[c]laims 1, 2, 4-11, and 13-21 integrate the alleged abstract idea into a practical application because the claims are directed to a technological improvement for training machine learning models for human action recognition and identifying an intensity level for the recognized action, which is an improvement in the field of computer vision.” (Response at p. 12-14).
Examiner’s Response:
Applicant submits that the instant claims are directed to “a technological improvement for a technological improvement for training machine learning models for human action recognition and identifying an intensity level for the recognized action, which is an improvement in the field of computer vision.” (Response at p. 12). Applicant points to the specification as a “drawback to the supervised deep learning approach of action recognition is that less focus is given to predicting the intensity of the action than to detect the action itself.” (Response at p. 12 (Specification ¶ 0003 (“Background”)). Also, Applicant points to the instant claims as reflecting the improvement.
Under Step 2A Prong Two, “integration” may be based on the improvements in the functioning of a computer or an improvement to any other technology or technical field. (MPEP § 2106.04(d)(1)). The evaluation requires, [i]n sum, that (1) the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. Next, (2) if the specification sets forth such an improvement, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement.
Examiner respectfully submits that the disclosure Background sets out an intended result, but is not directed to the specific improvement. The additional elements of the claim is directed to generic computer components (processor, memory device, a trained spatio-temporal LSTM) and the use of these components to implement the abstract idea, (MPEP § 2106.04(f)), that does not serve to integrate the abstract idea into a practical application, as is set out above in detail.
The specification does, however, disclose “[t]he integrated model is analyzed herein by applying it to videos of human actions with different action intensities to demonstrate that it is able to achieve an accuracy of 89 .16% on an intensity indexing generated dataset. The integrated model demonstrates the ability of a neuro-fuzzy inference module to effectively estimate the intensity index of human actions.” (Specification ¶ 0015). Portions of the Specification also set out “sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement,” such as Specification relating to “adding it to the attention distribution as fuzzy membership weights and computing their fuzzy entropy. The weights are the change of the coordinates' locations from the last frame multiplied by their corresponding attention weights. Using known fuzzy entropy methods, the fuzzy entropy of the attention vector can be calculated, which is indirectly related to intensity.” (Specification ¶ 0030 (emphasis added by Examiner)).
However, under the second leg of the evaluation, if the specification sets forth such an improvement, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. In this respect, the claims simply recite, generally, the use of a known fuzzy entropy method in relation to attention weights, which does not “reflect the disclosed improvement” under the Specification.
Accordingly, Applicant’s arguments under Step 2A Prong Two, directed to “integration of the abstract idea,” are unpersuasive.
7. “Applicant asserts that the claims of the present application are directed to patent eligible subject matter because the claims are directed to a technical improvement for the functionality of computing device, which is analogous to Example 39 of Subject Matter Eligibility Examples.” (Response at p. 15).
Examiner’s Response:
Examiner respectfully disagrees because “Example 39,” pertains to an exemplar claim that lacks an “abstract idea” under Step 2A Prong One.
Example 39 recites, recites, inter alia, "creating a first training set comprising [(a)] the collected set of digital facial images, [(b)] the modified set of digital facial images, and [(c)] a set of digital non-facial images;" and "creating a second training set for a second stage of training comprising [(a)] the first training set and [(b)] digital non-facial images that are incorrectly detected as facial images after the first stage of training." (Example 39 (example claim)). The analysis explains that “[t]he claim [of Example 39] does not recite any of the judicial exceptions enumerated in the 2019 PEG. . . . Thus, the claim is eligible [under Step 2A Prong One] because it does not recite a judicial exception.” (Example 39 (analysis)). In other words, the claim of Example 39 recites “additional elements,” but no “abstract ideas,” and the evaluation ends.
As explained in the 2024 SME Guidance, “[t]here is no need to move to Step 2A, Prong Two if the claim does not recite a judicial exception in the first instance.” (2024 SME Guidance, 89 Fed. Reg. 137 at p. 58134 (17 July 2024)).
Applicant’s claims are not limited to additional elements, such as the claim of Example 39. Instead, Applicant’s claims recite abstract ideas (as well as “additional elements”) including activities of “process,” “recognize,” “generate,” “identify,” and “produce,” that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are mental processes, (MPEP § 2106.04(a)(2) sub III), as set out above in detail.
Accordingly, the Applicant’s instant claims have a fact pattern that differs from that of Example 39, and Applicant’s arguments are unpersuasive.
8. “Additionally, Applicant asserts that the claims do not recite generic computer components as alleged page 3 of the Office Action.
For example, Applicant asserts that at least the elements such as
‘a spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence,’
‘the spatiotemporal action recognition module having a first attention mechanism over time frames and a second attention mechanism over human key-points,’
‘the intensity index being produced based at least in part on inputting the plurality of attention weights to a first fuzzifier and inputting an initial intensity score into a second fuzzifier’
of claim 1 are not related to generic computer components because not of computer-related systems in the field of computer vision use these components.” (Response at p. 16).
Examiner’s Response:
Examiner respectfully disagrees. The generic components of Applicant’s instant claims is the “[u]se of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to “process,” “recognize,” “generate,” “identify,” and/or “produce”) or simply adding a general purpose computer or computer components after the fact to an abstract idea . . .does not . . . provide significantly more.” (MPEP § 2106.05(f)(2)).
With regard to the term “module,” these components are not defined by the Applicant’s disclosure. However, the plain meaning “module” is self-contained unit with distinct functionality, which under a broadest reasonable interpretation covers execution of a function with a generic computer component including a computer and/or a processor and a memory, which is not inconsistent with the Applicant’s disclosure. Figure 11 recites a “block diagram of the machine system 100 in accordance with a representative embodiment:”
PNG
media_image1.png
670
574
media_image1.png
Greyscale
(see Specification ¶ 0082 & Fig. 11). Further, Applicant’s disclosure submits that “known” components are used in the implementation of the abstract idea. (see, e.g., Specification ¶ 0035 (“known model can be used that is known to achieve state-of-the-art results”); Specification ¶ 0037 (“known LSTM models”); Specification ¶ 0040 (“known fuzzy entropy methods”)). That is, the claims do not require specialized computing components.
Accordingly, the instant claims are not rendered patent eligible by stating an abstract idea and instructing “apply it on a computer.” (MPEP § 2106.05(f)),
Claim Rejections – 35 U.S.C. § 103
9. “Applicant respectfully submits that the above-cited references [of Carreira in view of Zhang], either alone or in combination, fail to show or suggest at least the following element of claim 1:
1. A machine learning system for recognizing actions performed by a subject and estimating an intensity of the recognized actions, the machine learning system comprising:
a processor configured to perform an integrated model comprising:
a spatio-temporal action recognition module configured to process key-point coordinates over time obtained from a plurality of video frames input to the machine learning system to recognize an action taken by a subject, the spatio-temporal action recognition module being configured to generate a plurality of attention weights, the spatio-temporal action recognition module comprising a spatio-temporal Long Short-Term Memory (LSTM) model that has been trained using a dataset to recognize a plurality of user actions of a plurality of different user action intensities in a respective video sequence, the video frames input being obtained from a video source connected via a video connection or from a memory device, the spatio-temporal action recognition module having a first attention mechanism over time frames and a second attention-mechanism over human key-points that are configured to identify an engagement of a human key-point coordinate in a respective time frame for the recognized action, the human key-point coordinate being associated with at least one of the plurality of attention weights; and
a fuzzy intensity index calculation module configured to receive the plurality of attention weights output by the spatio-temporal action recognition module to produce an intensity index associated with the recognized action, the intensity index being produced based at least in part on inputting the plurality of attention weights to a first fuzzifier and inputting an initial intensity score into a second fuzzifier, and
a memory device in communication with the processor.
(Response at p. 18 (emphasis added by Applicant)). “Applicant asserts that Carreira fails to show or suggest that ‘the spatio-temporal action recognition module having a first attention mechanism over time frames and a second attention mechanism over human keypoints that are configured to identify an engagement of a human key-point coordinate in a respective time frame for the recognized action, the human key-point coordinate being associated with at least one of the plurality of attention weights,’ as recited in claim 1. In fact, Carreira is silent with regard to ‘a first attention mechanism over time frames and a second attention mechanism over human key-points,’ as recited in claim 1. Therefore, Carreira does not disclose or suggest at least these elements of claim 1.” (Response at p. 19).
“Zhang is silent ‘a first fuzzifier’ and ‘a second fuzzifier,’ much less of ‘the intensity index being produced based at least in part on inputting the plurality of attention weights to a first fuzzifier and inputting an initial intensity score into a second fuzzifier,’ as recited in claim 1. Therefore, Zhang does not disclose or suggest at least these elements of claim 1.” (Response at pp. 19-20).
Examiner’s Response:
In view of the support provided by the Applicant’s disclosure, (see Specification ¶ 0025 (“The attention weights, along with the coordinate' s tensor, are then fed to the kinetic fuzzy intensity analysis module 3. The kinetic fuzzy intensity analysis module 4 computes an initial intensity score based on fuzzy entropy measures. The fuzzy inference module 5 converts the intensity score and the attention weights into fuzzy sets using an adaptive membership function”)), Examiner finds Applicant’s arguments persuasive, and WITHDRAWS the rejection under Section 103.
Conclusion
10. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
11. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
(US Published Application 20180129873 to Alghazzawi et al.) teaches a method comprising the steps of frame-by-frame, extracting behaviour features from video data associated with a scene, providing the behaviour features to an input of a recognition module comprising an interval Type 2 Fuzzy Logic (IT2FLS) based recognition model and classifying candidate object behaviour for a plurality of candidate objects in a current frame by selecting a candidate behaviour model having a highest output degree for each candidate object.
(Han et al., “Global Spatio-Temporal Attention for Action Recognition Based on 3D Human Skeleton Data,” IEEE (Mar 2020)) teaches that, by integrating the proposed global spatial attention (for spatial information) and accumulative learning curve (for temporal processing) models into the LSTM framework and taking the human skeletal joints as inputs, a global spatio-temporal action recognition framework (GL-LSTM) is constructed to recognize human actions. Diff is introduced as the preprocessing method to enhance the dynamic of the features, thus to get distinguishable features in deep learning.
(Thacker et al., “Analysis of Fuzzification Process in Fuzzy Expert System,” ScienceDirect (2018)) teaches The fuzzy expert systems are oriented towards handling uncertain or imprecise information. The fuzzy expert system is used in the domains where the input variables do not have fixed values. The success of fuzzy system depends upon the selection of appropriate membership function. The paper presents the analysis of fuzzification process of Fuzzy expert systems implemented in the domains of health care, education, career selection, real estate and finance. The parameters used for analyzing the systems are the input factors, type of membership function used for fuzzification, de-fuzzification of fuzzy sets generated. Based on analysis of the fuzzy expert system, the paper presents recommendations for selecting appropriate membership function.
(Skrlj et al., “Feature Importance Estimation with Self-Attention Networks,” arXiv (Feb 2020)) teaches feature importance estimates, assessed by the proposed Self-Attention Network (SAN) architecture, are compared with the established ReliefF, Mutual Information and Random Forest-based estimates, which are widely used in practice for model interpretation. An output of a feature ranking can be defined as a real-valued list whose j-th element is the estimated feature importance of the j-th feature. A typical approach for comparing such lists is to compute the Jaccard indices [17] between the sets of top-ranked features. However, this score takes into account feature importances only implicitly (via the order of the features) and is consequently unstable and often too pessimistic. Thus, we use its fuzzy version FUJI (the Fuzzy Jaccard Index).
12. Any inquiry concerning this communication or earlier communications from the Examiner should be directed to KEVIN L. SMITH whose telephone number is (571) 272-5964. Normally, the Examiner is available on Monday-Thursday 0730-1730.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, KAKALI CHAKI can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.L.S./
Examiner, Art Unit 2122
/BRIAN M SMITH/Primary Examiner, Art Unit 2122
1 Reference markers added to the claims for the limited purpose of the subject matter eligibility evaluation under Office guidance.