DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in response to Application as filed on September 3, 2024. Claims 1-20 are pending.
Claim Objections
Claims 1, 7, 8, 14, and 15 are objected to because of the following informalities:
Claim 1 recites “assign the at least one of the one or more motion classes to the body in motion,” with similar language in claims 8 and 15. It is unclear how a motion class is assigned to the body. It would appear that the motion class is assigned to the image data that is captured and not to the body. Appropriate correction is required.
Claim 7 recites “capture additional image data of a body in motion” and “one or more motion classes” with similar language in claim 14. It is unclear if the body in motion and the motion classes in claim 7 are the same body and motion classes as the in claim 1. It is suggested that this language be amended to read -- capture additional image data of the body in motion-- and -- the one or more motion classes-- for clarity.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a), as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, at the time the application was filed, had possession of the claimed invention.
The claims recite the following computer/processor-based function limitations:
` 1) determine, based on a first set of frames of the image data, one or more predictions for one or more motion classes (found in claims 1, 8, 15);
2) in response to receiving an additional frame of the image data, revise the one or more predictions for the one or more motion classes (found in 1, 8, 15);
3) in response to determining that the one or more predictions for at least one of the one or more motion classes satisfies a stability threshold (found in claims 1, 8, 15);
4) wherein the determination that the repetition has ended is based on a predicted duration of the one or more predictions that satisfies the stability threshold for the motion class (found in claims 4, 11, and 18);
5) predict, based on one or more body poses captured in the first set of frames, an initialization for the motion class (found in claims 5, 12, and 19);
6) wherein the one or more updated confidence values are determined in accordance with a bias toward the at least one of the one or more motion classes based on the confidence values for the at least one of the one or more motion classes satisfying the stability threshold (found in claims 7 and 14).
According to MPEP 2161.01 I., when examining computer-implemented claims, examiners should determine whether the specification discloses the computer and the algorithm (e.g., the necessary steps and/or flowcharts) that perform the claimed functions in sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor possessed the claimed subject matter at the time of filing. An algorithm is defined, for example, as "a finite sequence of steps for solving a logical or mathematical problem or performing a task." Microsoft Computer Dictionary (5th ed., 2002). Applicant may "express that algorithm in any understandable terms including as a mathematical formula, in prose, or as a flow chart, or in any other manner that provides sufficient structure." Finisar Corp. v. DirecTV Grp., Inc., 523 F.3d 1323, 1340, 86 USPQ2d 1609, 1623 (Fed. Cir. 2008) (internal citation omitted). It is not enough that one skilled in the art could write a program to achieve the claimed function because the specification must explain how the inventor intends to achieve the claimed function to satisfy the written description requirement. See, e.g., Vasudevan Software, Inc. v. MicroStrategy, Inc., 782 F.3d 671, 681-683, 114 USPQ2d 1349, 1356, 1357 (Fed. Cir. 2015) (reversing and remanding the district court’s grant of summary judgment of invalidity for lack of adequate written description where there were genuine issues of material fact regarding "whether the specification show[ed] possession by the inventor of how accessing disparate databases is achieved"). If the specification does not provide a disclosure of the computer and algorithm in sufficient detail to demonstrate to one of ordinary skill in the art that the inventor possessed the invention a rejection under 35 U.S.C. 112(a) for lack of written description must be made.
With regard to above-referenced the limitations 1-6, the specification appears to provide disclosure of a “network” to implement the functions, but lacks a complete description of what constitutes a “network” and also the algorithms (e.g., flow chart, mathematical model, code, prose) of how to implement these functions using the network. ¶10 of the specification describes “Generally, a network may be trained to ingest image data, determined body pose information, and based on body pose information, make the prediction as to an activity being performed. The network may be trained to predict the activity being performed based on body pose in a current frame, as well as prior frames. Prediction information may be generated by the network, for example on a frame-by-frame basis, for each of the set of user activities.” However, there are no examples or description of what kind constitutes a “network” that is contemplated or the steps of how the network trained. ¶11 continues “In some embodiments, the network may predict, based on the pose information, an initialization and duration of the corresponding activity. That is, while the activity is in progress, a prediction can be made as to the end of the activity, thereby allowing prediction to be made in real time without having image data of the full activity, which would normally be available in an offline mode but is not available when performing the predictions in real time.” ¶¶15-17 describe generally, a network may be trained to ingest image data, determined body pose information, and based on body pose information, make the prediction as to an activity being performed. The network may be trained to predict the activity being performed based on body pose in a current frame, as well as prior frames. Prediction information may be generated by the network, for example on a frame-by-frame basis, for each of the set of user activities. As a prediction information stabilizes over time, at least one of the set of activities can be identified of the activity be performed in the image data.” However, the specification is devoid of any description of what the network is or the steps and/or operations (i.e., algorithm) of its training such that it can perform the claimed functions.
The specification states “According to one or more embodiments, a same or different network may be trained to predict future achievable results from a current pose. In some embodiments, the network may consider the current body pose, along with body poses from prior frames. The network may be trained for a predefined set of activities, such as exercises or other repeatable activities. Confidence values are determined for each of these activities based on the current pose and/or prior pose data. The confidence values indicate a likelihood of an outcome for each of the set of activities.” However, there is no description of the steps and/or operations are contemplated to “determine, based on a first set of frames of the image data, one or more predictions for one or more motion classes” or “in response to receiving an additional frame of the image data, revise the one or more predictions for the one or more motion classes” as recited in limitations 1) and 2).
With regard to limitation 3); ¶¶22, 27, 38, and 41 of the specification describe that “The stability threshold may be satisfied, for example, if a peak of the heatmap remains stable over a predefined time period, such as a number of frames” and “the stability threshold is met may be determined by comparing the predictions for different activities. For example, the stability threshold may include determining that a measured level of stability for a particular activity is sufficiently greater than a measured level of stability for the remaining activities.” In addition, these paragraphs indicate “The peak may stabilize if the coordinates associated with the peak stay the same or within a margin of error for some predetermined about of time or frames.” However, there is not description of what constitutes a “peak” and/or the steps or operations (algorithm) to determine if the peak is stable or how one determines the coordinates of a peak. In addition, there is no description of what constitutes or the operations used to “measure a level of stability.”
With regard to limitation 4), ¶¶34, 38, 41, and 44 describe “a set of confidence values are determined for a set of initiation and duration characteristics for each exercise.” However, there is no description of the steps or operations (algorithm) of how to determine the set of initiation and duration characteristics for each exercise or what even constitutes an initiation characteristic. Additionally, these paragraphs describe plotting “along the x-axis, a measure of the initiation of the motion class (e.g., the squat).” However, there is no description of what constitutes an initiation or the operations to measure an initiation. In addition, there is no description of the steps/operations to determine a predicted duration that satisfies the stability threshold for the motion class or how to determine a repletion has ended based on the predicted a duration that satisfies the stability threshold.
With regard to limitation 5, ¶¶18, 21, and 38 of the specification describe a graph in which the x-axis indicates a frame or time at which the activity is predicted to be initiated” and “a set of confidence values are determined for a set of initiation and duration characteristics for each exercise.” However, there is no description of the steps or operations (algorithm) of how to predict, based on one or more body poses captured in the first set of frames, an initialization for the motion class.
With regard to limitation 6), ¶¶22, 31, and 47 of the specification describe “in some embodiments, the network is trained to show bias toward multiple repetitions of a motion class. As such, upon the conclusion of a motion, the peak prediction 430D for a next motion may quickly begin showing confidence values 425D for the squat prediction 420D.” However, there is no description of the steps or operations (algorithm) of how a network is trained to show bias or how to determine updated confidence values in accordance with a bias toward the motion class.
Therefore, Applicant has not described what the steps/operations (algorithms) for limitations 1-6 are to achieve the claimed functions recited in Applicant’s claims. Therefore, one skilled in the art is left to “guess” at what is meant by the limited description provided and what constitutes Applicant’s invention.
The specification must explicitly disclose the algorithms for performing the claimed functions, and simply reciting the claimed function in the specification will not be a sufficient disclosure for an algorithm which, by definition, must contain a sequence of steps. Blackboard, Inc. v. Desire2Learn, Inc., 574 F.3d 1371, 1383-85, 91 USPQ2d 1481, 1491-93 (Fed. Cir. 2009); Net MoneyIN, Inc. v. VeriSign, Inc., 545 F.3d 1359, 1366-67, 88 USPQ2d 1751, 1756-57 (Fed. Cir. 2008); Ex parte Rodriguez, 92 USPQ2d 1395, 1405-06 (Bd. Pat. App. & Inter. 2009). Here the specification provides no such description, therefore, the specification lacks adequate written description for the claimed functions of the computing device. As a result, claims 1, 3, 5, 7, 8, 10, 12, 14, 15, 17, and 19 contain subject matter which lacks adequate written description, and for at least these reasons, claims 1, 3, 5, 7, 8, 10, 12, 14, 15, 17, and 19 are found to fail the written description requirement. Claims 2-7, 9-14, and 16-20 depend from a rejected base claim, and therefore also lack adequate written description.
The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 4, 5, 7, 11, 12, 14, 18, and 19 are rejected under 35 U.S.C. § 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention.
In re claims 4, 11, and 18, the claims recite the limitations “the stability threshold for the motion class” at line 3. There is a lack of clear antecedent basis for these limitations in the claims. While claim 1 recites a stability threshold it is not recited as pertaining to a motion class. In addition, it is unclear which motion class is “the” motion claims recited in claims 4, 11, and 18. For purposes of examination this term is interpreted as reciting a stability threshold of the at least one of the one or more motion classes.
In re claims 5, 12, and 19, the language “predict, based on one or more body poses captured in the first set of frames, an initialization for the motion class” is indefinite. It is not clear what is meant by “an initialization.” The term “initialization” is not defined by the claim and the specification does not provide an objective standard for ascertaining the meaning of the term such that one of ordinary skill in the art would be reasonably apprised of the scope of the claim. For purposes of examination this term is interpreted as reciting the start of the motion of the body associated with the at least one of the one or more motion classes. In addition, the recitation “the confidence values for the one or more classes comprises,” lacks antecedent basis.
In re claims 7 and 14, the claims recite “the confidence values for the at least one of the one or more motion classes satisfying the stability threshold,” which lack clear antecedent basis. Claim 2 recites “a confidence value for each of a set of potential durations” but does not recite confidence values for the at least one of the one or more motion classes satisfying the stability threshold. For purposes of examination this term is interpreted as reciting a bias toward the at least one of the one or more motion classes based on the predictions for the at least one of the one or more motion classes satisfying the stability threshold.
Claims 5, 12, and 19, depend from a rejected base claim, and therefore are rejected for at least the reasons provided for the base claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
A patent may be obtained for “any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof.” 35 U.S.C. § 101. The Supreme Court has held that this provision contains an important implicit exception: laws of nature, natural phenomena, and abstract ideas are not patentable. Alice Corp. Pty. Ltd. v. CLS Bank Int’l, 134 S. Ct. 2347, 2354 (2014); Gottschalk v. Benson, 409 U.S. 63, 67 (1972) (“Phenomena of nature, though just discovered, mental processes, and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work.”). Notwithstanding that a law of nature or an abstract idea, by itself, is not patentable, the application of these concepts may be deserving of patent protection. Mayo Collaborative Servs. v. Prometheus Labs., Inc., 132 S. Ct. 1289, 1293-94 (2012). In Mayo, the Court stated that “to transform an unpatentable law of nature into a patent eligible application of such a law, one must do more than simply state the law of nature while adding the words ‘apply it.” Mayo, 132 S. Ct. at 1294 (citation omitted).
In Alice, the Supreme Court reaffirmed the framework set forth previously in Mayo “for distinguishing patents that claim laws of nature, natural phenomena, and abstract ideas from those that claim patent-eligible applications of these concepts.” Alice, 134 S. Ct. at 2355. The first step in the analysis is to “determine whether the claims at issue are directed to one of those patent-ineligible concepts.” Id. If the claims are directed to a patent-ineligible concept, then the second step in the analysis is to consider the elements of the claims “individually and ‘as an ordered combination” to determine whether there are additional elements that “transform the nature of the claim’ into a patent-eligible application.” Id. (quoting Mayo, 132 S. Ct. at 1298, 1297). In other words, the second step is to “search for an ‘inventive concept’-i.e., an element or combination of elements that is ‘sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the [ineligible concept] itself.” Id. (brackets in original) (quoting Mayo, 132 S. Ct. at 1294). The prohibition against patenting an abstract idea “cannot be circumvented by attempting to limit the use of the formula to a particular technological environment or adding insignificant post-solution activity.” Bilski v. Kappos, 561 U.S. 593, 610-11 (2010) (citation and internal quotation marks omitted). The Court in Alice noted that “[s]imply appending conventional steps, specified at a high level of generality,’ was not ‘enough’ [in Mayo] to supply an ‘inventive concept.” Alice, 134 S. Ct. at 2357 (quoting Mayo, 132 S. Ct. at 1300, 1297, 1294).
Examiners must perform a Two-Part Analysis for Judicial Exceptions. In Step 1, it must be determined whether the claimed invention is directed to a process, machine, manufacture or composition of matter.
Claims 1-20 are directed to method, system, and non-transitory computer readable medium. As such, the claimed invention falls into the broad categories of invention. However, even claims that fall within one of the four subject matter categories may nevertheless be ineligible if they encompass laws of nature, physical phenomena, or abstract ideas. See Diamond v. Chakrabarty, 447 U.S. at 309.
In Step 2A, it must be determined whether the claimed invention is ‘directed to’ a judicially recognized exception. According to the specification, “This disclosure is directed to systems, methods, and computer readable media for exercise tracking and prediction.” (¶9).
Independent claim 1 recites the following (with emphasis):
A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:
capture image data of a body in motion;
determine, based on a first set of frames of the image data, one or more predictions for one or more motion classes;
in response to receiving an additional frame of the image data, revise the one or more predictions for the one or more motion classes; and
in response to determining that the one or more predictions for at least one of the one or more motion classes satisfies a stability threshold, assign the at least one of the one or more motion classes to the body in motion.
The underlined portions of claim 1 generally encompass the abstract idea, with substantially identical features in claims 8 and 15. Claims 2-7, 9-14, and 16-20 further define the abstract idea such as by defining the types of predictions made or how the predictions are made. Under prong 2, the claimed invention encompasses an abstract idea in the form of organizing human activity and/or mental processes. The claims recite a determining how many repetitions of an exercise a user makes, for example, during a training session. This is an abstract concept of organizing human activity because it is drawn to a method of managing personal behavior during the training sessions. Furthermore, the method can be performed in the mind of a human and/or with the aid of pencil and paper.
The tracking of user behavior during exercise including some form of counting or grading activity is basic to the physical activity training process. The CRM, method, and system in the instant application simply seek to automate this well-known activity using generic computers recited at a high level of generality, and, therefore, the claims are directed to the abstract concept sub-grouping of "managing personal behavior or relationships or interactions between people", for example, a trainer observing the physical activity of a client, including tracking the user’s activity.
In addition, the claims also recite a mental process (i.e., observations, evaluations, judgments, and opinions). But for the recitation of computer readable medium storing instructions executed on one more computing systems and “computer implemented,” nothing in the claimed method or operations precludes the recitations from practically being performed in the mind. For example, “capture image data of a body in motion” can be performed by a trainer watching video of a client exercising; “determine, based on a first set of frames of the image data, one or more predictions for one or more motion classes” may be performed by a trainer observing the video and judging “I think the client is performing a squat”; “in response to receiving an additional frame of the image data, revise the one or more predictions for the one or more motion classes” may be performed by a trainer observing more of the video and judging “I now think the client is performing a burpee”; and in response to determining that the one or more predictions for at least one of the one or more motion classes satisfies a stability threshold, assign the at least one of the one or more motion classes to the body in motion” may be performed by a trainer observing the video and further judging “Indeed, the client is definitely performing a burpee.” If a claim, under its broadest reasonable interpretation, covers performance of recitations in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas.
Therefore, under prong 2, the claimed invention encompasses an abstract idea in the form of mental processes and/or certain methods of organizing human activity.
Under prong 2, the instant claims do not integrate the abstract idea into a practical application. In other words, the claims do not (1) improve the functioning of a computer or other technology, (2) effect a particular treatment or prophylaxis for a disease or medical condition (3) are not applied with any particular machine, (4) do not effect a transformation of a particular article to a different state, and (5) are not applied in any meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim, as a whole, is more than a drafting effort designed to monopolize the exception, the claims are directed to the judicially recognized exception of an abstract idea. See MPEP §§ 2106.05(a)-(c), (e)-(h).
While certain physical elements (i.e., elements that are not an abstract idea) are present in the claims, such features do not affect an improvement in any technology or technical field and are recited in generic (i.e., not particular) ways. Similarly, the abstract idea does not improve the functioning of these physical elements. In recent cases, the CAFC has made it clear that the term “practical application” means providing a technical solution to a technical problem in computers or networks per se. To be patent-eligible, the claimed invention must improve the computer as a computer or network as a network. Applicant’s invention does not meet these requirements. Applicant’s invention uses computers to process data and evaluate a user activity. This does not improve the computer qua computer. Instead, Applicant’s invention uses generic computers and networks as a tool to implement the abstract idea. As such, the claims are not eligible under Section 101.
Step 2B requires that if the claim encompasses a judicially recognized exception, it must be determined whether the claimed invention recites additional elements that amount to significantly more than the judicial exception. The additional elements or combination of elements other than the abstract idea per se amounts to no more than: a system having a processor and memory configured to perform the abstract idea.
The specification with regard to these elements admits, that “Electronic device 500 may be part of a multifunctional device, such as a mobile phone, tablet computer, personal digital assistant, portable music/video player, wearable device, or any other electronic device that includes a camera system” (spec. ¶49). Additionally, the specification indicates with regard to device components “ Electronic Device 500 may include one or more processors 530, such as a central processing unit (CPU);” “Electronic Device 500 may also include a memory 540. Memory 540 may include one or more different types of memory, which may be used for performing device functions in conjunction with processor 530. For example, memory 540 may include cache, ROM, and/or RAM. Memory 540 may store various programming modules during execution, including applications module 565, body tracking module 570, and motion estimation module 575.” (Spec. ¶50)
Therefore, the specification describes the electronic devices and components thereof in generic and functional terms, which illustrates that these are merely off-the-shelf computer components arranged in conventional ways. As a result, nothing in Applicant’s specification indicates the computer system performs anything other than well understood, routine, and conventional functions, such as receiving, storing, processing, and outputting. See, Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350, 1355 (ed. Cir. 2016) (“Nothing in the claims, understood in light of the [S]pecification, requires anything other than off-the-shelf, conventional computer, network, and display technology for gathering, sending, and presenting the desired information.”); see also Alice, 573 US. at 224—26 (receiving, storing, sending information over networks insufficient to add an inventive concept); buySAFE, Inc. v. Google, Inc., 765 F.3d 1340, 1355 (ed. Cir, 2014) (That a computer receives and sends the information over a network-—with no further specification—is not even arguably inventive.”). At best, Applicant’s claimed subject matter simply uses generic processing circuitry to perform the abstract idea of converting input data from one form to output another (e.g., images/frames to predictions and/or a repetition count). As noted above, the use of a generic computer system does not alone transform an otherwise abstract idea into patent-eligible subject matter. As our reviewing court has observed, “after Alice, there can remain no doubt: recitation of generic computer limitations does not make an otherwise ineligible claim patent-eligible.” DDR Holdings, 773 F.3d at 1256 (citing Alice, 573 U.S. at 223).
Taking the claimed elements individually yields no difference from taking them in combination because each element simply performs its respective function as discussed above. The claims do not purport to improve the functioning of a computer itself, nor do they effect an improvement in any other technology or technical field. Instead, the additional features merely amount to an instruction to apply the abstract idea using generic, functional, and conventional components well-known in the art. Viewed as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Therefore, claims 1-20 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter. See Alice Corporation Pty. Ltd. v. CLS Bank International, et al., 573 U.S. 208 (2014).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1-5, 8-12, and 15-19 are rejected under 35 U.S.C. § 103 as being unpatentable over U.S. Publication No. 2022/0138966 by Sung et al. (“Sung”) in view of US Publication No. 2024/0042281 by Kashyap et al. (“Kashyap”).
In re claims 1, 8, and 15, Sung discloses a non-transitory computer readable medium, a method, and a system including one or more processors and a computer readable code executable by one or more processors [¶¶10, 17, among others, describe methods and systems for analyzing and classifying a number of repetitive motions in a video including a non-transitory computer-readable medium storing computer-executable instructions which, when executed by a processor, cause the processor to perform operations for determining a number of repetitive motions in a video]: capture image data of a body in motion [¶¶16, 83, 84, 141, 142, among others, describe video can be captured by a camera on a mobile device including features of a body of a moving human]; determine, based on a first set of frames of the image data, one or more predictions for one or more motion classes [¶66-74, 121, 141, 142, among others, describe machine learning to determine poses input from images and confidence maps of body parts to identify the motions including generating classifications associated with motions such as high knee, jogging in place, jumping jack, jump rope, lunge, squat, squat jump, combinations thereof, and/or the like, based on patterns associated with the principle components of landmarks associated with a given motion]; in response to receiving an additional frame of the image data, revise the one or more predictions for the one or more motion classes [¶¶51, 66-74, 142, among others, describe classifying in incremental fashion on subsequent frames of a video operations for each updated video frame added to the video of the activity;]; and in response to determining that the one or more predictions for at least one of the one or more motion classes, assign the at least one of the one or more motion classes to the body in motion [¶¶78-82,121-128,142, among others, describe detection, classification, and counting of the different movements and the total number of movements using a neural network].
Sung discloses determining points identifying the landmark's motion and a duration parameter associated with the duration of the at least one repetitive motion, and inputting the points and the duration parameter to a machine learning algorithm to obtain a classification result of the repetitive motion, the classification result identifying the repetitive motion into a predetermined type, where the machine learning algorithm can include a multi-layer neural network. Sung does not explicitly teach determining the predictions satisfy a stability threshold. However, Kashyap teaches classification system to detect and/or classify poses, movements, or exercises within images captured by the system including repetitions using a classification system to provide a prediction that the user is performing a certain exercise with a given threshold confidence or accuracy over time using a pose heatmap for a set of available poses to be classified (e.g., the set of all available or possible poses) to map channel-wise peaks, see, e.g., ¶¶82-100, 118, 196.
Sung and Kashyap are both considered to be analogous to the claimed invention because they are in the same field of movement/exercise classification. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Sung to include a determination that the motion predictions satisfy a stability threshold, as taught by Kashyap, in order to improve or enhance the accuracy of the inferences output by the different frameworks, see, e.g., ¶95.
In re claims 2, 9, and 16, Sung discloses the one or more predictions comprises, for each motion class, a confidence value for each of a set of potential durations for a corresponding motion class [¶¶ 15, 21, 66-75, 81, 141, 142, among others, describe a duration parameter associated with the duration of the at least one repetitive motion; and inputting the points and the duration parameter to a machine learning algorithm to obtain a classification result of the repetitive motion, the classification result identifying the repetitive motion into a predetermined type].
In re claims 3, 10, and 17, Sung discloses wherein the motion is a repeated motion [¶¶10-15 describe analyzing and classifying a number of repetitive motions in a video] and determining that a repetition of the at least one of the one or more motion classes has ended [¶¶66-74, 141, 142, among others describe segmenting the video images into repetitions, each having a beginning and end marked by zero crossings]; and in response to determining that the repetition has ended, modify a repetition count for the at least one of the one or more motion classes [¶¶66-74, among others describe increasing the count for each repetition].
In re claims 4, 11, and 18, Sung discloses the determination that the repetition has ended is based on a predicted duration of the one or more predictions class [¶¶66-74, 141, 142, among others describe durations determining points identifying the landmark's motion based on a duration parameter associated with the duration of the at least one repetitive motion, and inputting the points and the duration parameter to a machine learning algorithm where the duration is a periodic. predetermined duration (e.g., 0.1 seconds, 5 seconds, 1 minute, 5 minutes, etc.) to within a threshold error tolerance (e.g., about 1%, about 5%].
Sung discloses determining points identifying the landmark's motion and a duration parameter associated with the duration of the at least one repetitive motion, and inputting the points and the duration parameter to a machine learning algorithm to obtain a classification result of the repetitive motion, the classification result identifying the repetitive motion into a predetermined type, where the machine learning algorithm can include a multi-layer neural network. Sung does not explicitly teach determining the predictions satisfy a stability threshold. However, Kashyap teaches classification system to detect and/or classify poses, movements, or exercises within images captured by the system including repetitions using a classification system to provide a prediction that the user is performing a certain exercise with a given threshold confidence or accuracy over time using a pose heatmap for a set of available poses to be classified (e.g., the set of all available or possible poses) to map channel-wise peaks [¶¶82-100, 118, 196].
Sung and Kashyap are both considered to be analogous to the claimed invention because they are in the same field of movement/exercise classification. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Sung to include a determination that the motion predictions satisfy a stability threshold, as taught by Kashyap, in order to improve or enhance the accuracy of the inferences output by the different frameworks, see, e.g., ¶95.
In re claims 5, 12, and 19, Sung discloses predicting, based on one or more body poses captured in the first set of frames, an initialization for the motion class, wherein the determination that the repetition has ended is further based on the predicted initialization for the motion class [¶¶66-81, 141, 142, among others, describe determining durations associated with different repetitive motion including determining zero crossing which correspond to the start of the motion].
Claims 6, 13, and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over Sung in view of Kashyap and further in view of US Publication No. 2026/0097264 by Erickson et al. (“Erickson”).
In re claims 6, 13, and 20, Sung does not explicitly discloses in response to determining that the repetition has ended resetting the confidence values for each of the one or more motion classes. However, Erickson teaches resetting the confidence values If the count reaches a previously defined threshold (e.g., a function of time), the corresponding accumulator pixels and the motion history image are reset to zero, as the motion is not recent, see, e.g., ¶159.
Sung and Erickson are both considered to be analogous to the claimed invention because they are in the same field of movement/exercise count classification. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Sung to include resetting the confidence values, as taught by Erickson, in order to improve or enhance the accuracy of the inferences output by the different frameworks, see, e.g., ¶102.
Claims 7 and 14 are rejected under 35 U.S.C. § 103 as being unpatentable over Sung in view of Kashyap and Erikson and further in view of US Publication No. 2017/0004285 by Anderton et al. (“Anderton”).
In re claims 7 and 14, Sung discloses capturing additional image data of a body in motion comprising a second set of frames captured subsequent to the first set of frames [¶¶51, 66-82]; and determining, based on the second set of frames of the image data, one or more updated confidence values for one or more motion classes [¶¶78-82, 121-128,141,142]. However, Sung does not explicitly teach wherein the one or more updated confidence values are determined in accordance with a bias toward the at least one of the one or more motion classes based on the confidence values for the at least one of the one or more motion classes. However, Anderton teaches updated confidence values are determined in accordance with a bias toward the at least one of the one or more motion classes based on the confidence values for the at least one of the one or more motion classes [¶46, among others, describes processing the sensor data to determine the repetitions may include Neural Networks (such as artificial intelligence systems) which involves “training” the system to recognize various signal configurations that represent typical signals for various exercise activities, and it subsequently storing various “decoding biases” that enable the system to recognize future repetitions that conform to the required pattern (and hence score a high probability via the weights or biases)].
Sung and Anderton are both considered to be analogous to the claimed invention because they are in the same field of movement/exercise count classification. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Sung to include updated confidence values determined in accordance with a bias toward a motion class, as taught by Anderton, in order to improve or enhance the accuracy of the motion detection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and is listed on the attached Notice of References Cited.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Andrew Bodendorf whose telephone number is (571) 272-6152. The examiner can normally be reached M-F 9AM-5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xuan Thai can be reached on (571) 272-7147. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW BODENDORF/Examiner, Art Unit 3715
/XUAN M THAI/Supervisory Patent Examiner, Art Unit 3715
Prosecution Insights