DETAILED ACTION
Remarks
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Applicant is advised that should claims 8 and 9 be found allowable, claims 8 and 9 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Under the broadest reasonable interpretation, the terms of the claim are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP 2111.
STEP 1 = YES: The claimed invention is to a process and product, and thus fall under one of the four statutory categories (Step 1: YES).
STEP 2A, Prong 1 = YES: The claim(s) recite(s) a series of steps which can be practically performed by one or more humans through mental process (i.e., observation, evaluation, judgement, and/or opinion)(see MPEP § 2106.04(a)(2), subsection III) and/or certain methods of organizing human activity (i.e., managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) (see MPEP § 2106.04(a)(2), subsection II), as indicated for each limitation below. Moreover, the claims recite steps akin to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, which the court in Electric Power Group held to recite a mental process. Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016). This includes:
1. A…method, comprising:
predicting a first action step and a last action step based on an initial visual observation and a goal visual state (mental process, observation and evaluation);
retrieving multiple procedural plans from a procedural knowledge graph (PKG), trained using a set of training instructional videos, which start with the first action step and end with the last action step (mental process, observation and evaluation);
generating a procedure plan using the multiple procedural plans (mental process, evaluation); and
generating an instructional video based on the procedure plan (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, teaching; see TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner.)).
2. The method of claim 1, wherein the PKG includes action steps represented as nodes; and edges representing transitions between action steps (see claim 1).
3. The method of claim 2, wherein the edges include transition probabilities between action steps (see claim 1).
4. The method of claim 1, wherein predicting the first action step and the last action step includes using …the set of training instructional videos (mental process, evaluation).
5. The method of claim 1, further comprising: prompting…with the first action step and last action step to generate additional procedural plans; and incorporating the additional procedural plans … in generating the procedure plan (mental process, evaluation).
6. The method of claim 1, wherein generating the procedure plan includes …takes as input the initial visual observation, the goal visual state, and the multiple procedural plans (mental process, evaluation).
7. The method of claim 6, further comprising training … using annotated action steps … (mental process, evaluation).
8. The method of claim 1, wherein the instructional video includes a sequence of action steps for transitioning from the initial visual observation to the goal visual state (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, teaching).
9. The method of claim 1, wherein the instructional video includes a sequence of action steps for transitioning from the initial visual observation to the goal visual state (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, teaching).
10. The method of claim 1, wherein generating the procedure plan includes …to optimize a sequence of action steps (mental process, evaluation).
11. The method of claim 1, wherein the instructional video is generated for a medical procedure in a healthcare setting (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, teaching).
12. A system, comprising: …
access a procedural knowledge graph (PKG) constructed using a set of training instructional videos (mental process, evaluation);
predict a first action step and a last action step based on an initial visual observation and a goal visual state (mental process, observation and evaluation);
retrieve multiple procedural plans from the PKG that start with the first action step and end with the last action step (mental process, evaluation);
generate a procedure plan using the multiple procedural plans retrieved from the PKG (mental process, evaluation); and
generate an instructional video based on the procedure plan (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, teaching; see TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner.)).
13. The system of claim 12, wherein the PKG includes: nodes representing action steps extracted from training instructional videos; and edges connecting the nodes, the edges representing transitions between action steps and including transition probabilities between action steps (see claim 12).
14. The system of claim 12, …predict the first action step and the last action step … using a set of training instructional videos (mental process, evaluation).
15. The system of claim 12, …prompt … with the first action step and last action step to generate additional procedural plans; and incorporate the additional procedural plans … in generating the procedure plan (mental process, evaluation).
16. The system of claim 12, further comprising: a procedure planning…that takes as input the initial visual observation, the goal visual state, and the multiple procedural plans (mental process, evaluation).
18. The system of claim 12, wherein the instructional video includes a sequence of action steps for transitioning from the initial visual observation to the goal visual state (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, teaching).
19. The system of claim 12, wherein the procedure plan is generated…to optimize a sequence of action steps (mental process, evaluation).
20. The system of claim 12, wherein the instructional video is generated for a medical procedure in a healthcare setting (mental process, evaluation).
The steps identified above are akin to mental processes and/or certain methods of organizing human activity, and thus fall within an enumerated category of abstract ideas. Note that even if most humans would use a physical aid (e.g., pen and paper, video recording device) to help them complete the recited steps above, the use of such physical aid does not negate the mental nature of these limitations. Therefore, the claims recite an abstract idea (Step 2A, Prong 1: YES).
STEP 2A, Prong 2 = NO: This judicial exception is not integrated into a practical application.
To the extent the claims recite additional elements related to defining a computer environment using trained models/LLMs/AI to implement the abstract idea above (i.e., Generally defining the claimed system and method as computer-implemented or performed by a memory storing instructions; and a processor configured to execute the instructions; In order to perform the judicial exception, using a step recognition model trained on the set of training instructional videos; prompting a large language model (LLM) with the first action step and last action step; using a procedure planning model that takes as input the initial visual observation, the goal visual state, and the multiple procedural plans; training the procedure planning model using annotated action steps from the training of instructional videos as supervision signals; using artificial intelligence techniques to optimize a sequence of action steps; using a step recognition model trained on a set of training instructional videos), they are recited at a high level of generality such that they do not amount to a particular machine or technical improvement thereof, nor do they represent an improvement in any other technology. The recited models, LLM, and artificial intelligence are referred to by name alone, and as being trained to perform certain tasks, without any technical detail defining how they achieve these results, e.g., define how the model/LLM/AI operate or are trained by adjusting values, that results in an improvement. The specification does not identify any improvements as to how the models, LLM, or artificial intelligence itself operates. To the extent the specification does disclose a technical improvement of any kind, which Examiner does not concede, the claims as a whole do not include limitations that reflect a technical improvement disclosed in the specification. Therefore, the claimed method and system do not recite an improvement to the computer itself, or the recited models, LLM, or AI. Further, merely defining the data input into a model/LLM/AI does not change how the model/LLM/AI fundamentally operates – the model/LLM/AI works the same way. The claims do not explain how the model/LLM/AI values or parameters are adjusted based on the claimed training either (the set of training instructional videos or the initial visual observation, the goal visual state, and the multiple procedural plans) to help the model/LLM/AI better understand and generate content relevant to the specific task. Thus, the claimed use of models/LLM/AI is akin to the facts of Recentive Analytics, Inc. v. Fox Corp. (applying generic machine learning to a new data environment without disclosing any improvement to the machine learning model itself), and thus do not integrate the judicial exception into a practical application. See Recentive Analytics, No. 2023-2437 (Fed. Cir. Apr. 18, 2025). Furthermore, to the extent the video generating steps are interpreted as inherently requiring an additional element to perform, the claims recite these steps at a high level of generality with no technical details how the instructional video is generated, and thus amounts to insignificant extra-solution activity and being used in their ordinary capacity (e.g., video recording device records video). In other words, to the extent the claims require a video recording device, it serves merely as a tool execute the abstract idea, and thus does not integrate the judicial exception. Thus, the generic manner which these additional elements are claimed amount to mere instructions to implement the abstract idea in a computer environment, i.e., field of use, and thus do not integrate the judicial exception into a practical application.
It should be noted that because the courts have made it clear that mere physicality or tangibility of an additional element or elements is not a relevant consideration in the eligibility analysis, the physical nature of the physical components identified above does not affect this analysis. See MPEP 2106.05(I) for more information on this point, including explanations from judicial decisions including Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 224-26 (2014). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception. Therefore, the claims are directed to an abstract idea (Step 2A, Prong 2: YES).
STEP 2B = NO: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because as provided under Prong 2, the additional elements are recited at a high level of generality, and for the purpose of insignificant extra-solution activity. Moreover, the specification of the instant application further demonstrates that the additional elements are recited for their well-understood, routine and conventional functionality, which refers to elements of the computer system in a manner that indicates that the additional elements are sufficiently well-known that the specification does not need to describe the particulars of such additional elements to satisfy 35 U.S.C. § 112(a)(e.g., see par. 0064: various other input devices and/or output devices can be included in processing system 300, depending upon the particular implementation of the same, as readily understood by one of ordinary skill in the art; par. 0068: Each computer program may be tangibly stored in a machine-readable storage media or device (e.g., program memory or magnetic disk) readable by a general … purpose programmable computer). Further, to the extent the claims require a video recording device, it serves merely as a tool execute the abstract idea, and thus does not amount to significantly more than the judicial exception based on the high level of generality for which the generating steps are recited in the claims. Thus, the additional elements defining the field of use as a computer-implemented environment to generate instructional video using generically defined models/LLM/AI amount to merely automating a manual process, which the courts have held to be insufficient in showing an improvement in computer-functionality. See Credit Acceptance Corp. v. Westlake Services, 859 F.3d 1044, 1055, 123 USPQ2d 1100, 1108-09 (Fed. Cir. 2017); see also LendingTree, LLC v. Zillow, Inc., 656 Fed. App'x 991, 996-97 (Fed. Cir. 2016) (non-precedential).
Therefore, the claims are not directed to significantly more than the abstract idea (Step 2B: NO).
Therefore, claims 1-20 are not directed to patent eligible subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure, including:
US 2024/0161464 A1 to MARTIN (Abstract: Embodiments described herein provide systems and methods for training video models to perform a task from an input instructional video. A procedure knowledge graph (PKG) may be generated with nodes representing procedure steps, and edges representing relationships between the steps. The PKG may be generated based on text and/or video training data which includes procedures (e.g., instructional videos). Using the PKG, a video model may be trained using the PKG to provide supervisory training signals for a number of tasks. Once the model is trained, it may be fine-tuned for a specific task which benefits from the model being trained in a way that makes the model embed procedural information when encoding videos).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James Hull whose telephone number is 571-272-0996. The examiner can normally be reached on Monday-Friday from 8:00am to 5:00pm MST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xuan Thai, can be reached at telephone number 571-272-7147. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/JAMES B HULL/Primary Examiner, Art Unit 3715
Prosecution Insights