Prosecution Insights
Last updated: July 17, 2026
Application No. 18/180,721

DETECTING ACTIONS IN VIDEO USING MACHINE LEARNING AND BASED ON BIDIRECTIONAL FEEDBACK BETWEEN PREDICTED TYPE AND PREDICTED EXTENT

Final Rejection §101§103
Filed
Mar 08, 2023
Examiner
POTTS, RYAN PATRICK
Art Unit
2672
Tech Center
2600 — Communications
Assignee
International Business Machines Corporation
OA Round
2 (Final)
79%
Grant Probability
Favorable
3-4
OA Rounds
0m
Est. Remaining
99%
With Interview

Examiner Intelligence

Grants 79% — above average
79%
Career Allowance Rate
196 granted / 247 resolved
+17.4% vs TC avg
Strong +39% interview lift
Without
With
+39.3%
Interview Lift
resolved cases with interview
Typical timeline
2y 11m
Avg Prosecution
19 currently pending
Career history
271
Total Applications
across all art units

Statute-Specific Performance

§101
1.2%
-38.8% vs TC avg
§103
77.1%
+37.1% vs TC avg
§102
6.7%
-33.3% vs TC avg
§112
14.2%
-25.8% vs TC avg
Black line = Tech Center average estimate • Based on career data from 247 resolved cases

Office Action

§101 §103
CTFR 18/180,721 CTFR 89031 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Response to Arguments Applicant’s arguments, see Remarks at page 18, filed 19 March 2026, with respect to the objection to the drawings have been fully considered and are persuasive. The objection has been withdrawn. 07-38-01 AIA Applicant’s arguments, see Remarks at page 18 , filed 19 March 2026 , with respect to the objection to the specification have been fully considered and are persuasive. The objection has been withdrawn. Applicant’s arguments, see Remarks at pages 18-19, filed 19 March 2026, with respect to the rejection of claims 8-13 under 35 U.S.C. 101 have been fully considered but are not persuasive. Claims 8-13 are directed to a “computer program product” and the only potential source of physical structure is recited as “a computer-readable storage medium”. Paragraph 74 of the specification does not provide a limiting definition of “computer program product” that excludes transitory signals. Rather, the disclosure merely opines that “computer program product” is used to describe a set of storage media and tangible storage devices. The description omits any statement that essentially requires the scope of the term “computer program product” to only include physical or tangible storage. In other words, the scope could also include transitory signals. Regarding the term “computer readable storage medium”, paragraph 74 provides in relevant part: A computer program product embodiment ("CPP embodiment" or “CPP”) is a term used in the present disclosure to describe any set of one, or more, storage media (also called "mediums") collectively included in a set of one, or more, storage devices that collectively include machine readable code corresponding to instructions and/or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can retain and store instructions for use by a computer processor. ... A computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se , such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media. As will be understood by those of skill in the art, data is typically moved at some occasional points in time during normal operations of a storage device, such as during access, de-fragmentation or garbage collection, but this does not render the storage device as transitory because the data is not transitory while it is stored. (emphasis added). The first sentence essentially states that CPP describes some storage media that include tangible storage devices, which leaves open the possibility that CPP also describes transitory signals. The statement “A computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se” refers to the collective whole of the described embodiments, e.g., the “set” described in paragraph 74, not the interpretation from the point of view of any POSITA outside of the disclosure. Therefore, the scope of the term “computer readable storage medium” encompasses transitory signals. Accordingly, the rejection is maintained. Applicant’s arguments, see Remarks at page 19, filed 19 March 2026, with respect to the rejection of claims 2, 3, 9, 10, 15 and 16 under 35 U.S.C. 112(b) have been fully considered and are persuasive. The rejection has been withdrawn. Applicant’s arguments, see Remarks at page 19, filed 19 March 2026, with respect to the rejection under 35 U.S.C. 103 have been fully considered. Applicant argues the combination of Li with Cheng “would provide different operations and outputs, and would not achieve the features of the present disclosure” but provides no specific reasoning to support the assertion. Applicant argues Li does not teach elements (i) and (ii) of the “refinement” step. Examiner respectfully disagrees. On page 2686, Li explains that the crosstalk blocks (CT-Block) use the outputs of the classification and localization task-related blocks (TC-Block) as inputs, and outputs two feature maps for each of the classification and localization branches, which are the i th stage DCFA outputs. The TC-Block output is an initial localization predication and an initial classification prediction. The predictions are refined via crosstalk and equation (4). Li tests three different crosstalk directions. See Li Table IV and Figure 4. The three directions are the localization features influencing the classification features, the classification features influencing the localization features, and bidirectional influence. Thus, Li discloses refining the classification based on the initial localization (localization influence classification or bidirectional) and/or refining the localization based on the initial classification. Accordingly, the rejection is maintained. Claim Objections 07-29-01 AIA Claim 3 is objected to because of the following informalities: there should be an instance of “comprises” or the like between “refining the initial video clip” and “detecting” . Appropriate correction is required. 07-29-01 AIA Claim 11 is objected to because of the following informalities: “computer-implemented method” should be changed to “computer program product” to be consistent with the other claims that ultimately depend from claim 8 . Appropriate correction is required. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. The Official Gazette Notice 1351 OG 212 dated February 23, 2010, http://www.uspto.gov/web/offices/com/sol/og/2010/week08/TOC.htm#ref20 states, “The broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent.” A claim drawn to such a computer readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation "non-transitory" to the claim. Claims 8-13 recite a “computer program product” that comprises a “computer readable storage medium.” The specification, e.g. paragraph 74, at best provides exemplary embodiments of non-transitory computer-readable storage media, but does not provide a limiting definition that prohibits an interpretation of the recited “computer readable storage medium” from encompassing one or more transitory signals. See MPEP 2111.01. Therefore, claims 8-13 are rejected for being directed to non-statutory subject matter. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed inventions absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claims 1, 4-8, 11-14 and 17-20 are r ejected under 35 U.S.C. 103 as being unpatentable over T ALLFormer: Temporal Action Localization with a Long-memory Transformer to Cheng in view of Dense Crosstalk Feature Aggregation for Classification and Localization in Object Detection (published 1 November 2022) to Li. R egarding claim 1 , Cheng teaches a computer-implemented method of video processing for action detection using machine learning (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.), the computer-implemented method comprising: identifying an action depicted in a video (The Temporal Boundary Localization Module (TBLM) outputs a predicted localization (start and stop times) from a localization head (i.e., regression branch) and outputs a predicted classification from a classification head (i.e., classification branch). The classification head identifies each action in the video that it was trained. See Cheng at FIG. 2 on pg. 6, section 3.4, and FIG. 3 on pg. 20.), the video comprising one or more images (The videos include multiple sets of frames that correspond to different action types. See e.g., FIG. 1.); predicting an initial type of the action based on a classification module (The classification head identifies each action in the video that it was trained. See Cheng at FIG. 2 on pg. 6, section 3.4, and FIG. 3 on pg. 20.) of one or more machine learning models (The classification heads predicts a type of the action (e.g., eating, cooking, mixing). TALLFormer is a Transformer-based model designed for temporal action localization and the whole system is trained end-to-end, which includes the classification and localization heads. See Cheng at Abstract); and predicting, in the video, an initial video clip (action instance) depicting the action (a sampled video clip), including determining a starting point and an ending point of the initial video clip in the video (Cheng, pg. 5, section 3, “Given an untrimmed video V … with T RGB frames, our TALLFormer model aims to predict a set of action instances … where M is the number of action instances in V . Each action instance is a four-element tuple that represents the start timestamp of action, end timestamp of action, action class and probability of this instance respectively.”), wherein the initial video clip is predicted based on a localization module of the one or more machine learning models (The localization head (i.e., regression branch) outputs a predicted start time and stop time of the action classified by the classification head. The classification head (branch) outputs a label that identifies which type of action is happing (e.g., “Eating” or “Making a cake”) and the localization head (branch) outputs temporal boundaries — the start and stop times of the action segment in the untrimmed video. The boundaries correspond to the action recognized by the classification head. See Cheng at FIG. 1 on pg. 2, FIG. 2 on pg. 6, section 3.4, and FIG. 3 on pg. 20.); performing a refinement that includes (i) refining the initial type of the action or (ii) refining the initial video clip based on the type of the action (Each time the model’s parameters are updated (refined), the localization and classification predictions are updated (refined) as well. The type of classified action is based on the same shared feature representation used to make the localization determination. See Cheng at pg. 3, section 2, “Analogous to TAL, there is a surge of the usage of anchor-free methods in object detection. … We take inspiration from these methods to design a basic anchor-free localizer, along with making full use of the temporal insights of videos to propose novel refinement strategy and consistency learning.”; pg. 6, section 4.1, “Our model is trained for 16 epochs using Adam”. The classification and localization heads are bidirectionally coupled because they share the same backbone features and are trained jointly. As the model is trained, each head provides output during inference that modifies their shared representation during training (a bidirectional feedback mechanism), causing the model to learn features that jointly promote accurate action recognition and precise boundary detection.), wherein the refinement produces a refined type pf the action and a refined video clip and is performed by one or more computer processors (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.); and outputting an indication (e.g., a refine start time, a refined stopping time) of the refined type (i.e., where a re-classified type of action occurs) or of the refined video clip (The output of TALLFormer includes new localizations and classifications based on the parameter update from the prior instance of the model being trained or re-trained.), but does not teach that which is explicitly taught by Li. Li teaches performing a refinement that includes (i) refining the classification based on the localization (Li, pg. 2690, “Cls ← Reg”, “Cls ↔ Reg”) and (ii) refining the localization based on the classification (Li, pg. 2690, “Cls → Reg”, “Cls ↔ Reg”); and outputting an indication of the classification (Li, Fig. 1(a), “Classification”) and of the refined localization (Li, Fig. 1(a), “Classification”; pg. 2692, section IV.D, “DCFA achieves more complex and comprehensive regions of interest for feature aggregation by fusing two differentiated feature branches, and finally effectively improves the detection accuracy”; pg. 2694, “In terms of efficiency, our method has higher accuracy with similar inference speed and fewer parameters/FLOPs. This work is just the beginning, there are still some limitations that need further research, such as more effective cross-fusion to exploit or resolve the large performance difference between Cls → Reg and Cls ← Reg, more efficient crosstalk block with fewer parameters, extending two-branch crosstalk to multi-branch crosstalk for multi-task fusion such as classification-location-segmentation. We hope our work is a meaningful attempt to the multi-task fusion problem in object detection, and can also be directly used in practical applications to improve efficiency and accuracy.”). Cheng discloses an attention-based two-stage object detector that “effectively fuses” computed features (see pg. 5, section 3) with two branches in a detection head (registration (localization) branch and classification branch (see Cheng at pg. 20, FIG. 3) with a Transformer-based backbone for refinement of features in a Temporal Action Localization (TAL). Thus, Cheng shows that it was known in the art before the effective filing date of the claimed invention to use such architectures for action classification and localization in video (i.e., sequence of images), which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, generating accurate and precise classifications and localizations of classifier categories. Li investigates “a more comprehensive cross-fusion between classification and localization to resolve the misalignment problem between the two tasks” (pg. 2693, section V) and discloses a bidirectional attention mechanism for classification and localization in object detection (see FIG. 1). To verify “the generalization ability of DFCA”, Li applies DCFA to one-stage detectors, “to two-stage detectors and the latest transformer-based backbones”, which showed improvements in Average Precision (AP) precision (see Li at pg. 2689, section IV.C). While Li applies DCFA to image classification and localization, the improvement of resolving the misalignment problem is applicable to classification and localization in object detection generally. Thus, Li shows that it was known in the art before the effective filing date of the claimed invention to use a bidirectional attention mechanism in an object detector that performs the task of classification and the task of localization, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, generating accurate and precise classifications and localizations of classifier categories (types or labels). A person of ordinary skill in the art would have been motivated to modify the attention mechanism of TALLFormer disclosed by Cheng to be two attention modules that operate as the bidirectional attention mechanism disclosed by Li to thereby refine the classification and localization outputs using bidirectional feedback between the localization module and the classification module. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of improving efficiency, accuracy, and substantially mitigating the misalignment problem. Regarding claim 4 , Cheng in view of Li teaches the computer-implemented method of claim 1, wherein the initial type of the action is refined by an attention module ( Acls and Areg . See FIG. 1 of Li) of the one or more machine learning models, the attention module comprising a localization-to-classification attention module (Localization influencing classification in the bidirectional crosstalk. See Li, pg. 2694). The rationale for obviousness is the same as provided for claim 1. Regarding claim 5 , Cheng in view of Li teaches the computer-implemented method of claim 1, wherein the initial video clip is refined (localization refinement) by an enhancement module of the one or more machine learning models, the enhancement module comprising a classification-to-localization enhancement module (Classification influencing localization in the bidirectional crosstalk. See Li, pg. 2694). The rationale for obviousness is the same as provided for claim 1. Regarding claim 6 , Cheng in view of Li teaches the computer-implemented method of claim 1, wherein a bidirectional feedback mechanism is provided between the classification and localization modules (Li, Abstract, “we introduce bidirectional cross talk detection head in a systematic manner to provide a full deep cross-fusion between classification and localization.”; pg. 2687, section III.D, “adding cross-fusion features back to their respective branches”), and wherein the bidirectional feedback mechanism is provided to increase a measure of accuracy of one or more machine learning models, the measure of accuracy pertaining to at least one of classification and localization (Li, pg. 2692, section IV.D, “DCFA achieves more complex and comprehensive regions of interest for feature aggregation by fusing two differentiated feature branches, and finally effectively improves the detection accuracy”; pg. 2694, “In terms of efficiency, our method has higher accuracy with similar inference speed and fewer parameters/FLOPs.”). The rationale for obviousness is the same as provided for claim 1. Regarding claim 7 , Cheng in view of Li teaches wherein the computer-implemented method of claim 1, wherein outputting the indication of the refined type or of the refined video clip comprises outputting indications of the refined type (classification output after refinement. See Cheng at section 3.4) and of the refined video clip (localization output after refinement. See Cheng at section 3.4), respectively. Claims 8 and 11-13 substantially correspond to claims 1 and 4-6 by reciting a computer program product of video processing for action detection using machine learning, the computer program product comprising: a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising the steps of the methods of claims 1 and 4-6 (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.). The rationale for obviousness is the same as provided for claim 1. Claims 14 and 17-20 substantially correspond to claims 1 and 4-7 by reciting a system of video processing for action detection using machine learning, the system comprising: one or more computer processors; and a memory containing a program executable by the one or more computer processors to perform an operation comprising the steps of the methods of claims 1 and 4-7 (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.). The rationale for obviousness is the same as provided for claim 1 . 07-21-aia AIA Claim s 2, 9, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of Li and in further view of Transformers in Action: Weakly Supervised Action Segmentation to Ridley . Regarding claim 2 , Cheng in view of Li teaches the computer-implemented method of claim 1, wherein refining the initial type of the action comprises detecting a misclassification of the type of the action, and reclassifying the type of action to a corrected classification (every forward pass through the model to generate a new classification is a re-classification, whether the refined class changes or stays the same. See Cheng at FIG. 2), but does not teach that which is explicitly taught by Ridley. Ridley teaches wherein refining the initial type of the action reclassifies the type of the action from a misclassification as being at least part of contiguous (i.e., next or near in time or sequence 1 ) actions of two or more different types (Ridley, pg. 1, FIG. 1, “We use transformers to segment videos given only a ground-truth list of actions. The visualization of the attention mechanism illustrates how the model focuses on the starts or ends of actions comparing the attention map to the ground truth. The mechanism appears to put more emphasis on the areas around action transitions rather than the transitions themselves.”; pg. 8, section 4.2, “However, even with this ability, failure cases still exist where quick action transitions can create some ambiguity that the decoding must resolve”). Cheng in view of Li is analogous to the claimed invention for the reasons provided above. Ridley discloses an attention-based transformer model that designs the attention to focus on action-to-action boundaries (i.e., contiguity of two actions) to mitigate misclassifications of transition areas or boundaries between two actions of different types. Thus, Ridley shows that it was known in the art before the effective filing date of the claimed invention to design a TAL-based machine learning model to correct misclassifications at action-to-action boundary transitions, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, generating accurate and precise action classifications and their start and stop times. A person of ordinary skill in the art would have been motivated to add more action-to-action misclassifications as disclosed by Ridley to the ground truth dataset of Cheng in view of Li, to thereby train the model on misclassifications located temporally at action-to-action transition boundaries to refine contiguous actions of different types into a single type. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of improving the model’s performance in the areas that are hardest to accurately localize the start and end times of contiguous actions. Claim 9 substantially corresponds to claim 2 by reciting a computer program product of video processing for action detection using machine learning, the computer program product comprising: a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising the steps of the method of claim 2 (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.). The rationale for obviousness is the same as provided for claim 2. Claim 15 substantially corresponds to claim 2 by reciting a system of video processing for action detection using machine learning, the system comprising: one or more computer processors; and a memory containing a program executable by the one or more computer processors to perform an operation comprising the steps of the methods of claim 2 (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.). The rationale for obviousness is the same as provided for claim 2 . 07-21-aia AIA Claims 3, 10, and 16 are r ejected under 35 U.S.C. 103 as being unpatentable over C heng in view of Li, in further view of Ridley and in further view of Temporal Action Localization with Coarse-to-Fine Network to Zhang. R egarding claim 3 , Cheng in view of Li teaches the computer-implemented method of claim 1, wherein refining the initial video clip (Cheng — updating the model to generate a new localization prediction. See Cheng at pg. 20, section A.1, “the Temporal Boundary-Localization Module (TBLM) aims to produce the action boundaries and categories for each action instance.”) comprises detecting an incorrect localization of the type of the action and localizing the type of the action to a corrected localization (start time and/or stopping time of action instance), but does not teach that which is explicitly taught by Ridley. Ridley teaches batches of frames that that depict two or more contiguous actions that are shortened or lengthened (Ridley, pg. 1, FIG. 1, “We use transformers to segment videos given only a ground-truth list of actions. The visualization of the attention mechanism illustrates how the model focuses on the starts or ends of actions comparing the attention map to the ground truth. The mechanism appears to put more emphasis on the areas around action transitions rather than the transitions themselves.”; pg. 8, section 4.2, “However, even with this ability, failure cases still exist where quick action transitions can create some ambiguity that the decoding must resolve”; Refining the start and end positions shortens or lengthens temporal action boundaries.). The rationale for obviousness is the same as provided for claim 2. Cheng in view of Li and in further view of Ridley does not teach that which is explicitly taught by Zhang. Zhang teaches an incorrect localization (Zhang, Figure 4, lower example, “CPM” between “#923” and “#1365”) being longer than a correct localization (Zhang, Figure 4, lower example, “CFNET” between “#945” and “1341”). Cheng, Li, and Ridley are analogous to the claimed invention for the reasons provided above. Zhang discloses a boundary transformer model (BTM) that refines an incorrect localization to a corrected, shorter localization. Thus, Zhang shows that it was known in the art before the effective filing date of the claimed invention to design a TAL-based machine learning model to refine incorrect localizations at action boundaries, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, generating accurate and precise action classifications and their start and stop times. A person of ordinary skill in the art would have been motivated to train the model disclosed by Cheng in view of Li and in further view of Ridley until a predicted localization duration becomes shorter than an incorrect localization as disclosed by Zhang, to thereby train the model to more closely follow the ground truth durations for each action. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of improving the model’s performance in boundary areas including action and background boundaries and action-to-action boundaries.. Claim 10 substantially corresponds to claim 3 by reciting a computer program product of video processing for action detection using machine learning, the computer program product comprising: a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising the steps of the method of claim 3 (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.). The rationale for obviousness is the same as provided for claim 3. Claim 16 substantially corresponds to claim 3 by reciting a system of video processing for action detection using machine learning, the system comprising: one or more computer processors; and a memory containing a program executable by the one or more computer processors to perform an operation comprising the steps of the methods of claim 3 (Cheng, pg. 9, section 4.2, “Our models are trained on 4× RTX A6000 GPUs.”; For testing, “either 12GB (RTX 3080) or 32GB (Tesla V100)” was used. Cheng at pg. 10); The description of GPUs and the results of TALLFormer in Table 3 demonstrate that a computer was used, which would include a processor executing stored instructions to train and/or use TALLFormer by processing video data with the GPU.). The rationale for obviousness is the same as provided for claim 3. Conclusion 07-39 AIA THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN P POTTS whose telephone number is (571)272-6351. The examiner can normally be reached M-F, 9am-5pm EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /RYAN P POTTS/Examiner, Art Unit 2672 /SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672 Application/Control Number: 18/180,721 Page 2 Art Unit: 2672 Application/Control Number: 18/180,721 Page 3 Art Unit: 2672 Application/Control Number: 18/180,721 Page 4 Art Unit: 2672 Application/Control Number: 18/180,721 Page 5 Art Unit: 2672 Application/Control Number: 18/180,721 Page 6 Art Unit: 2672 Application/Control Number: 18/180,721 Page 7 Art Unit: 2672 Application/Control Number: 18/180,721 Page 8 Art Unit: 2672 Application/Control Number: 18/180,721 Page 9 Art Unit: 2672 Application/Control Number: 18/180,721 Page 10 Art Unit: 2672 Application/Control Number: 18/180,721 Page 11 Art Unit: 2672 Application/Control Number: 18/180,721 Page 12 Art Unit: 2672 Application/Control Number: 18/180,721 Page 13 Art Unit: 2672 Application/Control Number: 18/180,721 Page 14 Art Unit: 2672 Application/Control Number: 18/180,721 Page 15 Art Unit: 2672 Application/Control Number: 18/180,721 Page 16 Art Unit: 2672 Application/Control Number: 18/180,721 Page 17 Art Unit: 2672 Application/Control Number: 18/180,721 Page 18 Art Unit: 2672 1 https://web.archive.org/web/20210420031412/https://www.merriam-webster.com/dictionary/contiguous
Read full office action

Prosecution Timeline

Mar 08, 2023
Application Filed
Dec 19, 2025
Non-Final Rejection mailed — §101, §103
Mar 17, 2026
Applicant Interview (Telephonic)
Mar 17, 2026
Examiner Interview Summary
Mar 19, 2026
Response Filed
Jun 04, 2026
Final Rejection mailed — §101, §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12665066
DETECTION AND EVALUATION OF A SURGICAL TIME-OUT CONFIRMING SURGICAL DETAILS FROM CAPTURED VIDEO OF AN OPERATING ROOM
3y 7m to grant Granted Jun 23, 2026
Patent 12666162
IMAGE DATA PROCESSING METHOD AND IMAGE PROCESSING PROCESSOR
3y 1m to grant Granted Jun 23, 2026
Patent 12658311
ANALYZING SURGICAL VIDEOS TO DETERMINE COMPLIANCE WITH SELECTED SURGICAL GUIDELINES
3y 1m to grant Granted Jun 16, 2026
Patent 12658307
LABELING METHOD AND COMPUTING DEVICE BY DETECTING LESION REGION IN IMAGES USING A LEARNED NETWORK FUNCTION
2y 11m to grant Granted Jun 16, 2026
Patent 12648740
METHOD, APPARATUS AND SYSTEM FOR PREDICTION OF NEONATAL BRAIN DEVELOPMENT PROGNOSIS
3y 5m to grant Granted Jun 09, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

3-4
Expected OA Rounds
79%
Grant Probability
99%
With Interview (+39.3%)
2y 11m (~0m remaining)
Median Time to Grant
Moderate
PTA Risk
Based on 247 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month