DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This is in response to the applicant response filed on 07/15/2026. In the applicant’s response, claims 1, 10, and 15-17 were amended. Accordingly, claims 1-20 are pending and being examined. Claims 1, 10, and 15 are independent form.
Claim Rejections - 35 USC § 101
3. The claim rejections under 35 USC § 101 make in the previous office action mailed on 4/16/2026, have been withdrawn in view of applicant’s amendment.
Claim Rejections - 35 USC § 102
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
6. Claims 1-7, 9-15, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Schmerge et al (“ELIχR: Eliminating Computation Redundancy in CNN-Based Video Processing”, 2021, hereinafter “Schmerge”).
Regarding claim 1, Schmerge discloses a method of optimizing a network (the CNN-based video processing method, called the “ELIxR; see the title) comprising:
obtaining a video comprising a plurality of video frames (the ELIxR sequentially receives the first and next images of a video stream; see Sec. IV, para.1, and para.2);
computing, using a machine learning model, a global dependency value based on the plurality of video frames (the ELIxR detects “a set of bounding boxes (BBs) that contain all the relevant changes” between the first image and the next image and thus identifies and ignores the redundant areas, i.e., the temporal static areas, which are outside the BBs; Sec. IV, para.2; see the dependent/redundant areas (i.e., the unshaded region) and the shared region (the changed region) shown in the next input of fig.2);
deactivating a filter of the machine learning model while retaining one or more other filters of the machine learning model, based on the global dependency value (see fig.2 and Sec. IV-B, para.1, for the next input frame, the ELIxR only recomputes the changed region (see, the shaded region in the “Next input” image shown in the bottom left of in fig.2 b) ) while retaining the other regions, by using 3x3 filter instead of using 4x4 filter used for the “initial run”. It should be noticed that: for the next image, the CNN’s recomputing region for memorizing the “updated intermediate results” is the shared region having size 3x3 instead of 4x4 used for the initial run for the input image, thus the size of a filter in the CNN has been deduced to 3x3 from 4x4 and some of them are deactivated), to obtain a lightweight policy neural network (wherein ELIxR “implements a lightweight change propagation algorithm to automatically determine which data to recompute for each new frame based on changes in the input.” See Abstract, lines 12-15, and see Sec. I, para.6, lines 1-3, in the left col. on page 35); and
processing, using the lightweight policy neural network, at least a portion of the video (the ELIxR outputs the “final output”; see fig.2, b)).
Regarding claim 2, Schmerge discloses the method of claim 1, further comprising: scaling an active filter of the machine learning model by a multiplicative scaling factor based on the global dependency value (see the unrolled matrix multiplication shown by fig.3; wherein the row vectors will be multiplied by a column vector that represents
the values of the kernel filters to produce the overall output.).
Regarding claim 3, Schmerge discloses the method of claim 2, wherein processing at least the portion of the video comprises: generating a feature map for a subsequent frame of the video based on the active filter (ibid.).
Regarding claim 4, Schmerge discloses the method of claim 1, further comprising: generating a plurality of global dependency values corresponding to the plurality of video frames, respectively (see the identified non-shaded region shown in the next input of fig.2 b)).
Regarding claim 5, Schmerge discloses the method of claim 4, wherein: the filter is deactivated based at least in part on the plurality of global dependency values (see fig.2 and Sec. IV-B, para.1; wherein for the next image, the CNN’s recomputing region for memorizing the “updated intermediate results” is the shared region having size 3x3 instead of 4x4 used for the initial run for the input image, thus the size of a filter in the CNN has been deduced to 3x3 from 4x4 and some of them are deactivated).
Regarding claim 6, Schmerge discloses the method of claim 1, wherein: the global dependency value is generated by an attention mechanism of the machine learning model (see fig.2; wherein the redundant region is detected by the ELIxR including a CNN).
Regarding claim 7, Schmerge discloses the method of claim 6, wherein: the attention mechanism is a self-attention layer (ibid.).
Regarding claim 9, Schmerge discloses the method of claim 1, wherein: the filter comprises a filter of a convolutional layer of the machine learning model (ibid.).
Regarding claim 10, 15, 18, each of them essentially is an inherent variation of claim 1, thus it is interpreted and rejected for the reasons set forth in the rejection of claim 1. It should be noticed that: all of the “saved intermediate results” and the “updated intermediate results” shown in fig.2 are the features and/or the aggregated features generated by the ELIxR including a CNN.
Regarding claim 11, Schmerge discloses the method of claim 10, wherein identifying the temporally consistent feature comprises: aggregating the features using an attention mechanism (the ELIxR including a CNN; see fig.2 and Sec. VII-B--“Network Modification and Approximations”).
Regarding claim 12, Schmerge discloses the method of claim 11, wherein: the attention mechanism is a self-attention mechanism (see fig.2).
Regarding claim 13, Schmerge discloses the method of claim 10, further comprising: adjusting weights of the machine learning by a multiplicative scaling factor based on the temporally consistent feature (see Sec. VII-B--“Network Modification and Approximations”).
Regarding claim 14, 19, 20, Schmerge discloses the method of claim 10, wherein deactivating the filter comprises: generating a mask that identifies the filter for deactivation (see the identified non-shaded region shown in the next input of fig.2 b); see the unrolled matrix multiplication shown by fig.3; wherein the row vectors will be multiplied by a column vector that represents the values of the kernel filters to produce the overall output).
Claim Rejections - 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 8 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Schmerge in view of Kaplanyan et al (US 2018/0204314, hereinafter “Kaplanyan”).
Regarding claim 8, 16, Schmerge does not explicitly disclose, the global dependency value is generated by a recurrent neural network. However, using a RNN for processing spatiotemporal images is well-known and widely used in the field of video processing. As evidence, Kaplanyan teaches a recurrent neural network (RNN) based method for retaining state information between image frames. See fig.9 and para.21. It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was made to incorporate the teachings of Kaplanyan into the teachings of Schmerge and utilize a RNN for processing spatiotemporal images taught by Kaplanyan. Suggestion or motivation for doing so would have been to retain state information between image frames as taught by Kaplanyan Therefore, the claim is unpatentable over Schmerge in view of Kaplanyan.
Regarding claim 17, the combination of Schmerge and Kaplanyan discloses, wherein: the temporal attention component includes an encoder (Kaplanyan, see para.19: “the neural network may include an autoencoder. For example, the neural network may include an autoencoder that learns to reconstruct its inputs. In another embodiment, the auxiliary information may be passed through the autoencoder unchanged”).
Response to Arguments
9. Applicant’s arguments, filed on 07/15/2026, have been fully considered but they are not persuasive.
On page 9, applicant argues:
“In the Recompute Run of Schmerge, an additional dynamic resize operation is performed to prevent the changed region from increasing in size by using previous intermediate results again. Through the dynamic resize operation, the size of the changed region is reduced from 4x4 to 2x2.
In other words, Schmerge merely discloses reducing the size of the changed region from 4x4 to 2x2 during the dynamic resize process of the Recompute Run.”
The examiner respectfully points out that the applicant misinterprets the ELIxR in Schmerge. As explained in the rejections of the claims, wherein the ELIxR does not only reduces the recomputed region from 4x4 to 3x3 but also retains the values of the other unchanged regions in order. Thus, ELIxR “implements a lightweight change propagation algorithm to automatically determine which data to recompute for each new frame based on changes in the input.” The argument is unpersuasive.
Conclusion
10. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
11. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RUIPING LI whose telephone number is (571)270-3376. The examiner can normally be reached 8:30am--5:30pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HENOK SHIFERAW can be reached on (571)272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov; https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center, and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RUIPING LI/Primary Examiner, Ph.D., Art Unit 2676
Prosecution Insights