DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/13/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to the claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Prior art Wason (Wason et al., US 2021/0141867 A1) has been newly added to assist in teaching the newly added claim amendments. Prior art Hwangbo (Hwangbo et al., US 2020/0012864 A1) is no longer used within the current rejection.
Claims 1-20 are pending; claims 1, 3, 4, 6-8, 11, 13, 16-18, and 20 have been currently amended.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 11-19, 1-10, and 20 (1-20) are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al., US 2019/0311202 A1 (Lee) and further in view of Wason et al., US 2021/0141867 A1 (Wason).
Regarding claim 11, Lee teaches a method, comprising:
in an electronic device (electronic device) ([0040]):
receiving video data including a set of video frames (receiving a video stream) ([0031]);
creating, based on the received video data, a synthetic shot dataset including a set of synthetic shots (generating synthetic training images from the video stream) ([0031]) (wherein the training images, i.e. static images, represent frames from the video and are artificially synthesized) ([0047] and [0057]);
pre-training a machine learning (ML) model (neural network) ([0031]) based on the created synthetic shot dataset (first, pre-training the neural network using synthetically generated training images) ([0031]);
selecting, from the received video data (receiving a video stream) ([0031]), training data including a first subset of video frames, wherein the first subset of video frames corresponds to a first synthetic shot from the set of synthetic shots (wherein the training data corresponds to synthesized training images) ([0047]) (wherein the training images, i.e. static images, represent frames from the video and are artificially synthesized) ([0047] and [0057]);
fine-tuning the pre-trained ML model based on the training data (fine-tuning the neural network using training videos) ([0031]);
selecting, from the received video data, a test video frame, from the first subset of video frames in the set of video frames (selecting from the video stream a reference frame) ([0059]);
applying the fine-tuned ML model on the test video frame (wherein the fine-tuned network can be used on a reference frame of the video stream) ([0059]) , wherein
determining, by the fine-tuned ML model (the fine-tuned neural network) ([0031]), features associated with the test video frame (extracting features from the reference video frame) ([0059]).
The Examiner would also like to point out that Lee’s computer system 200 includes receiving training video data 220 and training image data 230 for training the neural network 210 (Fig. 2; [0047-0048]); wherein it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that since the system takes video data for training and frames from video data for training (for creating synthetic training images) (Fig. 2; [0047-0048]) that the training images that represent frames of a video (and used to make synthesized training images) can be the video that is received as training video data.
However, Lee does not explicitly teach selecting, from the received video data, a test video frame, “wherein the test video frame succeeds” the first subset of video frames in the set of video frames; “comparing the determined features associated with the test video frame with features associated with the first subset of video frames; and determining, based on the comparison, the features associated with the test video frame is dissimilar with the features associated with the first subset of video frames; determining the test video frame corresponds to an anomaly based on the determination that the features associated with the test video frame is dissimilar with the features associated with the first subset of video frames” or “labelling the first subset of video frames as a single shot, based on the determination that the select test video frame corresponds to the anomaly, wherein the set of video frames is segmented into a set of shots based on the labeling of the first subset of video frames as the single shot; and controlling a rendering of the set of shots segmented from the set of video frames on a display device”.
Wason teaches that by analyzing a digital video file, the disclosed systems can identify video frames corresponding to a scene and a term sequence corresponding to a subset of the video frames (Abstract); wherein selecting, from the received video data (digital video file 300) (Fig. 3; [0065]), a test video frame (selecting a “selected frame”) ([0067]), wherein the test video frame succeeds the first subset of video frames in the set of video frames (wherein the selected frame can be an initial frame that is after a scene; i.e. after a contiguous set of frames) (Fig. 3; [0067]); wherein determining features associated with the test video frame (determining image features of the selected frame) ([0067]); comparing the determined features associated with the test video frame with features associated with the first subset of video frames (comparing the selected frame to the contiguous set of frames based on their image features) ([0067-0069]); and determining based on the comparison, the features associated with the test video frame is dissimilar with the features associated with the first subset of video frames (determining, based on the comparison of features, that the selected frame doesn’t match (i.e. satisfying a matching threshold) the contiguous set of frames features) ([0067]); determining the test video frame corresponds to an anomaly (determining that the selected frame corresponds to an initial frame for a different scene) ([0067]) based on the determination that the features associated with the test video frame is dissimilar with the features associated with the first subset of video frames (based on determining that the comparison of features, that the selected frame doesn’t match (i.e. satisfying a matching threshold) the contiguous set of frames features) ([0067]); and labelling the first subset of video frames as a single shot (labeling the continuous set of frames as a single scene) (Fig. 3; [0064-0067] and [0143]), based on the determination that the test video frame corresponds to the anomaly (based on knowing that the selected frame is from a separate scene) (Fig. 3; [0064-0067]), wherein the set of video frames is segmented into a set of shots based on the labeling of the first subset of video frames as the single shot (wherein the set of frames from digital video file 300 are labeled into different scenes based on the first set of continuous frames being labeled as a scene) (Fig. 3; [0064-0067] and [0143]); and controlling a rendering of the set of shots segmented from the set of video frames on a display device (a data server, a communication server, or a web-hosting server and can generate, store, receive, and/or transmit any type of data, including user inputs requesting a rendering of a video) ([0048-0049]) (wherein the interface may include one or more devices for presenting output to a user, including, but not limited to, a graphics engine, a display (e.g., a display screen)) ([0182]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee to include detecting an “anomaly” based on matching features since it allows the system to identify an initial frame and an ending frame for a set of frames corresponding to each of the scenes (Wason; [0067]).
Regarding claim 12, Lee teaches wherein the received video data includes at least one of weight information or morphing information, associated with each video frame of the set of video frames (to capture objects at different sizes, the frames are processed in different input scales (e.g., 0.5, 0.75, and 1) and the results from which can be averaged) ([0096]). Wason teaches wherein the received video data includes at least one of weight information or morphing information, associated with each video frame of the set of video frames (to identify image features within the frames of the digital video file 300, in some embodiments, the contextual translation system 112 resizes each frame to a smaller size (e.g. 256 pixels in length or 512 pixels in length)) ([0068]).
Regarding claim 13, Lee teaches wherein the creation of the synthetic shot dataset is based on synthetic data creation information (generating synthetic training images from the video stream) ([0031]) including at least one of information about inpainting information associated with white noise of objects, artificial motion information, object detection pre-training information (pre-trained for image recognition or classification for video object segmentation) ([0051] and [0074-0076]), or a structural information encoding, associated with the each video frame of the set of video frames (associated with the video frames) ([0051] and [0074-0076]).
Regarding claim 14, Lee teaches wherein at least one of the pre-training or the fine- tuning of the ML model is based on the synthetic data creation information (wherein the pre-training the neural network is based on using synthetically generated training images) ([0031]).
Regarding claim 15, Lee teaches wherein the ML model (neural network) ([0031]) corresponds to at least one of a motion tracking model, an object tracking model (object detection/tracking) ([0031]), or a multi-scale temporal encoder-decoder model (an encoder-decoder network) (Fig. 7; [0006], [0031], and [0062]).
Regarding claim 16, Lee teaches the application of a fine-tuned ML model (fine-tuned neural network) ([0031]). However, Lee does not explicitly teach “determining an anomaly score associated with the test video frame, wherein the determination of whether the test video frame corresponds to the anomaly is further based on the determination of the anomaly score associated with the test video frame”.
Wason teaches further comprising: determining an anomaly score (percentage of matching image features) ([0067]) associated with the test video frame (percentage of matching image features for the selected frame) ([0067]), wherein the determination of whether the test video frame corresponds to the anomaly (determining that the selected frame corresponds to an initial frame for a different scene) ([0067]) is further based on the determination of the anomaly score associated with the test video frame (based on determining that the matching percentage of the selected frame is lower than a matching threshold) ([0067]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee to include detecting an “anomaly” based on matching features since it allows the system to identify an initial frame and an ending frame for a set of frames corresponding to each of the scenes (Wason; [0067]).
Regarding claim 17, Lee teaches further comprising updating the training data to include the test video frame, based on the test video frame not corresponding to the anomaly (the parameters of the neural network can be determined by back propagation of errors of loss values between pixel values for a same training video frame or training image) ([0048]). Wason also teaches subsequent training iteration using back propagation to decrease a loss for the loss function ([0085]).
Regarding claim 18, Wason teaches further comprising controlling a storage of the labeled first subset of video frames as the single shot (the scene identifier stores or identifies the scenes using scene indicators) ([0143]), based on the determination that the test video frame corresponds to the anomaly (determining that the selected frame corresponds to an initial frame for a different scene) ([0067]).
Regarding claim 19, Lee teaches wherein the video data is received from a temporally weighted data buffer (wherein it would be obvious to one of ordinary skill in the art that the received images can come from storage, such as a buffer) (such as stored on computer 205) (Fig. 2; [0045-0046] and [0049]), and the ML model corresponds to a multi-head multi-model system (wherein the neural network architecture can include multiple interrelated neural networks run in parallel) ([0038-0039], [0129], and [0132]).
Regarding claim 1, see the rejection to claim 11, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 2, see the rejection to claim 12, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 3, see the rejection to claim 13, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 4, see the rejection to claim 14, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 5, see the rejection to claim 15, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 6, see the rejection to claim 16, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 7, see the rejection to claim 17, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 8, see the rejection to claim 18, as well as prior art Wason for circuitry (computer hardware) ([0171]), for they teach all the limitations within the claim language.
Regarding claim 9, Lee teaches wherein the video data is received from a temporally weighted data buffer (wherein it would be obvious to one of ordinary skill in the art that the received images can come from storage, such as a buffer) (such as stored on computer 205) (Fig. 2; [0045-0046] and [0049]).
Regarding claim 10, Lee teaches wherein the ML model corresponds to a multi-head multi-model system (wherein the neural network architecture can include multiple interrelated neural networks run in parallel) ([0038-0039], [0129], and [0132]).
Regarding claim 20, see the rejection made to claim 11 as well as prior art Lee for a non-transitory computer-readable medium (a non-transitory computer-readable medium) ([0104]) having stored thereon, computer-executable instructions (having stored thereon computer-readable instructions) ([0104]) that when executed by an electronic device (executable by a processor of a computer system) ([0104]), causes the electronic device to execute operations (the execution of such instructions configures the computer system to perform the specific operations) ([0104]), for they teach all the limitations within the claim language.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J VANCHY JR whose telephone number is (571)270-1193. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J VANCHY JR/Primary Examiner, Art Unit 2666 Michael.Vanchy@uspto.gov
Prosecution Insights