Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 8/26/26 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 4, 8, 9, 11, 15, 16 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Harkness (10,885,343) in view of Feng (CN114095754A), and further in view of Wang (WO2017161998A1).
Regarding claim 1, Harkness discloses an apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory, the at least one processor configured to:
Receive, over a network, a set of video frames (205);
identify a discontinuity in the set of video frames (215), wherein the discontinuity is caused by a condition of the network (note column 3, lines 42-67; col. 5, lines 4-14; col. 6, line 43, to col. 7, line 10);
generate one or more replacement frames associated with the discontinuity based on at least one video frame selected from among the set of video frames (220); and
provide the one or more replacement frames to a user (225).
However, Harkness does not disclose that determine, based on frame number metadata indicating an ordering of the set of video frames, that the discontinuity corresponds to a gap in the set of video frames as received, identified from missing or incomplete frame numbering; and
wherein the one or more replacement frames are generated by a machine-learning model trained to create replacement frames based on contextual information, the contextual information including at least event data as now claimed.
Feng, from the similar field of endeavor, teaches that the one or more replacement frames are generated by a machine-learning model trained to create replacement frames based on contextual information, the contextual information including at least event data (note page 8-14 of the translated text).
Harkness teaches that the replacement frame can be generated by conventional trained neural network (note col. 4, lines 24-27).
Thus, in view of Feng, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include machine learning model of Feng into Harkness to perform the well known replacement frame generation based on the contextual information including the event data as claimed.
In addition, Feng does not explicitly disclose the newly included limitation: “behaviors occurring before or after an identified discontinuity”.
However, in page 9 of the translated text, Feng discloses the following:
example, if the video section to be parsed comprises a video frame C and a video frame D. the video frame C and the video frame D are the image of the human riding, the multi-mode information extracted from the video frame C by the electronic device comprises: Scene information: grassland; Role information: people and horses; The position information of the role: a pixel position in the video frame; The posture information of the role: Two water chestnut (“water chestnut” is incorrection translation; it should be “horseshoes”) falls on the ground, and two horseshoes are not on the ground. The multi- modality information extracted from the video frame D by the electronic device includes: Scene information: grassland; Role information: people and horses; The position information of the role: a pixel position in the video frame; The posture information of the role: The four water chestnut (horseshoes) falls on the ground. the electronic device according to the multi-mode information of the video frame C and the video frame D from the plurality of scenario events matching the target scenario event is used for representing character movement event. The semantic information of the video segment to be parsed generated according to the target scenario event is: People ride on the grassland and run on the grassland.
step 1043, according to the semantic information, determining the second number of video frame to be supplemented between two adjacent display of the second video frame.
According to above passages, the people whose riding horses and running on the grassland can be interpreted as behavior of the people because riding and running are being performed by the people. Thus, above quoted passages meet the “behavior” limitation as claimed.
In addition, according to Harkness (col. 3, lines 42-67), the missing frames can be caused by many different properties or factors, such as network conditions, and the system monitoring module 175 is intended to indiscriminately detect these missing frames. That is, the system monitoring module 175 only detects the missing frames irrespective of how the missing frames came to be.
With respect to the video frame discontinuity determining function, Wang teaches a step of missing or discontinuity frame detection (note step 103 in Fig. 4). The step enables a missing or discontinuity of the video frame to be detected easily based on the numberings of the video frames. The translation of the Wang patent publication is cited as follows:
Step 103: The video frame queue is transmitted, and detecting whether there is a frame loss in the video frame queue.
Since each video frame in the video frame queue is independently numbered separately, in the process of transmitting the video frame queue, it is possible to detect whether the number of each frame conforms to the corresponding setting of the number of each video frame in the encoded video frame queue. The relationship is judged whether there is a frame loss in the video frame. For example, the video frame queue obtained by numbering each video frame by the digital serial number according to the sequence of the video frames can determine whether there is a frame loss by judging whether the digital serial number is continuous, for example, by letter pairs according to the order of the video frames. A video frame queue obtained by numbering a video frame can determine whether there is a frame loss by judging whether the arrangement of the letters still maintains the original setting rule.
Therefore, knowing a missing or discontinuity video frame detection means is required in Harkness (215), it would have been obvious to on of ordinary skill in the art before the effective filing date of the claimed invention to include Wang into Harkness so that the missing or discontinuity video frames could be detected easily.
Regarding claim 2, Harkness discloses to generate the one or more replacement frames, the at least one processor is configured to: provide at least one video frame selected from among the set of video frames to a generative machine-learning model; and receive the one or more replacement frames from the generative machine-learning model (note col. 5, lines 42-50).
Regarding claim 4, Harkness discloses that the generative machine-learning model is camera-specific (205 and 210).
Regarding claims 8, 9, 11, 15, 16 and 18, see similar rejections as set forth above.
Claim(s) 3, 7, 10, 14, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Harkness (10,885,343) in view of Feng (CN114095754A), further in view of Klein et al. (2021/0049409).
Regarding claim 3, Harkness does not disclose that the generative machine-learning model is trained using video frames collected by two or more imaging devices that have an overlapping field of view; however, Harkness does teach that the replacement frames may be generated by a generative network (col. 5, lines 48-50).
Klein, from the similar field of endeavor, teaches that when more than two or more cameras shares an overlapping field of view, video frames and any information or attributes generated from one camera can be provided and utilized by other cameras to train and/or improve a machine-learning model to classify and detect objects in image data or video frames (par. 16, 24, 29 and 31). In other words, missing frames of one camera can be generated based on information from another camera by using the machine-learning model (par. 51). This is more precise than by just using one camera alone.
Thus, in view of Klein, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include Klein into the combination of Harkness and Feng so that the machine learning model could be improved by utilizing overlapping video frames generated from two or more cameras.
Regarding claim 7, Harkness does not disclose that the at least one processor is further configured to: receive an input from the user, the input providing a quality indication for the one or more replacement frames; however, Harkness does teach that the replacement frames may be generated by a generative network (col. 5, lines 48-50).
Klein teaches a labeled training data, such as image and/or video data annotated by humans (par. 16). The annotated data can be used to indicate quality of the video frames generated. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include Klein into Harkness to perform the well known functions as claimed.
Regarding claims 10, 14, and 17, see similar rejections as set forth above.
Claim(s) 5, 6, 12, 13, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Harkness (10,885,343) in view of Feng (CN114095754A), further in view of Reda et al. (2021/0067735).
Regarding claim 5, Harkness does not disclose that to generate the one or more replacement frames, the at least one processor is configured to: provide audio data to a generative machine-learning model. Reda, from the similar field of endeavor, teaches that the audio data is provided to the generative machine-learning model (note par. 80).
Therefore, knowing that audio is required in the missing frames of Harkness, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include Reda into Harkness so that the audio data could be generated through the machine-learning model.
Regarding claims 6, 13 and 20, Harkness or Feng does not disclose that the machine-learning model is trained by applying a loss function to compare predicted output values with target output values as claimed. Reda, teaches such training (note par. 66 and steps 508, 510 and 514). By applying a loss function on the ML model training, dropped or missing frames can be replaced properly (note par. 66). Thus, in view of Harkness, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include Reda into the combination of Harkness and Feng so that the quality of replacement frame(s) could be further improved.
Regarding claims 12 and 19, see rejection to claim 5.
Response to Arguments
Applicant’s arguments, filed 8/26/26, with respect to the rejection(s) of claim(s) 1-20 under Harkness in view of Feng have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Wang.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL LEE whose telephone number 571-272-7349. The examiner can normally be reached on Monday through Thursday from 9:00 am to 6:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, John Miller, can be reached on 571-272-7353. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/MICHAEL LEE/ Primary Examiner,
Art Unit 2422
Prosecution Insights