Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment and remarks filed September 1st, 2026. In the amendment, claims 1, 3, 10-11, 16-17, and 19 were amended, claim 15 was cancelled and no claims were added. As such, claims 1-14 and 16-21 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on August 27th, 2026, is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s arguments, see Page 7, filed September 1st, 2026, with respect to the rejected claims 3-5, 11-12, and 19-21 under 35 U.S.C. 112b have been fully considered and are persuasive. The rejected claims 3-5, 11-12, and 19-21 under 35 U.S.C. 112b have been withdrawn.
Applicant’s arguments with respect to the rejections are not persuasive for the following reasons:
35 U.S.C 101:
Applicant argues that amended claim 1 is not directed merely to selecting, generating, or determining data in the abstract, but to a specific video-processing training-sample generation technique because the claim recites that the contrastive learning model is for video processing, the plurality of data sequences are video sequences, and the first and second data segments are video segments (Pages 8 of Remarks). The Examiner respectfully disagrees. As set forth in the updated rejection at Step 2A Prong 1, the selecting, generating, and determining steps remain mental processes that can be performed by observation, evaluation, and/or judgement, and reciting that the data are video sequences and video segments only narrows the type of data operated upon. Selecting a data frame from among data sequences based on a color range is itself an observation and evaluation that can be performed by human judgement. For example, one can select a data frame from a video mentally by comparing the colors present in two video sequences. Limiting an abstract idea to a particular technological environment or field of use such as video does not remove the claim from the mental process grouping. Therefore, the claim still recites the identified abstract idea.
Applicant argues that the claimed technique is integrated into a practical application because constructing negative samples in the claimed manner introduces richer appearance disturbance, generates more negative samples without expanding the training set, and thereby improves the accuracy of the contrastive learning model and downstream models, citing specification paragraphs 35-37 and 42-48 (Page 8 of Remarks). The Examiner respectfully disagrees. As set forth at Step 2A Prong 2 and Step 2B of the updated rejection, an improved training sample yields a better result of the abstract process, which is an improvement to the analytical outcome rather than an improvement to the functioning of a computer or to any other technology. The additional elements, namely obtaining the data segments and applying the result for training the contrastive learning model, amount to insignificant extra-solution activity of data gathering and to mere instructions to apply the abstract idea using a computer as a tool (MPEP 2106.05(f) and (g)). The asserted gains in model accuracy and training efficiency are benefits of the abstract idea itself and do not impose a meaningful technological limitation. Therefore, the exception is not integrated into a practical application and the claim does not amount to significantly more.
Applicant argues that independent claims 16 and 17 recite the same subject matter and are eligible for the same reasons, and that dependent claims 2-14 and 18-21 are eligible by virtue of their dependency (Page 8 of Remarks). The Examiner respectfully disagrees. Claims 16 and 17 recite the same abstract idea as claim 1, and their additional elements, the electronic device with a processing unit and memory and the non-transitory computer-readable storage medium, are recited at a high level of generality as tools to implement the abstract idea, as set forth in the updated rejection. The rejection of claims 1-14 and 16-21 under 35 U.S.C. 101 is therefore maintained.
35 U.S.C 103:
Applicant argues that Wang selects its static background-noise frame purely at random and gives no criterion conditioned on color, appearance, or any property of the data sequences, and that none of the cited references teach selecting the data frame based on a color range of the plurality of data sequences (Pages 9-10 of Remarks). The Examiner respectfully disagrees, and notes that this argument is directed to the prior combination and does not address the current grounds of rejection. The limitation is relied upon as taught by Wang in view of Lee, not by Wang alone. Wang is relied on for selecting a data frame from a further data sequence than the second data sequence, and Lee supplies the color-range basis by building a per-video color histogram over a sequence of frames and selecting a video based on a threshold comparison of those histograms. Therefore, the combination of Wang and Lee teaches the limitation as a whole.
Applicant argues that Lee does not cure the deficiency because Lee is directed to video matching and copy detection, determining that videos match when a comparison metric is at or below a threshold, whereas amended claim 1 uses the color range to select a contrasting data sequence, arguing that Lee teaches away from the claimed selection (Page 11 of Remarks). The Examiner respectfully disagrees. Amended claim 1 recites selecting the data frame based on a color range of the plurality of data sequences and it does not recite selecting a sequence whose color range contrasts with or differs from that of the second data sequence. The argued contrast limitation appears only in the specification and is not in the claim. Lee builds a per-video color histogram over a sequence of frames and selects a video based on a thresholded comparison of those histograms, which teaches selecting a video based on a color range as claimed regardless of whether the threshold identifies a similar or a dissimilar video. Therefore, the argument is not persuasive.
Applicant argues that a person of ordinary skill would not have had a reason to modify Wang's random frame selection by incorporating Lee's video-matching metric and using it for the different purpose of selecting a color-contrast video sequence and that there is no reasonable expectation of success (Page 11 of Remarks). The Examiner respectfully disagrees. As stated in the updated rejection, a person of ordinary skill would have combined Wang's selection of a static frame from a further video with Lee's technique of comparing color-histogram color ranges in order to enable a color-driven selection of the donor video sequence and to improve contrastive learning performance. Wang expressly teaches drawing the interfering frame from another video, and Lee provides a known color-histogram mechanism for selecting a video based on its colors, so combining Lee's known technique with Wang's frame selection teaches the limitation.
Claim Objections
Claims 1-14 and 16-21 are objected to because of the following informalities:
Claims 1, 16, and 17: “selecting a data frame from a further data sequence than the second data sequence” should read “selecting a data frame from a data sequence other than the second data sequence”.
Claim 1: “determining a negative sample pair for training the contrastive learning model” should read “determining the negative sample pair for training the contrastive learning model”.
Claim 2: “updating a data frame in the second data segment with the noise data frame” should read “updating a second data frame in the second data segment with the noise data frame”. Each occurrence of “the data frame” associated with the second data segment in claims 5, 6, 18, and 21 should be corrected as well.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 and 16-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an
abstract idea without significantly more.
Claim 1
Step 1: The claim recites a method; therefore, it is directed to the statutory category of a process.
Step2A Prong 1: The claim recites:
selecting a data frame from a further data sequence than the second data sequence in the plurality of data sequences based on a color range of the plurality of data sequences: This limitation recites a mental process because it involves the observation/evaluation/judgement to pick a frame from among a set of data sequences based on a color range, which can be performed mentally or with a pen and paper.
generating a third data segment based on the second data segment and the data frame: This limitation recites a mental process because it involves the evaluation/judgement to construct a new data segment from an existing segment and a single frame, which can be performed mentally or with a pen and paper.
and determining a negative sample pair… based on the first data segment and the third data segment: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion to designate two data segments as forming a negative pair, which can be performed mentally or with a pen and paper.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
obtaining a first data segment from a first data sequence in a plurality of data sequences… and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments: Mere data gathering recited at a high level of generality, and thus is an insignificant extra-solution activity (MPEP 2106.05(g)).
…for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
obtaining a first data segment from a first data sequence in a plurality of data sequences… and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments: The additional element of “obtaining” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
…for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
The elements in combination as an ordered whole still do not amount to significantly more than the judicial exception (i.e., the abstract idea of mental processes for selecting a frame from one data sequence, generating a third data segment by combining that frame with another segment, and determining the result as a negative sample pair). The claim merely describes a process of applying known data-handling steps (selecting a portion of one sequence, modifying another sequence using that selection, and labeling the pair) and merely recites performing standard data processing steps (obtaining data segments from sequences and outputting a designated negative sample pair). The recitation of training the contrastive learning model and using the model to generate a negative sample pair merely indicates a technological environment in which the abstract ideas are applied, without improving the functioning of a computer or the model itself.
Therefore, the claim as a whole remains focused on the abstract idea and fails Step 2B of the eligibility analysis.
Claim 2
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
generating the third data segment comprises: generating a noise data frame based on the data frame: This limitation recites the evaluation/judgement to construct a noise frame from an input frame, which can be performed by pen and paper and human judgement.
and updating a data frame in the second data segment with the noise data frame: This limitation recites a mental process because it involves the evaluation/judgement to substitute or modify a frame using another frame, which can be performed by pen and paper and human judgement.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 3
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
generating the noise data frame comprises: generating an intermediate data frame by adjusting dimensions of the data frame according to a predetermined ratio: This limitation encompasses a mental process because it involves the evaluation/judgement to perform the resizing, which can be performed by pen and paper.
generating a plurality of copied intermediate data frames by copying the intermediate data frame: This limitation recites a mental process because it involves the observation/evaluation to replicate a frame multiple times, which can be performed by pen and paper.
and generating the noise data frame by joining the plurality of copied intermediate data frames: This limitation recites a mental process because it involves the evaluation/judgement to concatenate or tile multiple frames into a single composite, which can be performed by pen and paper.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 4
Step 1: A process, as above.
Step2A Prong 1: This claim does not recite any new abstract ideas but depends on claim 3 which recites abstract ideas.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
the dimensions of the data frame comprise at least one of a width and a height: The limitation merely describes the type of data being processed and thus amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
the dimensions of the data frame comprise at least one of a width and a height: The limitation merely describes the type of data being processed and thus amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself which cannot provide inventive concept (MPEP 2106.05(h)).
Even when considered in combination, this additional element represents mere instructions to apply
an exception and therefore does not provide an inventive concept. The claim is ineligible.
Claim 5
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
in response to determining that a resolution of the noise data frame is different from that of the data frame in the second data segment, performing at least one of: This limitation recites a mental process because it involves the observation/evaluation/judgement to compare resolutions of two frames and to crop or scale one to match the other, which can be performed by pen and paper and human judgement.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
cropping the noise data frame to the resolution of the data frame in the second data segment; scaling the noise data frame to the resolution of the data frame in the second data segment: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
cropping the noise data frame to the resolution of the data frame in the second data segment; scaling the noise data frame to the resolution of the data frame in the second data segment: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself which cannot provide inventive concept (MPEP 2106.05(h)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 6
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
updating the data frame in the second data segment with the noise data frame comprises: for a given data point in the data frame in the second data segment, obtaining a data value of the given data point: This limitation recites a mental process because it involves it involves the observation/evaluation/judgement of updating a data frame with noise and obtaining a data value from the datapoint, which can be performed by pen and paper.
obtaining a corresponding data value of a data point in the noise data frame corresponding to the given data point: This limitation recites a mental process because it involves it involves the observation/evaluation/judgement of obtaining a data value of a data point in the noise data based on a given data point.
determining a data value of a data point in the data frame corresponding to the given data point based on the data value, the corresponding data value, and a weight of the noise data frame: This limitation recites a mental process because it involves it involves the observation/evaluation/judgement of determining a data value corresponding to a data point in the data frame based on certain weights and values.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 7
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
obtaining the first data segment comprises: selecting the first data segment satisfying a predetermined length from the first data sequence, the first data segment comprising only a single shot: This limitation recites a mental process because it involves the observation/evaluation/judgement to identify a fixed length single shot span within a data sequence, which can be performed by pen and paper and human judgement.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, this additional element represents mere instructions to apply
an exception and therefore does not provide an inventive concept. The claim is ineligible.
Claim 8
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
obtaining the second data segment comprises: selecting the second data segment satisfying the predetermined length from the second data sequence, the second data segment comprising only a single shot: This limitation recites a mental process because it involves the observation/evaluation/judgement to identify a fixed length single shot span within a data sequence, which can be performed by pen and paper and human judgement.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, this additional element represents mere instructions to apply
an exception and therefore does not provide an inventive concept. The claim is ineligible.
Claim 9
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
determining a data range of a plurality of data frames in the second data sequence: This limitation encompasses a mental concept because it involves the observation/evaluation/judgement of determining a data range based on the plurality of data frames in a data sequence.
determining a given data range of a plurality of data frames in a given data sequence in the further data sequence: This limitation encompasses a mental concept because it involves the observation/evaluation/judgement of determining a data range based on the plurality of data frames in a further data sequence.
and in response to determining that a difference between the data range and the given data range satisfies a predetermined condition, selecting the data frame from the given data sequence: This limitation encompasses a mental concept because it involves the observation/evaluation/judgement of selecting a data frame from a data sequence based on a predetermined condition.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 10
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
generating the third data segment further comprises: adjusting a sampling frequency of a plurality of data frames in the third data segment: This limitation recites a mental process because it involves the observation/evaluation/judgement of updating/adjusting a constant number for the sampling frequency, which can be performed by pen or paper. See Paragraph 35 which discloses how the sampling frequency is adjusted.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, this additional element represents mere instructions to apply
an exception and therefore does not provide an inventive concept. The claim is ineligible.
Claim 11
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
generating a fourth data segment by adjusting a sampling frequency of a plurality of data frames in the first data sequence: This limitation recites a mental process because it involves the observation/evaluation/judgement of adjusting a parameter (sampling frequency) to generate a fourth data segment, which can be performed by pen and paper. See Paragraph 35 which discloses how the sampling frequency is adjusted.
and determining a negative sample pair… based on the first data segment and the fourth data segment: This limitation recites a mental process because it involves the observation/evaluation/judgement of determining a negative sample
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 12
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
generating a fifth data segment based on the fourth data segment and the data frame: This limitation recites a mental process because it involves the observation/evaluation/judgement of generating a data segment based on a data frame and a different data segment, which can be performed by pen and paper.
and determining a negative sample pair…based on the first data segment and the fifth data segment: This limitation recites a mental process because it involves the observation/evaluation/judgement of determining a negative sample pair based on observed data.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 13
Step 1: A process, as above.
Step2A Prong 1: The claim recites:
obtaining a sixth data segment from the first data sequence: This limitation recites a mental process because it involves the observation/evaluation/judgement of obtaining a data segment.
generating a seventh data segment based on the sixth data segment and the data frame: This limitation recites a mental process because it involves the observation/evaluation/judgement of generating a data segment based on a data frame and a different data segment, which can be performed by pen and paper.
and determining a positive sample pair… based on the first data segment and the seventh data segment: This limitation recites a mental process because it involves the observation/evaluation/judgement of determining a positive sample pair based on observed data.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 14
Step 1: A process, as above.
Step2A Prong 1: This claim does not recite any new abstract ideas but depends on claim 1 which recites abstract ideas.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
training the contrastive learning model with the positive sample pair and the negative sample pair: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
training the contrastive learning model with the positive sample pair and the negative sample pair: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 16
Step 1: The claim recites an electronic device; therefore, it is directed to the statutory category of a
machine.
Step2A Prong 1: The claim recites:
selecting a data frame from a further data sequence than the second data sequence in the plurality of data sequences based on a color range of the plurality of data sequences: This limitation recites a mental process because it involves the observation/evaluation/judgement to pick a frame from among a set of data sequences, which can be performed by pen and paper and human judgement.
generating a third data segment based on the second data segment and the data frame: This limitation recites a mental process because it involves the evaluation/judgement to construct a new data segment from an existing segment and a single frame, which can be performed by pen and paper and human judgement.
and determining a negative sample pair for training the contrastive learning model based on the first data segment and the third data segment: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion to designate two data segments as forming a negative pair, which can be performed by pen and paper.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
[a]n electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform a method: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
obtaining a first data segment from a first data sequence in a plurality of data sequences… and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments: Mere data gathering recited at a high level of generality, and thus is an insignificant extra-solution activity (MPEP 2106.05(g)).
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
[a]n electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform a method: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
obtaining a first data segment from a first data sequence in a plurality of data sequences… and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments: The additional element of “obtaining” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and is well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 17
Step 1: The claim recites a non-transitory computer-readable storage medium; therefore, it is directed to the statutory category of an article of manufacture.
Step2A Prong 1: The claim recites:
selecting a data frame from a further data sequence than the second data sequence in the plurality of data sequences based on a color range of the plurality of data sequences: This limitation recites a mental process because it involves the observation/evaluation/judgement to pick a frame from among a set of data sequences, which can be performed by pen and paper and human judgement.
generating a third data segment based on the second data segment and the data frame: This limitation recites a mental process because it involves the evaluation/judgement to construct a new data segment from an existing segment and a single frame, which can be performed by pen and paper and human judgement.
and determining a negative sample pair for training the contrastive learning model based on the first data segment and the third data segment: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion to designate two data segments as forming a negative pair, which can be performed by pen and paper.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
[a] non-transitory computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing a method: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
obtaining a first data segment from a first data sequence in a plurality of data sequences… and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments: Mere data gathering recited at a high level of generality, and thus is an insignificant extra-solution activity (MPEP 2106.05(g)).
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
[a] non-transitory computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing a method: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
obtaining a first data segment from a first data sequence in a plurality of data sequences… and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments: The additional element of “obtaining” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
for training the contrastive learning model: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 18 recites similar limitations to claim 2. Therefore, claim 18 is rejected using the same rationale as claim 2.
Claim 19 recites similar limitations to claim 3. Therefore, claim 19 is rejected using the same rationale as claim 3.
Claim 20 recites similar limitations to claim 4. Therefore, claim 20 is rejected using the same rationale as claim 4.
Claim 21 recites similar limitations to claim 5. Therefore, claim 21 is rejected using the same rationale as claim 5.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 6, 9, 13-14, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (“Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning”, 2021) in view of Lee (US 20130148884 A1), and further in view of Ding (“Motion-aware Contrastive Video Representation Learning via Foreground-background Merging”, 2021).
Regarding claim 1,
Wang teaches [a] method of generating a negative sample pair of a contrastive learning model for video processing, comprising (Page 2 Introduction of Wang, “Specifically, given a video, we randomly select a static frame and add it to every other frames to construct a distracting video, as is shown in Figure 3. Then we force the model to pull the feature of the distracting video and the feature of the original video together by consistency regularization. In this way, we made a disturbance to the video background and require its feature to be consistent with the original video…”, Page 4 Section 3.3.2 Contrastive Learning, “The hard negative set N2i for xi is defined as: N2i = {x_h i |x_h i ̸ = xo i, x_h i ∈ x_i}, and the overall negative set is Ni = {N1i ∪N2i}. Then the final objective function is: [Equation 7] For efficiency, we randomly select one hard negative sample from N2i each iteration and we find more hard negative samples have a similar result experimentally.”
Wang teaches a self-supervised contrastive learning framework in which a model is trained using positive sample pairs constructed from a video with its background-noise-augmentation, and negative sample pairs drawn from other videos.):
obtaining a first data segment from a first data sequence in a plurality of data sequences for training the contrastive learning model (Page 3 Section 3.1, “The framework of the proposed BE is shown in Figure 2. For each input video x, we first randomly crop two fixed length clips from different spatial locations, denoted as x_o and x_v.”, Page 4 Section 3.3.2, “Given a video dataset D with N videos D={x1,x2,...,xN}, for each video xi, we randomly sample once in each epoch, obtaining xo i and xd i.”
Wang teaches that each video in the training dataset is treated as one data sequence, and that fixed-length clips are cropped from each video as data segments. The original clip x_o corresponds to the first data segment drawn from the first data sequence (the first video) within the plurality of training videos.),
and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments (Page 4 Section 3.3.2, “Given a video dataset D with N videos D={x1,x2,...,xN}, for each video xi, we randomly sample once in each epoch, obtaining xo i and xd i.… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity,”, Page 3 Section 3.1, “For each input video x, we first randomly crop two fixed length clips from different spatial locations, denoted as xo and xv.”, Page 3 Section 3.2, “for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video…”
Wang teaches the dataset D={x1,x2,…,xN} containing multiple videos (data sequences), where each video corresponds to a separate data sequence. From each video, Wang crops fixed length clips which correspond to the claimed data segments. The second data segment is a clip drawn from a different video (a clip s(xn)), which is the second data segment obtained from a second data sequence in the plurality of video sequences.);
selecting a data frame from a further data sequence than the second data sequence in the plurality of data sequences… (Page 3 Section 3.2, “Specifically, for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity…”, Page 6 Section 4.4, “Inter-Video Frame: randomly select one frame from another video, and add this static frame as noise to each frame of this video.”
Wang teaches using an Inter-Video Frame where a static frame is randomly selected from a video other than the video being modified. Under broadest reasonable interpretation, this static frame corresponds to a data frame selected from a further (i.e., additional) data sequence than the second data sequence in the plurality of data sequences.)
and determining a negative sample pair for training the contrastive learning model based on the first data segment and the [clip of a further data sequence] (Page 4 Section 3.3.2, “Contrastive learning … aims to learn an invariant representation for each sample, which is achieved by maximizing similarity of similar pairs over dissimilar pairs… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n= i}, each element in N1i is a clip and represents an identity… In order to make the objective more challenge, we introduce hard negatives, the different video clips… The hard negative set N2i for xi is defined as: N2i = {xh i |xh i = xo i,xh i ∈ xi}, and the overall negative set is Ni = {N1i ∪N2i}.”, Page 3 Fig. 2 Caption, “The model is trained by a existing self-supervised task together with a consistency constraint, with the goal of pulling the feature of the original video and that of the distracting video closer.”
Wang trains the contrastive learning model with an InfoNCE objective in which the anchor clip x_o (the first data segment) is contrasted against a negative set Ni. The clip drawn from a different video (s(xn)) is designated a negative and pushed apart from the anchor such that the anchor x_o together with the different video clip constitutes the negative sample pair (x_o, s(xn)).)
Wang does not teach selecting a [video] … in the plurality of data sequences based on a color range of the plurality of data sequences… generating a third data segment based on the second data segment and the data frame
Lee, in the same field of endeavor, teaches selecting a [video] … in the plurality of data sequences based on a color range of the plurality of data sequences (Paragraph 16 of Lee, "a color histogram for a video (e.g., corresponding to a video frame or a sequence of video frames from a video) includes a set of bins (also referred to herein as color bins) representing a respective set of possible colors that may be included in the video frame(s)", Paragraph 17, "obtaining a first color histogram corresponding to a sequence of frames of the first video ... and obtaining a second color histogram corresponding to a sequence of frames of the second video", Paragraph 52, "the comparison metric can be represented mathematically as the quantity ComparisonMetric and determined mathematically by summing the differences between the bin values of the baseline color histogram and the respective adjusted bin values of the trial color histogram", Paragraph 35, "the histogram comparator 150 may determine that the test video matches the particular reference video stored in the reference video library 105 if at least one of the comparison metrics quantifying comparison of the reference video with the test video is less than or equal to a threshold"
Lee builds for each video a color histogram over a sequence of frames that quantifies the colors present in that video. This per-video color histogram is the claimed color range of a data sequence. Lee then computes a comparison metric from the differences between two videos' color histograms and selects the video whose metric meets a threshold.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang’s selection of a static frame from a further video with Lee's technique of selecting a video by comparing color-histogram color ranges in order to enable a color-driven selection of the donor video sequence and improve contrastive learning performance (Paragraphs 43 and 36 of Lee).
Wang in view of Lee does not teach generating a third data segment based on the second data segment and the data frame.
Ding, in the same field of endeavor, teaches generating a third data segment based on the second data segment and the data frame (Page 5 Section 3.2 of Ding, " Denoting X,Y as foreground and background source clips, the synthetic clip
PNG
media_image1.png
25
260
media_image1.png
Greyscale
", Page 4 Figure 4 Caption, "We first randomly sample two clips from a video and use FAME to generate new clips by composing the original foreground onto various backgrounds from other videos"
Ding generates a new synthetic clip X_merge by compositing a clip X with content Y drawn from a different further video. Ding’s clip X corresponds to the second data segment and the cross-video content Y corresponds to the data frame content selected from the further data sequence that merges each other to form X_merge (third data segment).)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee's addition of a color-selected data frame to a clip with Ding’s compositing of a clip with content drawn from a different video so that the third data segment is generated from the second data segment obtained from a different data sequence and the data frame from a further data sequence in order to increase the appearance diversity of the constructed sample and improve the learned video representation (Section 4.3 of Ding).
Regarding claim 2,
Wang teaches generating the third data segment comprises: generating a noise data frame based on the data frame; and updating a data frame in the second data segment with the noise data frame (Page 6 Section 4.4, “One main operation in the background erasing is to generate a distracting video with background noise while retaining the temporal semantics… (a). Gaussian Noise: add an identical White Gaussian Noise on each frame.”, See Equation 1, “
PNG
media_image2.png
32
356
media_image2.png
Greyscale
”
Wang generates a noise data frame by multiplying the selected static frame by a weight lambda, creating a noise signal to be added to the original frame. Wang updates each frame (x-hat) of the second data segment by adding the noise data frame to it via a weighted linear combination.).
Regarding claim 6,
Wang teaches updating the data frame in the second data segment with the noise data frame comprises: for a given data point in the data frame in the second data segment, obtaining a data value of the given data point (Page 3 Section 3.1, “C is the number of channel and T is the length of time dimension. W and H are spatial size. At last, the extracted features fxo, fxd are pulled closer within the existing self-supervised methods.”, Page 3 Section 3.2, “Specifically, for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video, in which each frame ˆx is obtained by the following formula:
PNG
media_image3.png
33
361
media_image3.png
Greyscale
where λ is sampled from the uniform distribution [0,γ], x(j) means the j-th frame of x…”
Wang teaches that each frame x^j of the second data segment x is a structured tensor over channel x height x width where each channel position within x^j is a data point, and its pixel intensity is the data value of that data point. Equation 1 demonstrates obtaining the data value at each of these data points.);
obtaining a corresponding data value of a data point in the noise data frame corresponding to the given data point (Page 3 Section 3.2, “…we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video, in which each frame ˆx is obtained by the following formula:
PNG
media_image3.png
33
361
media_image3.png
Greyscale
… k denotes the index of the randomly selected frame and T is the length of the video sequence x…”, Page 2 Introduction, “Specifically, given a video, we randomly select a static frame and add it to every other frames to construct a distracting video…”
Wang teaches that x^k is the randomly selected static frame (noise data frame) which supplies a data value at the same channel position (i.e., the corresponding data point) used in the linear combination of Equation 1.);
and determining a data value of a data point in the data frame corresponding to the given data point based on the data value, the corresponding data value, and a weight of the noise data frame (Section 3.2 on Page 3, “each frame ˆx is obtained by the following formula:
PNG
media_image3.png
33
361
media_image3.png
Greyscale
where λ is sampled from the uniform distribution [0,γ]… the time derivative of xd is a linear transformation of the time derivative of xv, formally:
PNG
media_image4.png
41
262
media_image4.png
Greyscale
where δ represents the result of repeating the selected frame x(k) T times along the time dimension.”
Wang determines the value x-hat for each data point by combining the original data value x^j, the corresponding data value x^k from the noise data frame, and the weight (lambda), which controls the contribution of the noise data frame. X-hat is calculated as a weight combination of the original and noise values and Equation 2 further explains that this element-wise operation is performed across all data points. As a result, the updated x-hat represents the determined data value, which corresponds to the data point in the updated data frame.).
Regarding claim 13,
Wang teaches obtaining a sixth data segment from the first data sequence (Page 3 Section 3.1, “For each input video x, we first randomly crop two fixed length clips from different spatial locations, denoted as xo and xv.”, Page 4 Section 3.3.2, “for each video xi, we randomly sample once in each epoch, obtaining xo i and xd i.”
Wang teaches that two clips are cropped from each input video. The second clip x_v corresponds to the sixth data segment drawn from the first data sequence (first data segment x_o).);
generating a seventh data segment based on the sixth data segment and the data frame (Page 3 Section 3.2, “Specifically, for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video, in which each frame ˆx is obtained by the following formula:
PNG
media_image3.png
33
361
media_image3.png
Greyscale
“
Wang teaches that the same noise injection augmentation is applied to x_v to produce a distracting variant (seventh data segment), which is generated based on the sixth data segment and the same data frame.);
and determining a positive sample pair for training the contrastive learning model based on the first data segment and the seventh data segment (Page 4 Section 3.3.1, “For handcrafted pretext task, we use a consistency regularization term to pull the feature of xo closer to the feature of xd, and make them consistent in the temporal dimension.”, Page 4 Section 3.3.2, “Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity, then the InfoNCE[36] loss is improved as follows:
PNG
media_image5.png
61
319
media_image5.png
Greyscale
”
Wang teaches that the original and the distracting clip from the same video is the positive pair. The first data segment, seventh data segment pair is the positive sample pair.).
Regarding claim 14,
Wang teaches training the contrastive learning model with the positive sample pair and the negative sample pair (Page 4 Section 3.3.2, “In order to add a consistency constraint between xo and xd, we directly treat their features f(xo) and f(xd) as positive pairs instead of using MSEloss…each element in N1i is a clip and represents an identity, then the InfoNCE[36] loss is improved as follows… Then the final objective function is:
PNG
media_image6.png
61
317
media_image6.png
Greyscale
For efficiency, we randomly select one hard negative sample from N2i each iteration and we find more hard negative samples have a similar result experimentally.”
Wang teaches that the contrastive learning model is trained with the InfoNCE loss using positive pair (x_o, x_d) and the negative pairs (x_o, x_n). This is the training model with both positive and negative pairs.).
Regarding claim 16,
Wang teaches perform a method of generating a negative sample pair of a contrastive learning model for video processing, the method comprising (Page 2 Introduction of Wang, “Specifically, given a video, we randomly select a static frame and add it to every other frames to construct a distracting video, as is shown in Figure 3. Then we force the model to pull the feature of the distracting video and the feature of the original video together by consistency regularization. In this way, we made a disturbance to the video background and require its feature to be consistent with the original video…”, Page 4 Section 3.3.2 Contrastive Learning, “The hard negative set N2i for xi is defined as: N2i = {x_h i |x_h i ̸ = xo i, x_h i ∈ x_i}, and the overall negative set is Ni = {N1i ∪N2i}. Then the final objective function is: [Equation 7] For efficiency, we randomly select one hard negative sample from N2i each iteration and we find more hard negative samples have a similar result experimentally.”
Wang teaches a self-supervised contrastive learning framework in which a model is trained using positive sample pairs constructed from a video with its background-noise-augmentation, and negative sample pairs drawn from other videos.):
obtaining a first data segment from a first data sequence in a plurality of data sequences for training the contrastive learning model (Page 3 Section 3.1, “The framework of the proposed BE is shown in Figure 2. For each input video x, we first randomly crop two fixed length clips from different spatial locations, denoted as x_o and x_v.”, Page 4 Section 3.3.2, “Given a video dataset D with N videos D={x1,x2,...,xN}, for each video xi, we randomly sample once in each epoch, obtaining xo i and xd i.”
Wang teaches that each video in the training dataset is treated as one data sequence, and that fixed-length clips are cropped from each video as data segments. The original clip x_o corresponds to the first data segment drawn from the first data sequence (the first video) within the plurality of training videos.),
and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments (Page 4 Section 3.3.2, “Given a video dataset D with N videos D={x1,x2,...,xN}, for each video xi, we randomly sample once in each epoch, obtaining xo i and xd i.… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity,”, Page 3 Section 3.1, “For each input video x, we first randomly crop two fixed length clips from different spatial locations, denoted as xo and xv.”, Page 3 Section 3.2, “for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video…”
Wang teaches the dataset D={x1,x2,…,xN} containing multiple videos (data sequences), where each video corresponds to a separate data sequence. From each video, Wang crops fixed length clips which correspond to the claimed data segments. The second data segment is a clip drawn from a different video (a clip s(xn)), which is the second data segment obtained from a second data sequence in the plurality of video sequences.);
selecting a data frame from a further data sequence than the second data sequence in the plurality of data sequences… (Page 3 Section 3.2, “Specifically, for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity…”, Page 6 Section 4.4, “Inter-Video Frame: randomly select one frame from another video, and add this static frame as noise to each frame of this video.”
Wang teaches using an Inter-Video Frame where a static frame is randomly selected from a video other than the video being modified. Under broadest reasonable interpretation, this static frame corresponds to a data frame selected from a further (i.e., additional) data sequence than the second data sequence in the plurality of data sequences.);
and determining a negative sample pair for training the contrastive learning model based on the first data segment and the [clip of a further data sequence] (Page 4 Section 3.3.2, “Contrastive learning … aims to learn an invariant representation for each sample, which is achieved by maximizing similarity of similar pairs over dissimilar pairs… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n= i}, each element in N1i is a clip and represents an identity… In order to make the objective more challenge, we introduce hard negatives, the different video clips… The hard negative set N2i for xi is defined as: N2i = {xh i |xh i = xo i,xh i ∈ xi}, and the overall negative set is Ni = {N1i ∪N2i}.”, Page 3 Fig. 2 Caption, “The model is trained by a existing self-supervised task together with a consistency constraint, with the goal of pulling the feature of the original video and that of the distracting video closer.”
Wang trains the contrastive learning model with an InfoNCE objective in which the anchor clip x_o (the first data segment) is contrasted against a negative set Ni. The clip drawn from a different video (s(xn)) is designated a negative and pushed apart from the anchor such that the anchor x_o together with the different video clip constitutes the negative sample pair (x_o, s(xn)).)
Wang does not teach [a]n electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to… selecting a [video] … in the plurality of data sequences based on a color range of the plurality of data sequences… generating a third data segment based on the second data segment and the data frame;
Lee, in the same field of endeavor, teaches [a]n electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to… (Paragraph 62 of Lee, “the example … may be implemented using coded instructions (e.g., computer readable instructions) stored on a tangible computer readable medium such as a hard disk drive, a flash memory, a read-only memory (ROM)… The tangible computer readable medium can be local to and/or remote from the processor(s), device(s), apparatus, etc., that is/are to execute the coded instructions… any type of computer readable storage and to exclude propagating signals… stored on a non-transitory computer readable medium”)
selecting a [video] … in the plurality of data sequences based on a color range of the plurality of data sequences (Paragraph 16 of Lee, "a color histogram for a video (e.g., corresponding to a video frame or a sequence of video frames from a video) includes a set of bins (also referred to herein as color bins) representing a respective set of possible colors that may be included in the video frame(s)", Paragraph 17, "obtaining a first color histogram corresponding to a sequence of frames of the first video ... and obtaining a second color histogram corresponding to a sequence of frames of the second video", Paragraph 52, "the comparison metric can be represented mathematically as the quantity ComparisonMetric and determined mathematically by summing the differences between the bin values of the baseline color histogram and the respective adjusted bin values of the trial color histogram", Paragraph 35, "the histogram comparator 150 may determine that the test video matches the particular reference video stored in the reference video library 105 if at least one of the comparison metrics quantifying comparison of the reference video with the test video is less than or equal to a threshold"
Lee builds for each video a color histogram over a sequence of frames that quantifies the colors present in that video. This per-video color histogram is the claimed color range of a data sequence. Lee then computes a comparison metric from the differences between two videos' color histograms and selects the video whose metric meets a threshold.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang’s selection of a static frame from a further video with Lee's technique of selecting a video by comparing color-histogram color ranges in order to enable a color-driven selection of the donor video sequence and improve contrastive learning performance (Paragraphs 43 and 36 of Lee).
Wang in view of Lee does not teach generating a third data segment based on the second data segment and the data frame.
Ding, in the same field of endeavor, teaches generating a third data segment based on the second data segment and the data frame (Page 5 Section 3.2 of Ding, " Denoting X,Y as foreground and background source clips, the synthetic clip
PNG
media_image1.png
25
260
media_image1.png
Greyscale
", Page 4 Figure 4 Caption, "We first randomly sample two clips from a video and use FAME to generate new clips by composing the original foreground onto various backgrounds from other videos"
Ding generates a new synthetic clip X_merge by compositing a clip X with content Y drawn from a different further video. Ding’s clip X corresponds to the second data segment and the cross-video content Y corresponds to the data frame content selected from the further data sequence that merges each other to form X_merge (third data segment).)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee's addition of a color-selected data frame to a clip with Ding’s compositing of a clip with content drawn from a different video so that the third data segment is generated from the second data segment obtained from a different data sequence and the data frame from a further data sequence in order to increase the appearance diversity of the constructed sample and improve the learned video representation (Section 4.3 of Ding).
Regarding claim 17,
Wang teaches implementing a method of generating a negative sample pair of a contrastive learning model for video processing, the method comprising (Page 2 Introduction of Wang, “Specifically, given a video, we randomly select a static frame and add it to every other frames to construct a distracting video, as is shown in Figure 3. Then we force the model to pull the feature of the distracting video and the feature of the original video together by consistency regularization. In this way, we made a disturbance to the video background and require its feature to be consistent with the original video…”, Page 4 Section 3.3.2 Contrastive Learning, “The hard negative set N2i for xi is defined as: N2i = {x_h i |x_h i ̸ = xo i, x_h i ∈ x_i}, and the overall negative set is Ni = {N1i ∪N2i}. Then the final objective function is: [Equation 7] For efficiency, we randomly select one hard negative sample from N2i each iteration and we find more hard negative samples have a similar result experimentally.”
Wang teaches a self-supervised contrastive learning framework in which a model is trained using positive sample pairs constructed from a video with its background-noise-augmentation, and negative sample pairs drawn from other videos.):
obtaining a first data segment from a first data sequence in a plurality of data sequences for training the contrastive learning model (Page 3 Section 3.1, “The framework of the proposed BE is shown in Figure 2. For each input video x, we first randomly crop two fixed length clips from different spatial locations, denoted as x_o and x_v.”, Page 4 Section 3.3.2, “Given a video dataset D with N videos D={x1,x2,...,xN}, for each video xi, we randomly sample once in each epoch, obtaining xo i and xd i.”
Wang teaches that each video in the training dataset is treated as one data sequence, and that fixed-length clips are cropped from each video as data segments. The original clip x_o corresponds to the first data segment drawn from the first data sequence (the first video) within the plurality of training videos.),
and obtaining a second data segment from a second data sequence in the plurality of data sequences, wherein the plurality of data sequences are a plurality of video sequences, and the first data segment and the second data segment are video segments ((Page 4 Section 3.3.2, “Given a video dataset D with N videos D={x1,x2,...,xN}, for each video xi, we randomly sample once in each epoch, obtaining xo i and xd i.… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity,”, Page 3 Section 3.1, “For each input video x, we first randomly crop two fixed length clips from different spatial locations, denoted as xo and xv.”, Page 3 Section 3.2, “for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video…”
Wang teaches the dataset D={x1,x2,…,xN} containing multiple videos (data sequences), where each video corresponds to a separate data sequence. From each video, Wang crops fixed length clips which correspond to the claimed data segments. The second data segment is a clip drawn from a different video (a clip s(xn)), which is the second data segment obtained from a second data sequence in the plurality of video sequences.)
selecting a data frame from a further data sequence than the second data sequence in the plurality of data sequences… (Page 3 Section 3.2, “Specifically, for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity…”, Page 6 Section 4.4, “Inter-Video Frame: randomly select one frame from another video, and add this static frame as noise to each frame of this video.”
Wang teaches using an Inter-Video Frame where a static frame is randomly selected from a video other than the video being modified. Under broadest reasonable interpretation, this static frame corresponds to a data frame selected from a further (i.e., additional) data sequence than the second data sequence in the plurality of data sequences)
and determining a negative sample pair for training the contrastive learning model based on the first data segment and the [clip of a further data sequence] (Page 4 Section 3.3.2, “Contrastive learning … aims to learn an invariant representation for each sample, which is achieved by maximizing similarity of similar pairs over dissimilar pairs… Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n= i}, each element in N1i is a clip and represents an identity… In order to make the objective more challenge, we introduce hard negatives, the different video clips… The hard negative set N2i for xi is defined as: N2i = {xh i |xh i = xo i,xh i ∈ xi}, and the overall negative set is Ni = {N1i ∪N2i}.”, Page 3 Fig. 2 Caption, “The model is trained by a existing self-supervised task together with a consistency constraint, with the goal of pulling the feature of the original video and that of the distracting video closer.”
Wang trains the contrastive learning model with an InfoNCE objective in which the anchor clip x_o (the first data segment) is contrasted against a negative set Ni. The clip drawn from a different video (s(xn)) is designated a negative and pushed apart from the anchor such that the anchor x_o together with the different video clip constitutes the negative sample pair (x_o, s(xn)).)
Wang does not teach [a] non-transitory computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor… selecting a [video] … in the plurality of data sequences based on a color range of the plurality of data sequences… generating a third data segment based on the second data segment and the data frame.
Lee, in the same field of endeavor, teaches [a] non-transitory computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor… (Paragraph 62 of Lee, “the example … may be implemented using coded instructions (e.g., computer readable instructions) stored on a tangible computer readable medium such as a hard disk drive, a flash memory, a read-only memory (ROM)… The tangible computer readable medium can be local to and/or remote from the processor(s), device(s), apparatus, etc., that is/are to execute the coded instructions… any type of computer readable storage and to exclude propagating signals… stored on a non-transitory computer readable medium”)
selecting a [video] … in the plurality of data sequences based on a color range of the plurality of data sequences (Paragraph 16 of Lee, "a color histogram for a video (e.g., corresponding to a video frame or a sequence of video frames from a video) includes a set of bins (also referred to herein as color bins) representing a respective set of possible colors that may be included in the video frame(s)", Paragraph 17, "obtaining a first color histogram corresponding to a sequence of frames of the first video ... and obtaining a second color histogram corresponding to a sequence of frames of the second video", Paragraph 52, "the comparison metric can be represented mathematically as the quantity ComparisonMetric and determined mathematically by summing the differences between the bin values of the baseline color histogram and the respective adjusted bin values of the trial color histogram", Paragraph 35, "the histogram comparator 150 may determine that the test video matches the particular reference video stored in the reference video library 105 if at least one of the comparison metrics quantifying comparison of the reference video with the test video is less than or equal to a threshold"
Lee builds for each video a color histogram over a sequence of frames that quantifies the colors present in that video. This per-video color histogram is the claimed color range of a data sequence. Lee then computes a comparison metric from the differences between two videos' color histograms and selects the video whose metric meets a threshold.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang’s selection of a static frame from a further video with Lee's technique of selecting a video by comparing color-histogram color ranges in order to enable a color-driven selection of the donor video sequence and improve contrastive learning performance (Paragraphs 43 and 36 of Lee).
Wang in view of Lee does not teach generating a third data segment based on the second data segment and the data frame.
Ding, in the same field of endeavor, teaches generating a third data segment based on the second data segment and the data frame (Page 5 Section 3.2 of Ding, " Denoting X,Y as foreground and background source clips, the synthetic clip
PNG
media_image1.png
25
260
media_image1.png
Greyscale
", Page 4 Figure 4 Caption, "We first randomly sample two clips from a video and use FAME to generate new clips by composing the original foreground onto various backgrounds from other videos"
Ding generates a new synthetic clip X_merge by compositing a clip X with content Y drawn from a different further video. Ding’s clip X corresponds to the second data segment and the cross-video content Y corresponds to the data frame content selected from the further data sequence that merges each other to form X_merge (third data segment).)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee's addition of a color-selected data frame to a clip with Ding’s compositing of a clip with content drawn from a different video so that the third data segment is generated from the second data segment obtained from a different data sequence and the data frame from a further data sequence in order to increase the appearance diversity of the constructed sample and improve the learned video representation (Section 4.3 of Ding).
Claim 18 recites similar limitations to claim 2. Therefore, claim 18 is rejected using the same rationale as claim 2.
Claims 3-5 and 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (“Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning”, 2021) in view of Lee (US 20130148884 A1), further in view of Ding (“Motion-aware Contrastive Video Representation Learning via Foreground-background Merging”, 2021), and in further view of Yun (“CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features”, 2019).
Regarding claim 3,
Wang teaches generating the noise data frame comprises (Page 3 Section 3.2, “…we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video, in which each frame ˆx is obtained by the following formula:
PNG
media_image3.png
33
361
media_image3.png
Greyscale
… k denotes the index of the randomly selected frame and T is the length of the video sequence x…”, Page 2 Introduction, “Specifically, given a video, we randomly select a static frame and add it to every other frames to construct a distracting video…”
Wang teaches that x^k is the randomly selected static frame (noise data frame) which supplies a data value at the same channel position (i.e., the corresponding data point) used in the linear combination of Equation 1.):
Wang further teaches generating a plurality of copied intermediate data frames by copying the intermediate data frame (Page 3 Section 3.2, “
PNG
media_image7.png
39
257
media_image7.png
Greyscale
where δ represents the result of repeating the selected frame x(k) T times along the time dimension.”
Wang teaches that the donor frame x^k is replicated T times. Each of the T replicas is a copy of the donor frame. When the dimension adjustment is applied to the donor frame first, each of the T replicas carries the adjusted dimensions.);
and generating the noise data frame by joining the plurality of copied intermediate data frames (Page 3 Section 3.2, “…where δ represents the result of repeating the selected frame x(k) T times along the time dimension.”
Wang teaches that the T replicas are joined along the time dimension to form δ.)
Wang in view of Lee further in view of Ding does not teach generating an intermediate data frame by adjusting dimensions of the data frame according to a predetermined ratio;
Yun, in the same field of endeavor, teaches generating an intermediate data frame by adjusting dimensions of the data frame according to a predetermined ratio (Page 3 Section 3.1, “Like Mixup [47], the combination ratio λ between two data points is sampled from the beta distribution Beta (α, α). In our all experiments, we set α to 1, that is λ is sampled from the uniform distribution (0,1) ... The box coordinates are uniformly sampled according to:
PNG
media_image8.png
46
276
media_image8.png
Greyscale
”
Yun teaches that an image region is sized by scaling the source image’s width W and height H by the ratio sqrt(1 - Lambda). The sized region functions as the intermediate data frame and the ratio functions as the predetermined ratio for adjusting the dimensions of the data frame.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee further in view of Ding’s technique of replicating and joining the donor frame to form the noise δ within a computing device environment with Yun’s technique of sizing an image patch by a ratio before use in order to generate a resized donor frame having dimensions suitable for Wang’s replication, joining, and per-frame blending process (Section 3.1 of Yun).
Regarding claim 4,
Wang in view of Lee further in view of Ding does not teach the dimensions of the data frame comprise at least one of a width and a height.
Yun, in the same field of endeavor, teaches the dimensions of the data frame comprise at least one of a width and a height (Page 3 Section 3.1, “To sample the binary mask M, we first sample the bounding box coordinates B = (rx,ry,rw,rh) indicating the cropping regions on xA and xB… The box coordinates are uniformly sampled according to:
PNG
media_image8.png
46
276
media_image8.png
Greyscale
”
Yun uses W (width) and H (height) as the dimensions of the data frame.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee further in view of Ding’s technique of replicating and joining the donor frame to form the noise δ within a computing device environment with Yun’s technique of sizing an image patch by a ratio before use in order to generate a resized donor frame having dimensions suitable for Wang’s replication, joining, and per-frame blending process (Section 3.1 of Yun).
Regarding claim 5,
Wang teaches in response to determining that a resolution of the noise data frame is different from that of the data frame in the second data segment, performing at least one of (Page 4 Section 3.3.2, “Contrastive learning [16] aims to learn an invariant representation for each sample, which is achieved by maximizing similarity of similar pairs over dissimilar pairs….Given a particular video xi and clip sampling function s, the negative set N1i is defined as: N1i = {s(xn)|∀n ̸ = i}, each element in N1i is a clip and represents an identity, then the InfoNCE[36] loss is improved as follows:
PNG
media_image9.png
60
319
media_image9.png
Greyscale
”, Page 3 Figure 2 Caption, “…the goal of pulling the feature of the original video and that of the distracting video closer. (Best viewed in color).”
Wang determines that the original clip (first data segment) and the distracting video (third data segment) form a negative pair for contrastive learning, as they are pushed apart in the embedding space.):
Wang in view of Lee further in view of Ding does not teach cropping the noise data frame to the resolution of the data frame in the second data segment; scaling the noise data frame to the resolution of the data frame in the second data segment.
Yun, in the same field of endeavor, teaches cropping the noise data frame to the resolution of the data frame in the second data segment (Page 3 Section 3.1, “Let x ∈ RW×H×C and y denote a training image and its label, respectively… we first sample the bounding box coordinates B = (rx,ry,rw,rh) indicating the cropping regions on xA and xB. The region B in xA is removed and filled in with the patch cropped from B of xB… The box coordinates are uniformly sampled according to:
PNG
media_image10.png
49
270
media_image10.png
Greyscale
”
Yun teaches a source image xB corresponding to the “noise data frame” and a destination image xA corresponding to the “data frame in the second data segment”, where a bounding box B derived from the dimensions of xA is applied to both images. The cropped region from xB matches the width and height of the corresponding region in xA.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee further in view of Ding’s technique of replicating and joining the donor frame to form the noise δ within a computing device environment with Yun’s technique of sizing an image patch by a ratio before use in order to generate a resized donor frame having dimensions suitable for Wang’s replication, joining, and per-frame blending process (Section 3.1 of Yun).
Claim 19 recites similar limitations to claim 3. Therefore, claim 19 is rejected using the same rationale as claim 3.
Claim 20 recites similar limitations to claim 4. Therefore, claim 20 is rejected using the same rationale as claim 4.
Claim 21 recites similar limitations to claim 5. Therefore, claim 21 is rejected using the same rationale as claim 5.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (“Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning”, 2021) in view of Lee (US 20130148884 A1), further in view of Ding (“Motion-aware Contrastive Video Representation Learning via Foreground-background Merging”, 2021), and further in view of Chen (“Shot Contrastive Self-Supervised Learning for Scene Boundary Detection”, 2021).
Regarding claim 7,
Wang in view of Lee further in view of Ding’s does not teach obtaining the first data segment comprises: selecting the first data segment satisfying a predetermined length from the first data sequence, the first data segment comprising only a single shot.
Chen, in the same field of endeavor, discloses obtaining the first data segment comprises: selecting the first data segment satisfying a predetermined length from the first data sequence, the first data segment comprising only a single shot (Page 1 Introduction, “A shot is defined as a series of frames captured from the same camera over an uninterrupted period of time [40], while a scene is defined as a series of shots depicting a semantically cohesive part of a story [23] (see Figure 1 for an illustration).”, Page 3 Section 3.1, “Given a full-length input video, we first use standard shot detection techniques [38] to divide it into its constituent set of shots.”, Page 3 Section 3.1.1, “We therefore begin by constructing a 4D tensor (w,h,c,k) from each shot with uniformly sampled k frames each with w width, h height and c color channels.”, Page 6 Section 4.2.1, “Based on this ablation study, for all our experiments we use 3 frames per shot.”
Chen discloses selecting a first data segment of a predetermined length from a first data sequence, where the segment contains only a single shot. Chen also defines a shot as a continuous sequence of frames captured by the same camera without interruption, divides a full -length video into individual shots using shot detection, and then constructs a fixed-length tensor by uniformly sampling k frames from each shot. In Chen’s experimentation, 3 frames per shot was used.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee further in view of Ding’s teaching with Chen’s technique of partitioning videos into single shots in order to improve temporal consistency in the sampled training segments and reduce scene-change noise that could interfere with the contrastive signal (Section 3.1 of Chen).
Regarding claim 8,
Wang in view of Lee further in view of Ding does not teach obtaining the second data segment comprises: selecting the second data segment satisfying the predetermined length from the second data sequence, the second data segment comprising only a single shot.
Chen, in the same field of endeavor, discloses obtaining the second data segment comprises: selecting the second data segment satisfying the predetermined length from the second data sequence, the second data segment comprising only a single shot (Page 3 Section 3.1, “Given a full-length input video, we first use standard shot detection techniques [38] to divide it into its constituent set of shots… We therefore begin by constructing a 4D tensor (w,h,c,k) from each shot with uniformly sampled k frames each with w width, h height and c color channels.”, Page 4 Section 3.1.2, “Along with K negative keys SK, the K+1 shots (k0 ∪ SK) are encoded by a key encoder to form a (K+1)-class classification task…”
Chen teaches dividing videos into individual shots and sampling a fixed number of frames from each shot to create a segment. Section 3.1.2 further shows that the negative key shots are also generated using the same process. A second data sequence from any full-length video in the dataset is processed into a second data segment that satisfied the predetermined length and only comprises one shot.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee further in view of Ding’s teachings with Chen’s technique of partitioning videos into single shots in order to improve temporal consistency in the sampled training segments and reduce scene-change noise that could interfere with the contrastive signal (Section 3.1 of Chen).
Claims 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (“Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning”, 2021) in view of Lee (US 20130148884 A1), further in view of Ding (“Motion-aware Contrastive Video Representation Learning via Foreground-background Merging”, 2021), and in further view of Huan (“RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning”, 2021).
Regarding claim 10,
Wang in view of Lee does not teach generating the third data segment further comprises
Ding, in the same field of endeavor, teaches generating the third data segment further comprises (Page 5 Section 3.2 of Ding, " Denoting X,Y as foreground and background source clips, the synthetic clip
PNG
media_image1.png
25
260
media_image1.png
Greyscale
", Page 4 Figure 4 Caption, "We first randomly sample two clips from a video and use FAME to generate new clips by composing the original foreground onto various backgrounds from other videos"
Ding generates a new synthetic clip X_merge by compositing a clip X with content Y drawn from a different further video. Ding’s clip X corresponds to the second data segment and the cross-video content Y corresponds to the data frame content selected from the further data sequence that merges each other to form X_merge (third data segment).)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee's addition of a color-selected data frame to a clip with Ding’s compositing of a clip with content drawn from a different video so that the third data segment is generated from the second data segment obtained from a different data sequence and the data frame from a further data sequence in order to increase the appearance diversity of the constructed sample and improve the learned video representation (Section 4.3 of Ding).
Wang in view of Lee further in view of Ding does not teach adjusting a sampling frequency of a plurality of data frames in the third data segment.
Huang, in the same field of endeavor, teaches adjusting a sampling frequency of a plurality of data frames in the third data segment (Page 5 Section 4, “The possible playback speed s for clips in this paper is set to 1x (i.e., sampling frames consecutively) and 2x (i.e., sampling interval is set 2 frames).”, Page 3 Section 3, “Let V={vi} N i =1 be a video set containing N videos. We sample a clip ci from a video with si playback speed.”
Huang teaches sampling clips at different playbook speeds varying the frame sampling interval. 1x corresponds to sampling frames consecutively, and 2x corresponds to setting the sampling interval to 2 frames. Huang’s sampling frequency adjustment is a per-clip operation that operates on the data frames of any video clip. When Huang’s sampling frequency adjustment is applied to the data frames of Wang’s third data segment, the data frames of the third data segment are resampled at the adjusted frequency.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee further in view of Ding’s third data segment generation within a computing device environment with Huang’s technique of the playback speed by adjusting the frame sampling interval of a video clip in order to enhance the motion-sensitivity of the learned representation by adding a pace dimension to the negative-sample construction (Sections 3.2 and 4 of Huang).
Regarding claim 11,
Wang in view of Lee further in view of Ding does not teach generating a fourth data segment by adjusting the sampling frequency of a plurality of data frames in the first data sequence; and determining a negative sample pair for training the contrastive learning model based on the first data segment and the fourth data segment.
Huang, in the same field of endeavor, teaches generating a fourth data segment by adjusting a sampling frequency of a plurality of data frames in the first data sequence (Page 4 Section 3.2, “Given a video, we sample 3 clips ci, cj and ck with playback speeds si, sj and sk, respectively, where si = sj= sk.”, Page 5 Section 4, “The possible playback speed s for clips in this paper is set to 1x (i.e., sampling frames consecutively) and 2x (i.e., sampling interval is set 2 frames).”
Huang teaches sampling a clip c_k at a different playback speed s_k from the same source video v as the anchor clip c_i. The clip c_k is drawn from the first data sequence (the same source video as the anchor c_i) at an adjusted sampling frequency s_k (fourth data segment).);
and determining a negative sample pair for training the contrastive learning model based on the first data segment and the fourth data segment (Page 4 Section 3.2, “As the clips with the same playback speed share similar motion features, we expect that their features can be closer than the clips with different playback speed s. We achieve this object by using a triplet loss…
PNG
media_image11.png
22
281
media_image11.png
Greyscale
where p+ = d(mi,mj), p− = d(mi,mk)… We desire that the similarity of a positive pair is larger than a negative pair by a margin γ.”
Huang’s relative speed perception triplet loss treats the speed different clip c_k as the negative term p- and the same speed clip c_j as the positive term p+. Huang teaches that clips with different playback speeds are treated as negative pairs during contrastive training. Thus, the original clip corresponds to the first data segment, the speed-adjusted clip corresponds to the fourth data segment, and the pair formed between them corresponds to the claimed negative sample pair used for training the contrastive learning model.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Wang in view of Lee further in view of Ding’s third data segment generation within a computing device environment with Huang’s technique of the playback speed by adjusting the frame sampling interval of a video clip in order to enhance the motion-sensitivity of the learned representation by adding a pace dimension to the negative-sample construction (Sections 3.2 and 4 of Huang).
Regarding claim 12,
Wang teaches generating a fifth data segment based on the fourth data segment and the data frame (Page 3 Section 3.2, “Specifically, for a video sequence x, we randomly select one static frame and add it as a spatial background noise to every other frames to generate a distracting video, in which each frame ˆx is obtained by the following formula:
PNG
media_image12.png
23
271
media_image12.png
Greyscale
”
Wang teaches generating a fifth data segment based on the fourth data segment and the data frame by adding a selected static frame as spatial background noise to each frame of a base video segment using Equation 1. The fourth data segment corresponds to the base segment, the selected static frame corresponds to the data frame, and the augmented clip corresponds to the fifth data segment.);
and determining a negative sample pair for training the contrastive learning model based on the first data segment and the fifth data segment (Page 4 Section 3.3.1, “For handcrafted pretext task, we use a consistency regularization term to pull the feature of xo closer to the feature of xd, and make them consistent in the temporal dimension.”, Page 4 Section 3.3.2, “The hard negative set N2i for xi is defined as: N2i = {xh i |xh i ̸ = xo i,xh i ∈ xi}, and the overall negative set is Ni = {N1i ∪N2i}. Then the final objective function is:
PNG
media_image13.png
56
322
media_image13.png
Greyscale
”
Wang further teaches determining a negative sample pair for training the contrastive learning model based on the first data segment and the fifth data segment through its contrastive learning objective, which pushes augmented clips away from the original clip in the latent space. The first data segment corresponds to the anchor/original clip, the fifth data segment corresponds to the augmented clip, and the pair formed between them corresponds to the claimed negative sample pair.).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAJD MAHER HADDAD whose telephone number is (571)272-2265. The examiner can normally be reached Mon-Friday 8-5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.M.H./Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
Prosecution Insights