DETAILED ACTION
This Office Action is sent in response to the Applicant’s Communication received on 12/21/2022 for application number 18,069,974. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, IDS, and Claims.
Claims 1-32 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
This application includes one or more claim limitations that use the word “means,” and being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses the word “means” that is coupled with functional language without reciting sufficient structure to perform the recited function. Such claim limitations are:
“means for processing a frame of video data using…” in claim 32,
“means for applying an exponential-family prior…” in claim 32,
“means for generating a total loss value…” in claim 32, and
“means for training the neural network-based video encoder…” in claim 32
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. Support for the above “means for” can be found in paragraphs [0087, 0106-0107, 0140 – 0162], Figures 8, 9 and 10.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 10, 11, 19, 28, and 32 are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al. (KR20220098383A, see attached translation), hereinafter Jiang, in view of Dai et al. (CN114584780A, see attached translation), hereinafter Dai.
Regarding claim 1, Jiang teaches,
An apparatus for processing video data [Para 0001, Method and device for end-to-end neural compression using deep reinforcement learning], the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory [Para 0007, a device for end-to-end neural image compression using deep reinforcement learning includes at least one memory configured to store program code and at least one processor configured to read the program code and operate as instructed by the program code] and configured to: process a frame of video data using a first layer of a neural network-based video encoder (Para 0062, DNN layer), the neural network-based video encoder performing at least one quantization step [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames; Para 0062, the DNN is used as a state predictor that acts as a function approximator to estimate the action-value mapping function… A state predictor DNN typically consists of a set of convolutional layers followed by one or more fully connected layers.];
generate a total loss value for the neural network-based video encoder (Para 0088, the slope of the R-D loss) based on a sum of a loss value for the neural network-based video encoder (Para 0088, summing the slopes of the R-D loss for multiple input signals) and first layer output evaluation (Para 0088, slopes of the R-D loss for multiple input signals) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder];
and train the neural network-based video encoder (Para 0088, update the weight parameters of the DNN) based on the total loss value (Para 0088, slope of the R-D loss) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder].
Jiang teaches the above limitations of claim 1 including the first layer of the neural network-based video encoder.
Jiang does not teach apply an exponential-family prior to an output of neural network to generate output evaluation.
Dai teaches,
apply an exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output (Para 0010, feature map) to generate output evaluation (Para 0010, to generate a feature binary code stream) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Regarding claim 10, Jiang-Dai teach the limitations of claim 1.
Dai further teaches,
wherein the exponential-family prior comprises at least one of a Gaussian function (Para 0010, mixed Gaussian distribution encoding feature), a Laplacian function, or a univariate exponential-family prior (UEP) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream].
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Regarding claim 11, Jiang-Dai teach the limitations of claim 1.
Jiang further teaches,
wherein the loss value is based on a rate distortion loss function [Para 0042, Rate-Distortion (R-D) loss is calculated].
Regarding claim 19, Jiang teaches,
A method of processing video data [Para 0001, Method and device for end-to-end neural compression using deep reinforcement learning], the method comprising: processing a frame of video data using a first layer of a neural network-based video encoder (Para 0062, DNN layer), the neural network-based video encoder performing at least one quantization step [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames; Para 0062, the DNN is used as a state predictor that acts as a function approximator to estimate the action-value mapping function… A state predictor DNN typically consists of a set of convolutional layers followed by one or more fully connected layers.];
generating a total loss value for the neural network-based video encoder (Para 0088, the slope of the R-D loss) based on a sum of a loss value for the neural network-based video encoder (Para 0088, summing the slopes of the R-D loss for multiple input signals) and first layer output evaluation (Para 0088, slopes of the R-D loss for multiple input signals) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder];
and training the neural network-based video encoder (Para 0088, update the weight parameters of the DNN) based on the total loss value (Para 0088, slope of the R-D loss) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder].
Jiang teaches the above limitations of claim 19 including the first layer of the neural network-based video encoder.
Jiang does not teach applying an exponential-family prior to an output of neural network to generate output evaluation.
Dai teaches,
applying an exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output (Para 0010, feature map) to generate a first layer output evaluation (Para 0010, to generate a feature binary code stream) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Claim 28 is a method claim that recites similar limitations to claim 10. Therefore, claim 28 is rejected using the same rationale as claim 10.
Regarding claim 32, Jiang teaches,
An apparatus [Para 0001, Method and device for end-to-end neural compression using deep reinforcement learning] comprising: means for processing a frame of video data using a first layer of a neural network-based video encoder (Para 0062, DNN layer), the neural network-based video encoder performing at least one quantization step [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames; Para 0062, the DNN is used as a state predictor that acts as a function approximator to estimate the action-value mapping function… A state predictor DNN typically consists of a set of convolutional layers followed by one or more fully connected layers.];
Means for generating a total loss value for the neural network-based video encoder (Para 0088, the slope of the R-D loss) based on a sum of a loss value for the neural network-based video encoder (Para 0088, summing the slopes of the R-D loss for multiple input signals) and first layer output evaluation (Para 0088, slopes of the R-D loss for multiple input signals) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder];
and means for training the neural network-based video encoder (Para 0088, update the weight parameters of the DNN) based on the total loss value (Para 0088, slope of the R-D loss) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder].
Jiang teaches the above limitations of claim 32 including the first layer of the neural network-based video encoder.
Jiang does not teach means for applying an exponential-family prior to an output of neural network to generate a output evaluation.
Dai teaches,
Means for applying an exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output of neural network (Para 0010, feature map) to generate a output evaluation (Para 0010, to generate a feature binary code stream) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Claim(s) 2, 12, 13, 15, 20, 29, and 31 are rejected under 35 U.S.C. 103 as being unpatentable over Jiang in view of Dai and in further view of Yao et al. (US 20240177269 A1), hereinafter Yao.
Regarding claim 2, Jiang-Dai teach the limitations of claim 1 including the first layer output evaluation and the total loss value.
Jiang-Dai do not teach apply a negative log likelihood to output; and generate total loss value by summing loss value and the negative log likelihood of the output.
Yao teaches,
apply a negative log likelihood to output [Para 0034, LINF employs a two-stage training scheme. In the first stage, it is trained only with the negative log-likelihood loss L.sub.nll.];
and generate total loss value (Para 0034, total loss function L) by summing loss value (Para 0034, L.sub.pixel) and the negative log likelihood of the output (Para 0034, L.sub.nll) [Para 0034, LINF employs a two-stage training scheme. In the first stage, it is trained only with the negative log-likelihood loss L.sub.nll. In the second stage, it is fine-tuned with an additional L1 loss on predicted pixels L.sub.pixel, and the VGG perceptual loss on the patches predicted by the flow model L.sub.vgg. The total loss function L can be formulated as follows:
PNG
media_image1.png
99
480
media_image1.png
Greyscale
].
Yao is analogous to the claimed invention as they both relate to image processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang and Dai’s teachings to incorporate the teachings of Yao and provide a loss function comprising negative log likelihood in order to optimize the model’s output by handling layer-wise loss results.
Regarding claim 12, Jiang teaches,
An apparatus for processing video data [Para 0001, Method and device for end-to-end neural compression using deep reinforcement learning], the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory [Para 0007, a device for end-to-end neural image compression using deep reinforcement learning includes at least one memory configured to store program code and at least one processor configured to read the program code and operate as instructed by the program code] and configured to: process a frame of video data using a first layer of a neural network-based video encoder (Para 0062, DNN layer), the neural network-based video encoder performing at least one quantization step [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames];
and train the neural network-based video encoder (Para 0088, update the weight parameters of the DNN) based on the total loss value (Para 0088, slope of the R-D loss) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder].
Jiang teaches the above limitations of claim 12 including the first layer of the neural network-based video encoder and the neural network-based video encoder.
Jiang does not teach apply an exponential-family prior to an output to generate a output evaluation; generate a constraint based on a negative log likelihood output evaluation and a multiplier; generate a total loss value based on a sum of a loss value and the constraint.
Dai teaches,
apply an exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output (Para 0010, feature map) to generate a output evaluation (Para 0010, to generate a feature binary code stream) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Jiang-Dai do not teach generate a constraint based on a negative log likelihood output evaluation and a multiplier; generate a total loss value based on a sum of a loss value and the constraint.
Yao teaches
generate a constraint (Eq 10,
λ
1
L
n
l
l
) based on a negative log likelihood output evaluation (Eq 10,
L
n
l
l
) and a multiplier (Eq 10,
λ
1
)
[Para 0034,
PNG
media_image2.png
127
590
media_image2.png
Greyscale
where λ.sub.1, λ.sub.2, and λ.sub.3 are the scaling parameters];
generate a total loss value based on a sum of a loss value and the constraint
[Para 0034,
PNG
media_image2.png
127
590
media_image2.png
Greyscale
where λ.sub.1, λ.sub.2, and λ.sub.3 are the scaling parameters.
Yao is analogous to the claimed invention as they both relate to image processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang and Dai’s teachings to incorporate the teachings of Yao and provide a constraint for the loss function in order to tune scaling parameters to a desired output.
Claim 13 is an apparatus claim that recites similar limitations to claim 2. Therefore, claim 13 is rejected using the same rationale as claim 2.
Regarding claim 15, Jiang-Dai-Yao teach the limitations of claim 12.
Dai further teaches,
wherein the exponential-family prior comprises at least one of a Gaussian function [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream].
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang and Yao’s teachings to incorporate the teachings of Dai and provide a Gaussian function for its flexibility of modeling complex data.
Claim 20 is a method claim that recites identical limitations to claim 2. Therefore, claim 20 is rejected using the same rationale as claim 2.
Regarding claim 29, Jiang teaches,
A method for processing video data [Para 0001, Method and device for end-to-end neural compression using deep reinforcement learning], the method comprising: processing a frame of video data using a first layer of a neural network-based video encoder (Para 0062, DNN layer), the neural network-based video encoder performing at least one quantization step [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames; Para 0062, the DNN is used as a state predictor that acts as a function approximator to estimate the action-value mapping function… A state predictor DNN typically consists of a set of convolutional layers followed by one or more fully connected layers.];
and training the neural network-based video encoder (Para 0088, update the weight parameters of the DNN) based on the total loss value (Para 0088, slope of the R-D loss) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder].
Jiang teaches the above limitations of claim 29 including the first layer of the neural network-based video encoder and the neural network-based video encoder.
Jiang does not teach applying an exponential-family prior to an output to generate a output evaluation; generating a constraint based on a negative log likelihood output evaluation and a multiplier; generating a total loss value based on a sum of a loss value and the constraint.
Dai teaches,
applying an exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output (Para 0010, feature map) to generate a output evaluation (Para 0010, to generate a feature binary code stream) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Jiang-Dai do not teach generating a constraint based on a negative log likelihood output evaluation and a multiplier; generating a total loss value based on a sum of a loss value and the constraint.
Yao teaches
generating a constraint (Eq 10,
λ
1
L
n
l
l
) based on a negative log likelihood output evaluation (Eq 10,
L
n
l
l
) and a multiplier (Eq 10,
λ
1
)
[Para 0034,
PNG
media_image2.png
127
590
media_image2.png
Greyscale
where λ.sub.1, λ.sub.2, and λ.sub.3 are the scaling parameters];
generating a total loss value based on a sum of a loss value and the constraint
[Para 0034,
PNG
media_image2.png
127
590
media_image2.png
Greyscale
where λ.sub.1, λ.sub.2, and λ.sub.3 are the scaling parameters.
Yao is analogous to the claimed invention as they both relate to image processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang and Dai’s teachings to incorporate the teachings of Yao and provide a constraint for the loss function in order to tune scaling parameters to a desired output.
Regarding claim 31, Jiang teaches,
A non-transitory computer-readable medium (Para 0112, non-transitory computer-readable medium) having stored thereon instructions (Para 0007, program code) that, when executed by at least one processor (Para 0007, at least one processor) [Para 0007, a device for end-to-end neural image compression using deep reinforcement learning includes at least one memory configured to store program code and at least one processor configured to read the program code and operate as instructed by the program code; Para 0112, one or more processors execute a program stored on a non-transitory computer-readable medium], cause the at least one processor to: process a frame of video data using a first layer of a neural network-based video encoder (Para 0062, DNN layer), the neural network-based video encoder performing at least one quantization step [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames; Para 0062, the DNN is used as a state predictor that acts as a function approximator to estimate the action-value mapping function… A state predictor DNN typically consists of a set of convolutional layers followed by one or more fully connected layers.];
and train the neural network-based video encoder (Para 0088, update the weight parameters of the DNN) based on the total loss value (Para 0088, slope of the R-D loss) [Para 0088, the NC weight update module (970) calculates the slope of the R-D loss (e.g., by summing the slopes of the R-D loss for multiple input signals), which can be used via backpropagation to update the weight parameters of the DNN encoder and the DNN decoder].
Jiang teaches the above limitations of claim 31 including the first layer of the neural network-based video encoder and the neural network-based video encoder.
Jiang does not teach apply an exponential-family prior to an output to generate a output evaluation; generate a constraint based on a negative log likelihood output evaluation and a multiplier; generate a total loss value based on a sum of a loss value and the constraint.
Dai teaches,
apply an exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output (Para 0010, feature map) to generate a output evaluation (Para 0010, to generate a feature binary code stream) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Jiang-Dai do not teach generate a constraint based on a negative log likelihood output evaluation and a multiplier; generate a total loss value based on a sum of a loss value and the constraint.
Yao teaches
generate a constraint (Eq 10,
λ
1
L
n
l
l
) based on a negative log likelihood output evaluation (Eq 10,
L
n
l
l
) and a multiplier (Eq 10,
λ
1
)
[Para 0034,
PNG
media_image2.png
127
590
media_image2.png
Greyscale
where λ.sub.1, λ.sub.2, and λ.sub.3 are the scaling parameters];
generate a total loss value based on a sum of a loss value and the constraint
[Para 0034,
PNG
media_image2.png
127
590
media_image2.png
Greyscale
where λ.sub.1, λ.sub.2, and λ.sub.3 are the scaling parameters.
Yao is analogous to the claimed invention as they both relate to image processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang and Dai’s teachings to incorporate the teachings of Yao and provide a constraint for the loss function in order to tune scaling parameters to a desired output.
Claim(s) 3-5, 14, 21-23, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Jiang in view of Dai and Yao, and in further view of Mathur et al. (US 11841892 B2), hereinafter Mathur.
Regarding claim 3, Jiang-Dai-Yao teach the limitations of claims 1 and 2 including the first layer.
Jiang further teaches,
process the frame of video data using a second layer of the neural network-based video encoder (Para 0062, DNN layer) [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames; Para 0062, the DNN is used as a state predictor that acts as a function approximator to estimate the action-value mapping function… A state predictor DNN typically consists of a set of convolutional layers followed by one or more fully connected layers.];
Jiang teaches the above limitations of claim 3 including the second layer of the neural network-based video encoder.
Jiang does not teach apply the exponential-family prior to an output to generate a output evaluation and sum a negative log likelihood output evaluation and a negative log likelihood output evaluation.
Dai further teaches,
apply the exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output to generate a output evaluation (Para 0010, for each channel of the second feature map) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Jiang-Dai-Yao teach the above limitations of claim 3 including output evaluation.
Jiang-Dai-Yao do not teach sum a negative log likelihood output evaluation and a negative log likelihood output evaluation.
Mathur teaches,
sum a negative log likelihood and a negative log likelihood output evaluation [Col 4, lines 19-30, One model is:
PNG
media_image3.png
43
336
media_image3.png
Greyscale
where the total loss term
L
is the sum of the skipgram negative sampling loss
L
i
j
n
e
g
with the addition of a Dirichlet-likelihood term over document weights
L
d
].
Mathur is analogous to the claimed invention as they both relate to image processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang, Dai, and Yao’s teachings to incorporate the teachings of Mathur and provide summing the negative log likelihoods in order to improve model robustness by analyzing granular losses.
Regarding claim 4, Jiang-Dai-Yao-Mathur teach the limitations of claims 1-3 including the at least one processor (see claim 1), the summed negative log likelihood (see claim 3), the first layer output evaluation (see claim 1), and the second layer output evaluation (see claim 3).
Yao further teaches,
multiply negative log likelihood and negative log likelihood with a multiplier value (Para 0034, λ.sub.1)
[Para 0034,
PNG
media_image2.png
127
590
media_image2.png
Greyscale
where λ.sub.1, λ.sub.2, and λ.sub.3 are the scaling parameters].
Yao is analogous to the claimed invention as they both relate to image processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang, Dai, and Mathur’s teachings to incorporate the teachings of Yao and provide a coefficient that scales loss values in order to achieve a desired accuracy of model results.
Regarding claim 5, Jiang-Dai-Yao-Mathur teach the limitations of claims 1-4 including the at least one processor (see claim 1), the summed negative log likelihood (see claim 3), the first layer output evaluation (see claim 1), the second layer output evaluation (see claim 3), and the negative log likelihood multiplied by the multiplier value (see claim 4).
Mathur further teaches,
generate the total loss value (Col 4, lines 19-30, total loss term
L
) by summing the loss value (Col 4, lines 19-30,
L
d
) with the summed negative log likelihood (Col 4, lines 19-30,
∑
L
i
j
n
e
g
) and the negative log likelihood (Col 4, lines 19-30,
L
i
j
n
e
g
)
[Col 4, lines 19-30, One model is:
PNG
media_image3.png
43
336
media_image3.png
Greyscale
where the total loss term
L
is the sum of the skipgram negative sampling loss
L
i
j
n
e
g
with the addition of a Dirichlet-likelihood term over document weights
L
d
].
Mathur is analogous to the claimed invention as they both relate to image processing. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang, Dai, and Yao’s teachings to incorporate the teachings of Mathur and provide a total loss value deriving from the summation of a loss value, summed negative log likelihood and negative log likelihood in order to enhance learning by analyzing a plurality of loss function results.
Regarding claim 14, Jiang-Dai-Yao teach the limitations of claims 12 including the first layer.
Jiang further teaches,
process the frame of video data using a second layer of the neural network-based video encoder (Para 0062, DNN layer) [Para 0004, Given an input image or video sequence x, based on the input x, the DNN encoder computes a compact representation y that is quantized into a discrete quantized representation; Para 0041, Let value=[35]]]], be the sequence of input signals to be compressed, where signal… can be an image, a patch within an image, a video segment, a patch within a video segment, etc… generally represented as a video sequence containing d image frames; Para 0062, the DNN is used as a state predictor that acts as a function approximator to estimate the action-value mapping function… A state predictor DNN typically consists of a set of convolutional layers followed by one or more fully connected layers.];
determine the loss value for the neural network-based video encoder [Para 0042, Rate-Distortion (R-D) loss is calculated];
Jiang teaches the above limitations of claim 14 including the second layer of the neural network-based video encoder.
Jiang does not teach apply the exponential-family prior to an output to generate a output evaluation and sum negative log likelihood output evaluation and a negative log likelihood output evaluation, wherein the constraint is generated by multiplying the multiplier with summed negative log likelihood output evaluation and the negative log likelihood output evaluation.
Dai further teaches,
apply the exponential-family prior (Para 0010, Gaussian distributions is weightedly combined) to an output to generate a output evaluation (Para 0010, for each channel of the second feature map) [Para 0010, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream];
Dai is analogous to the claimed invention as they both relate to utilizing machine learning in image compression. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang’s teachings to incorporate the teachings of Dai and provide an exponential-family prior [Dai, Para 0007] in order to improve rate-distortion performance.
Jiang-Dai do not teach sum negative log likelihood output evaluation and a negative log likelihood output evaluation, wherein the constraint is generated by multiplying the multiplier with the summed negative log likelihood output evaluation and the negative log likelihood output evaluation.
Yao teaches,
wherein the constraint (Para 0034, negative log-likelihood loss L.sub.nll. In the second stage) is generated by multiplying the multiplier (Para 0034, λ.sub.1) with the summed negative log likelihood output evaluation (Para 0034, negative log-likelihood loss L.sub.nll. In the second stage… fine-tuned with an additional L1 loss on predicted pixels L.sub.pixel) and the negative log likelihood output evaluation (Para 0034, first stage… negative log-likelihood loss L.sub.nll) [Para 0034, LINF employs a two-stage training scheme. In the first stage, it is trained only with th