DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 10/31/2025 has been entered.
Response To Arguments
Remarks page 13-15, Applicant contends:
Woo fails to teach "the generating the set of dequantized numbers using at least one of a block-wise dequantization method, an individual dequantization method, and a static dequantization model method" as Woo fails to disclose dequantization using block-wise dequantization.
Response:
Woo, alone, may fail to explicitly teach the generating of dequantized numbers utilizing a blockwise method, but Woo was further in combination of primary reference Lu (one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986)). Lu teaches quantization and dequantization in the claim rejection for claim 1. Lu Figure 1(b), which was used in rejection of claim 1, shows that quantization and dequantization are modeled as the same thing but in reverse. The current specification also notes that blockwise is a form of ordering ([Current Application 0061]: "The numbers may also be organized in different orders. For example, the number may be organized block-wise to preserve the relative location information."). Thus performing blockwise dequantization would be obvious to one of ordinary skill in the art without undue experimentation given Lu and Woo, as the elements required to perform blockwise dequantization seem to be taught. No specifics in the specification appear to indicate any special requirements beyond utilizing blockwise ordering or features in the existing quantization/dequantization.
As a result, the rolling up of limitations from claim 5 into claim 1 to overcome prior art is not seen as persuasive, thus the 103 rejections are sustained.
Remarks page 16, Applicant contends:
New claim 21 patentably distinguishes over the cited art.
Response:
Applicant’s argument is considered, but is rendered moot by the current rejection for claim 21 containing elements or references not previously recited. Claim 21 invokes generating quantization keys using individual quantization or static quantization, then generating, using dequantization, numbers using an individual dequantization or static quantization. Current closest claim is claim 6, which notes the use of a corresponding dequantization for the used quantization method, but claim 6 does not specifically invoke the generation of the keys using the quantization methods, which the current rejections rejected utilizing the blockwise by Woo in claim 5, thus more search and consideration was necessary to provide a proper rejection.
Claim Objections
Claim 21 objected to because of the following informalities: Claim 21 recites “generating the set of dequantized numbers using at least one of an individual dequantization method, and a static dequantization method” where the comma in “method, and” is seen as a typo. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 8-11, 15-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lu et al (“Learning A Dep Vector Quantization Network for Image Compression”), referred to as Lu in this document, and further in view over Singh et al (“End to End Learning of Compressible Features”), referred to as Singh in this document, and further in view over Woo et al (“Optimal blockwise dependent quantization for stereo image coding”), referred to as Woo in this document, and further in view over Fernandez et al (“VQQL. Applying Vector Quantization to Reinforcement Learning”), referred to as Fernandez in this document.
Regarding Claim 1:
Lu teaches:
the method being performed by at least one processor, and the method comprising
[Lu Experimental Results page 10]: “Note that JPEG, JPEG 2000 and BPG methods were implemented using the ffmpeg and libbpg softwares running on a CPU [the method being performed by at least one processor, and the method comprising], while Rippel's method [18] was implemented on a GTX 980Ti GPU.”
Further support for other computer parts is given by the application being directed for the use in computer tasks [Lu Introduction page 1]: "Image compression, aiming to represent an image with as little bits as possible, plays a key role in image communication and computer vision applications... Recently, inspired by the great success of the deep convolutional neural network (DCNN) for computer vision tasks...". One of ordinary skill in the art is considered knowing the use of computer parts, such as processor, memory, and computer readable mediums are used in computer tasks.
the plurality of latent representations being received as encoded using a second neural network
([Lu II. Related Works B. Deep Learning Based Methods page 2]: “The basic idea of these methods is to first map an input image to a latent representation through an autoencoder [the plurality of latent representations being received as encoded using a second neural network], quantize the feature maps, followed by entropy encoding, and reconstruct the input by a decoder.”)
generating a set of quantization keys, using a third neural network
([Lu III. Proposed VQNET for Image Compression B. Proposed VQNET page 4]: “Hence, the quantization centers C can be jointly optimized with other network parameters [generating a set of quantization keys, using a third neural network]. ”)
Details on the teachings of quantization keys is shown in the mapping related to Woo. Further emphasis the quantization is part of a neural network, thus being able to be the third neural network, is shown in Figure 1 of Lu.
[Lu Figure 1 page 3]
PNG
media_image1.png
442
1349
media_image1.png
Greyscale
generating a set of dequantized numbers representing dequantized representations of the encoded plurality of latent representations, based on the set of quantization keys, using a fourth neural network
([Lu III. Proposed VQNET for Image Compression B. Proposed VQNET page 4]: “Note that the de-quantization [generating a set of dequantized numbers representing dequantized representations of the encoded plurality of latent representations, based on the set of quantization keys,] can also be computed as z^j=Cbj , which can also be implemented as a fully connected layer [using a fourth neural network] by setting the layer parameter matrix as Wr=C .”)
The latent representation in this case is simply the input to the model. The input being a latent representation is shown in Singh, as the input to the encoder network will be shown to be a latent representation.
generating a reconstructed image associated with the input image, based on the set of dequantized numbers
([Lu III. Proposed VQNET for Image Compression A. Overall Network Architecture page 3] “The decoder D(⋅) reconstructs [generating a reconstructed image associated with the input image, based on the set of dequantized numbers] the original image from the de-quantized features x^=D(Z^) , where Z^ denotes the recovered feature representation decoded from the bitstream.”)
wherein the first neural network, the second neural network, the third neural network, the fourth neural network, and the fifth neural network are jointly trained using a combination
a training latent representation distortion loss that is based on error introduced by latent representations
[Lu III Proposed VQNET for Image Compression C. Joint Optimization page 18]: "To achieve good compression performance, the encoder, quantizer and the decoder should be jointly optimized [wherein the first neural network, the second neural network, the third neural network, the fourth neural network, and the fifth neural network are jointly trained using a combination as Lu is teaching that good performance comes from jointly training elements together] via minimizing a rate-distortion objective function, as shown in Eq. (1). The distortion loss [a training latent representation distortion loss that is based on error introduced by latent representations] is used to ensure the accuracy of the decompressed image. The commonly used distortion loss is the mean square loss, i.e., L(^x; x) = ||^x -x||^2_2. In addition, other perceptual loss, e.g., the SSIM metric of [25] can also be used for better perceptual quality. The bit-rate loss for an input image x can be measured by the entropy of the quantized feature vectors..."
Noted here for context, the combination with Singh is done later in support of this mapping, as Singh notes jointly training that even involves the task models [Singh Introduction page 1]: “Our method jointly optimizes for the original training objective as well as compressibility, yielding features that are as powerful but only require a fraction of the storage cost.”
and a training latent compression rate loss that is based on density estimation of quantized representations
[Lu 3 Proposed VQnet for Image Compression A. Overall Network Architecture page 3]: “To achieve good compression performance, we try to optimize E(∙), Q(∙) and D(∙) such that both the distortion ℓ(x; ^x) and the bit-rates [and a training latent compression rate loss that is based on density estimation of quantized representations] can be minimized…”
Lu does not explicitly teach:
receiving a target image detection task or a target image segmentation task to be performed on an input image (Lu, as noted in quotes used for claim 1, does teach being related to computer vision tasks. However, for clarity of the rejection, another reference is used to ensure no confusion that the elements related to having tasks be performed involving the compression system are taught.)
receiving a plurality of latent representations of the input image the plurality of latent representations being generated using a first neural network, wherein the plurality of latent representations comprise a sequence of latent signals,
performing the target image detection task or the target image segmentation task on the reconstructed image using a fifth neural network
based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to the plurality of latent representations
A method of end-to-end task oriented latent image compression using deep reinforcement learning
of a training task prediction loss that is based on training labels
the generating the set of quantization keys using at least one of a block-wise quantization method, an individual quantization method, and a static quantization model method
the generating the set of dequantized numbers using at least one of a block-wise dequantization method, an individual dequantization method, and a static dequantization model method
Singh teaches:
receiving a plurality of latent representations of the input image the plurality of latent representations being generated using a first neural network
performing the target image detection task or the target image segmentation task on the reconstructed image using a fifth neural network ([Singh Introduction page 1]: “Razavian et al. [10] demonstrated that a pre-trained network as a feature generator [receiving a plurality of latent representations of the input image the plurality of latent representations being generated using a first neural network], coupled with a simple classifier such as a SVM [performing the target image detection task or the target image segmentation task on the reconstructed image using a fifth neural network] or logistic regression, performs surprisingly well and often outperforms hand tuned features on a variety of tasks.”)
wherein the plurality of latent representations comprise a sequence of latent signals
([Singh Introduction page 1]: “Features also need to be pre-computed and stored for certain types of applications where the target task evolves over time. A typical example is an indexing system or a content analysis system where the image features may be one of the many signals [wherein the plurality of latent representations comprise a sequence of latent signals] that the full model relies on.”)
receiving a target image detection task or a target image segmentation task to be performed on an input image
[Singh Abstract page 1]: “We propose a learned method that jointly optimizes for compressibility along with the task objective [receiving a target image detection task or a target image segmentation task to be performed on an input image where aspects related to images are noted earlier such as in Sing Introduction page 1: “A typical example is an indexing system or a content analysis system where the image features may be one of the many signals”] for learning the features.”
of a training task prediction loss that is based on training labels
[Singh 4 Related Work page 4]: "Joint optimization for compression: Torfason et al. [28] propose to extend an auto-encoding compression network by adding an additional inference branch over the bottleneck for auxiliary tasks. Our method does not use any auto-encoding penalty and directly optimizes for the entropy along with the task specific objective [of a training task prediction loss that is]."
[Sing 2 Compressible Feature Learning page 2]: “where x is the input variable (e.g., image pixels, or features), y the target variable (e.g., classification labels [based on training labels], or regression target), ^y is the prediction, f is often an artificial neural network (ANN) with parameters θ comprising its set of filter weights, and D is a set of training data.”
One of ordinary skill, prior to the effective filing date would have been motivated to combined Lu and Singh to incorporate generating latent representations to perform tasks with. Lu and Singh are of the same field of endeavor of machine learning. One of ordinary skill would have been motivated to combine Lu and Sing as a task model using data from a feature selection or latent representation model is known to match or outperform handmade feature selection ([Singh Introduction page 1]: “Razavian et al. [10] demonstrated that a pre-trained network as a feature generator or logistic regression, performs surprisingly well and often outperforms hand tuned features on a variety of tasks.”). The use of a sequence of latent signals is motivated from the common reliance of such information for neural network models ([Singh Introduction page 1]: “Features also need to be pre-computed and stored for certain types of applications where the target task evolves over time. A typical example is an indexing system or a content analysis system where the image features may be one of the many signals that the full model relies on.”).
Woo teaches:
based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to the plurality of latent representations
([Woo II. Dependent Bit Allocation B. Optimal Blockwise Dependent Quantization page 3]: “where q^n1 is a vector that contains the quantizer indexes [based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to the plurality of latent representations] of those blocks in F1 that are used to predict the current block in F2.”)
The latent representations are taught in Singh. The quantization index is seen as being the quantization key. Quantization state is seen as referring to previous inputs into the quantizer that were encrypted by a quantization key, thus is referring to a combination of key and input (which is noted by “of those blocks in F1 that are used to predict the current block in F2”).
the generating the set of quantization keys using at least one of a block-wise quantization method, an individual quantization method, and a static quantization model method
the generating the set of dequantized numbers using at least one of a block-wise dequantization method, an individual dequantization method, and a static dequantization model method
([Woo Introduction page 1]: “The main novelty of our work is the introduction of an algorithm for optimal blockwise [the generating the set of quantization keys using at least one of a block-wise quantization method, an individual quantization method, and a static quantization model method][the generating the set of dequantized numbers using at least one of a block-wise dequantization method, an individual dequantization method, and a static dequantization model method] dependent bit allocation for stereo image coding”)
Earlier elements the teaching dequantized numbers by Lu and quantization keys, where dequantization involving quantization keys is noted in more detail by Woo ([Woo II. Dependent Bit Allocation A. Definitions and Notations page 2]) in claim 4. The method being blockwise is taught in this quote by Woo.
One of ordinary skill, prior to the effective filing date would have been motivated to combined Lu and Woo to incorporate blockwise dependent quantization. Lu and Woo are of the same field of endeavor of machine learning. One of ordinary skill would have been motivated to combine Lu and Woo in order to incorporate the use of blockwise dependent quantization as the method can produce optimal results ([Woo Abstract page 1]: “In this paper, we address the problem of blockwise bit allocation for coding of stereo images and show how, given the special characteristics of the disparity field, one can achieve an optimal solution with reasonable complexity, whereas in similar problems in motion compensated video only approximate solutions are feasible”).
Fernandez teaches:
A method of end-to-end task oriented latent image compression using deep reinforcement learning
([Fernandez 2 Reinforcement Learning page 2]: “The main objective of reinforcement learning [A method of end-to-end task oriented latent image compression using deep reinforcement learning] is to automatically acquire knowledge to better decide what action an agent should perform at any moment to optimally achieve a goal. Among many different reinforcement learning techniques, Q-learning has been very widely used [18].”)
One of ordinary skill, prior to the effective filing date would have been motivated to combined Lu and Fernandez to incorporate reinforcement learning. Lu and Fernandez are of the same field of endeavor of machine learning. One of ordinary skill would have been motivated to combine Lu and Fernandez in order to incorporate the use of reinforcement learning, such as q-learning, to automate acquiring knowledge ([Fernandez 2 Reinforcement Learning page 2]: “The main objective of reinforcement learning is to automatically acquire knowledge to better decide what action an agent should perform at any moment to optimally achieve a goal. Among many different reinforcement learning techniques, Q-learning has been very widely used [18].”).
Regarding Claim 2:
The method of claim 1 is taught by Lu, Singh, Woo, Fernandez.
Lu teaches:
wherein the first neural network and the fifth neural network are trained by back- propagating a gradient
([Lu II. Related Works B. Deep Learning Based Methods page 3]: “where a continuous relaxation of quantization was adopted for back-propagation pass [wherein the first neural network and the fifth neural network are trained by back- propagating a gradient]”)
The first and fifth neural network portions are taught in Singh. This quote is used to show backpropagation.
Singh teaches:
computing a task prediction loss based on the target image detection task or the target image segmentation task… of the task prediction loss
([Singh Compressible Feature Learning page 2]: “In a typical supervised classification or regression problem, we are concerned with minimizing a loss function L over a set of parameters θ… where x is the input variable (e.g., image pixels, or features), y the target variable (e.g., classification labels, or regression target) [computing a task prediction loss based on the target image detection task or the target image segmentation task… of the task prediction loss], y^ is the prediction, f is often an artificial neural network (ANN) with parameters θ comprising its set of filter weights, and D is a set of training data.”)
Relevant section pertaining to the classification as an example and use with video compression. [Singh Related Work page 4]: “Biswas et al. [24] proposed an approach to classify H.264 compressed videos. Chadha et al. [25] used 3D CNN architecture for video classification that directly utilized compressed video bitstreams. Yeo et al. [26] designed a system for performing action recognition on videos compressed with MPEG. Kantorov et al. [27] proposed a method for extracting and encoding local video descriptors for action recognition on MPEG compressed video representation. Our work differs in that we jointly optimize for a compressible representation along with the target task.”
updating weight parameters of the first neural network and the fifth neural network
([Singh Abstract page 1]: “We propose a learned method that jointly optimizes [updating weight parameters of the first neural network and the fifth neural network] for compressibility along with the task objective for learning the features.”)
The first and fifth neural network are taught by Singh, thus the jointly learning of the tasks of these two neural networks would pertain to updating the first and the fifth neural network.
One of ordinary skill, prior to the effective filing date would have been motivated to combined Lu and Singh to incorporate computing a task loss and updating the model. Lu and Singh are of the same field of endeavor of machine learning. One of ordinary skill would have been motivated to combine Lu and Singh in order to incorporate the use of a task loss in order to optimize to a specific task objective, as not optimizing for the task objective can create sub optimal models ([Singh Abstract page 1]: “Traditional entropy based lossless compression methods are of little help as they do not yield desired level of compression, while general purpose lossy compression methods based on energy compaction (e.g. PCA followed by quantization and entropy coding) are sub-optimal, as they are not tuned to task specific objective.”).
Regarding Claim 3:
The method of claim 1 is taught by Lu, Singh, Woo, Fernandez
Singh teaches:
Wherein the target image detection task or the target image segmentation task
([Singh Introduction page 1]: “Razavian et al. [10] demonstrated that a pre-trained network as a feature generator, coupled with a simple classifier such as a SVM [Wherein the target image detection task or the target image segmentation task as the neural network performing a task is a classifier, the task performed in this case would be classification] or logistic regression, performs surprisingly well and often outperforms hand tuned features on a variety of tasks.”)
is performed based on the generated plurality of latent representations
([Singh Introduction page 1]: “Features also need to be pre-computed and stored for certain types of applications where the target task evolves over time. A typical example is an indexing system or a content analysis system where the image features [is performed based on the generated plurality of latent representations] may be one of the many signals that the full model relies on.”)
The motivation to combine is the same motivation for Singh in claim 1.
Regarding Claim 4:
The method of claim 1 is taught by Lu, Singh, Woo, Fernandez
Lu teaches:
generating a set of encoded quantization keys by entropy encoding the set of quantization keys
generating a set of decoded quantization keys by entropy decoding the set of encoded quantization keys
([Lu Introduction page 1]: “The idea of DCNN-based image compression is straightforward. First, an auto-encoder is trained to obtain the latent representation of an image, followed by the quantization of the features, and then the quantized feature coefficients are coded by an entropy encoding [generating a set of encoded quantization keys by entropy encoding the set of quantization keys] method (e.g., Huffman coding or arithmetic coding methods). The decoder inverses the compression process. [generating a set of decoded quantization keys by entropy decoding the set of encoded quantization keys as entropy encoding is noted for the compression process earlier related to the quantization]”)
Woo teaches:
wherein the set of dequantized numbers are generated based on the set of decoded quantization keys
([Woo II. Dependent Bit Allocation A. Definitions and Notations page 2]: “In our experiments, we use simple objective measures such as mean square error (MSE) and PSNR. Note that subjective evaluation of three-dimensional quality is still an open problem and is not very reliable and repeatable yet. Therefore, we measure the distortions of F1 and F2 using MSE, i.e., D1 = (F1 – F1(Q1))^2 and D2 = (F2 - ^F2(Q1, Q2, V))^2, where F(Q) denotes the decoded image when quantizer Q is used. The decoded target image [wherein the set of dequantized numbers are generated based on the set of decoded quantization keys] ^F2(Q1, Q2, V), can be reconstructed by adding the compensated target image with the DV field and the decoded DCD, I.e., ^F2(Q1, Q2, V) = F1(Q1, V) + E(Q2), where E = F2 – F1(Q1, V).”)
The decoded image is seen as the dequantized numbers, where the decoded image utilized quantization indexes (represented by their corresponding quantizers) were used to decode.
The motivation to combine with Woo is the same motivation to combine with Woo in claim 1.
Regarding Claim 8:
This claim is analogous to claim 1.
Regarding Claim 9:
This claim is analogous to claim 2.
Regarding Claim 10:
This claim is analogous to claim 3.
Regarding Claim 11:
This claim is analogous to claim 4.
Regarding Claim 15:
This claim is analogous to claim 1.
Regarding Claim 16:
This claim is analogous to claim 2.
Regarding Claim 17:
This claim is analogous to claim 3.
Regarding Claim 18:
This claim is analogous to claim 4.
Claims 6, 13, 19, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lu et al (“Learning A Dep Vector Quantization Network for Image Compression”), referred to as Lu in this document, and further in view over Singh et al (“End to End Learning of Compressible Features”), referred to as Singh in this document, and further in view over Woo et al (“Optimal blockwise dependent quantization for stereo image coding”), referred to as Woo in this document, and further in view over Fernandez et al (“VQQL. Applying Vector Quantization to Reinforcement Learning”), referred to as Fernandez in this document, and even further in view of Kianfar et al (“US 20210329267 A1”), referred to as Kianfar in this document, and even further in view of Groi et al (“Performance Comparison of AV1, JEM, VP9, and HEVC Encoders”), referred to as Groi in this document.
Regarding Claim 6:
The method of claim 1 is taught by Lu, Singh, Woo, Fernandez.
Lu teaches:
wherein a quantization method of the set of quantization keys is same as a dequantization method of the set of dequantized numbers
([Lu Introduction page 1]: “The idea of DCNN-based image compression is straightforward. First, an auto-encoder is trained to obtain the latent representation of an image, followed by the quantization of the features, and then the quantized feature coefficients are coded by an entropy encoding method (e.g., Huffman coding or arithmetic coding methods). The decoder inverses the compression process [wherein a quantization method of the set of quantization keys is same as a dequantization method of the set of dequantized numbers].”)
Woo teaches:
wherein based on the set of quantization keys using the block-wise quantization method as the quantization method, the set of dequantized numbers use the block-wise dequantization method as the dequantization method
([Woo Introduction page 1]: “The main novelty of our work is the introduction of an algorithm for optimal blockwise [wherein based on the set of quantization keys using the block-wise quantization method as the quantization method, the set of dequantized numbers use the block-wise dequantization method as the dequantization method, as Woo teaches the use of blockwise quantization and the response to arguments goes over how in combination with Lu the dequantization would be obvious] dependent bit allocation for stereo image coding”)
The teaching of quantization using block-wise quantization for quantization and dequantization is also noted in the rejection of claim 1.
Motivation to combine is the same motivation to combine with Woo in claim 1.
Lu does not explicitly teach:
wherein based on the set of quantization keys using the individual quantization method as the quantization method, the set of dequantized numbers use the individual dequantization method as the dequantization method
wherein based on the set of quantization keys using the static quantization model method as the quantization method, the set of dequantized numbers use the static dequantization model method as the dequantization method
Kianfar teaches:
wherein based on the set of quantization keys using the individual quantization method as the quantization method,
[Kianfar 0009]: "determine scalar quantized coefficients, wherein determining the scalar quantized coefficients comprises applying scalar quantization [wherein based on the set of quantization keys using the individual quantization method as the quantization method, as scalar quantization is noted to be interpreted as the same as individual quantization] to the scaled transform coefficients of the block"
the set of dequantized numbers use the individual dequantization method as the dequantization method
[Kianfar 0065]: "The residual information may be represented by, for example, quantized transform coefficients. Video decoder 300 may inverse quantize [the set of dequantized numbers use the individual dequantization method as the dequantization method] and inverse transform the quantized transform coefficients of a block to reproduce a residual block for the block."
The motivation to combine with Kianfar is the same motivation to combine with Kianfar in claim 21.
Groi teaches:
wherein based on the set of quantization keys using the static quantization model method as the quantization method,
[Groi Introduction page 2]: "While HM and JEM encoders are typically tested by using static quantization [wherein based on the set of quantization keys using the static quantization model method as the quantization method] parameter (QP) configurations according to their common test conditions (CTCs) [9][20] as well as according to the CfP [10], the VP9 and AV1 encoders are evaluated by using two different approaches: 1) based on static QP configurations matching the above HM and JEM CTCs and their corresponding target bit-rates; and 2) utilizing their built-in multi-pass rate-control mechanisms."
the set of dequantized numbers use the static dequantization model method as the dequantization method
[Groi Abstract page 1]: "When directly comparing AV1 and JEM both for static quantization parameter settings, AV1 produces an average bit-rate overhead of more than 100% relative to JEM at the same objective reconstruction quality [the set of dequantized numbers use the static dequantization model method as the dequantization method as Groi notes that reconstruction is performed as part of the process related to the formats that involve static quantization as noted in the previous quote] and, in addition, with a factor of ~2.7 in encoder run time. Even when operated in a two-pass ratecontrol mode, AV1 lags behind both the JEM and HM reference encoder with average bit-rate overheads of ~55% and ~9.5%, respectively, although the latter being configured along one-pass static quantization parameter settings."
The motivation to combine with Groi is the same as the motivation to combine in claim 21.
Regarding Claim 13:
This claim is analogous to claim 6.
Regarding Claim 19:
This claim is analogous to teachings of claim 6.
Regarding Claim 21:
The method of claim 1 is taught by Lu, Singh, Woo, Fernandez
Lu does not explicitly teach:
generating the set of quantization keys using at least one of an individual quantization method and a static quantization method
generating the set of dequantized numbers using at least one of an individual dequantization method, and a static dequantization method
Kianfar teaches:
generating the set of quantization keys using at least one of an individual quantization method and a static quantization method
[Kianfar 0009]: "determine scalar quantized coefficients, wherein determining the scalar quantized coefficients comprises applying scalar quantization [generating the set of quantization keys using at least one of an individual quantization method as scalar quantization is noted to be interpreted as the same as individual quantization] to the scaled transform coefficients of the block"
generating the set of dequantized numbers using at least one of an individual dequantization method, and a static dequantization method
[Kianfar 0065]: "The residual information may be represented by, for example, quantized transform coefficients. Video decoder 300 may inverse quantize [generating the set of dequantized numbers using at least one of an individual dequantization method] and inverse transform the quantized transform coefficients of a block to reproduce a residual block for the block."
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Lu and Kianfar. Lu and Kianfar are in the same field of endeavor of quantization or compression. One of ordinary skill in the art would have been motivated to combine Lu and Kianfar in order to use the data of things like video more efficiently ([Kianfar 0003]: "The video devices may transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing such video coding techniques.").
Groi teaches:
generating the set of quantization keys using at least one of an individual quantization method and a static quantization method
[Groi Introduction page 2]: "While HM and JEM encoders are typically tested by using static quantization [generating the set of quantization keys using at least one of an individual quantization method and a static quantization method] parameter (QP) configurations according to their common test conditions (CTCs) [9][20] as well as according to the CfP [10], the VP9 and AV1 encoders are evaluated by using two different approaches: 1) based on static QP configurations matching the above HM and JEM CTCs and their corresponding target bit-rates; and 2) utilizing their built-in multi-pass rate-control mechanisms."
generating the set of dequantized numbers using at least one of an individual dequantization method, and a static dequantization method
[Groi Abstract page 1]: "When directly comparing AV1 and JEM both for static quantization parameter settings, AV1 produces an average bit-rate overhead of more than 100% relative to JEM at the same objective reconstruction quality [generating the set of dequantized numbers using at least one of an individual dequantization method, and a static dequantization method as Groi notes that reconstruction is performed as part of the process related to the formats that involve static quantization as noted in the previous quote] and, in addition, with a factor of ~2.7 in encoder run time. Even when operated in a two-pass ratecontrol mode, AV1 lags behind both the JEM and HM reference encoder with average bit-rate overheads of ~55% and ~9.5%, respectively, although the latter being configured along one-pass static quantization parameter settings."
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Lu and Groi. Lu and Groi are in the same field of endeavor of quantization or compression. One of ordinary skill in the art would have been motivated to combine Lu and Groi in order to reduce storage or resource requirements for quality of output as encoders can reduce overhead for a quality of image ([Groi Conclusion page 11]: "An extensive performance comparison of AV1, VP9, JEM, and HEVC encoders was presented and discussed in detail, while the evaluation was done with both fixed QP and multi-pass rate-control configurations. According to the experimental results, the coding efficiency of AV1 and VP9 was shown to be significantly inferior to both HM and JEM encoders. Particularly, for the fixed QP configuration, the AV1 coding scheme produced an average bit-rate overhead of 47.7% and 111.8%, at the same objective quality, relative to HM and JEM encoders, respectively.")
Relevant quote from specification for showing that the idea of a rule based idea can be relevant for the interpretation of claims like claim 21, as the quantization is noted in the specification to be able to be rule based [Current Application 0070]: “The Reconstructor uses a dequantization method that corresponds to the quantization method used in the Key Generator. For example, when the quantization method is predetermined rule-based method like uniform quantization with a fixed step size, the dequantization method is also predetermined rule-based such as computing the dequantized number y',i as the multiplication of the QK kt,i with the quantization step size. When the quantization method is a statistic model like k-means, the dequantization method may be the centroid indexed by the QK kt,i. This disclosure does not put any restrictions on the specific dequantization methods used as the Reconstructor.” The use of neural networks and quantization keys for quantization is noted to be taught by Lu in this document.
Allowable Subject Matter
Claim 7, 14, and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is an examiner's statement of reasons for allowance:
The prior art of record teaches limitations as noted by the previous office action.
However, the claim 7 in the application is deemed to be directed to a nonobvious improvement over the prior art of record.
The examiner notes that the elements of the present patent application appear present in prior art, however, the depth of the current application’s combination provides a convincing argument that a proper motivation be present in order to combine the known elements. As noted in the previous office action for the rejection of claim 7, claim 7 is considered showing elements of Lu, Woo, and Fernandez. Existing art teaches the use of utilizing machine learning for quantization (as shown by Lu in previous claim 1 and 7 rejections) and aspects related to quantization keys (as shown by Woo in previous claim 1 and 7 rejections). However, while previous art is considered showing that reinforcement learning would be obvious to one of ordinary skill in the art, prior to the effective filing date (as shown by combination with Fernandez in previous claim 1), claim 7 as currently recited is considered showing more than mere use of reinforcement learning. Claim 7 shows specific utilization of q learning in regards to quantization using machine learning for tasks. A motivation to combine with Fernandez to incorporate aspects of claim 7 is considered lacking, as no particular indication or motivation to combine aspects of q learning with quantization using machine learning, as indicated by claim 7, appears to exist in prior art. As a result, claim 7 (which includes by dependency the currently amended recitation of claim 1) is considered nonobvious.
Claims 14 and 20 are considered analogous to claim 7, thus are considered allowable for the same reasons given for claim 7. Any claims dependent upon claim 7 or made dependent upon claim 7 are considered allowable for being dependent upon an allowable claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hannuksela et al (US 20210127140 A1) is relevant art as the reference pertains to using neural networks with encoding and decoding, as well as the use of blockwise quantization and entropy.
Kwong et al (US 20200137384 A1) is relevant art as the reference pertains to the use of quantization, entropy encoding, latent variables, video compression, and the use of neural networks to assist with the compression algorithm (GANs in the case of Kwong et al).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER D DEVORE whose telephone number is (703)756-1234. The examiner can normally be reached Monday-Friday 7:30 am - 5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.D.D./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129