Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim (7-9, 11-12), (1-3, 5-6), (13-15, 17-19) are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Chinese Patent Pub. No.: CN110602494A), in view of Yao (Chinese Patent Pub. No.: CN111641832A), hereinafter Yao, further in view of Li (Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compression, arXiv:1906.10057v2 [eess.IV] 28 Mar 2020), hereinafter Li.
Regarding claim 7, Wang teaches an encoding method (Image encoding and decoding system and encoding and decoding method based on deep learning. Title), comprising: determining a first image feature (S101: Perform a forward transform on the input image to obtain the feature coefficient matrix representing the image information. [0035]), a probability distribution (A deep learning-based conditional probability prior analysis module is used to analyze the feature coefficients and obtain prior feature values that characterize the conditional probability of the feature coefficients. [0013]), and a first hyper-prior feature of each feature point in a plurality of feature points of a to-be-encoded image (S102: Input the feature coefficient matrix into the prior analysis module and output the prior eigenvalue matrix representing the probability of the feature coefficients. [0036]); encoding first image features into a bit stream based on the probability distributions of the plurality of feature points (The entropy coding module is used to entropy code the quantized feature coefficients to obtain the feature coefficient bitstream under the guidance of the super-prior conditional probability. [0014]); and encoding the first hyper-prior features of the plurality of feature points into the bit stream (It is also used to entropy code the quantized super-prior feature values on the conditional probability model statistically analyzed on the training set to obtain the super-prior feature value bitstream. [0014]).
Wang does not teach the following limitations as further recited, but Yao further teaches dividing the plurality of feature points into a plurality of groups of feature points (Step 201: Divide the image to be encoded into multiple image blocks. [0161]); sequentially (It should be noted that when performing feature transformation on multiple image patches using a transform convolutional neural network, the multiple image patches can be input into the transform convolutional neural network sequentially, which is the implementation method described above. [0169]) encoding first image features of each group of feature points in the plurality of groups of feature points (Step 806: Encode the quantization result of the first transform domain component based on the probability distribution of the quantization result of the first transform domain component. [0214]) into a bit stream (The encoded stream of the first image block among multiple image blocks is obtained. The encoded stream includes a first bitstream and a second bitstream. [0056]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Yao to divide the feature points into groups of feature points and sequentially encode first image features of each group of feature points into a bit stream in order to guarantee the compression rate and distortion rate of the encoded image.
The combination of Wang and Yao does not teach the following limitations as further recited, but Li further teaches dividing the plurality of feature points into a plurality of groups of feature points (Fig. 2(e) Support sets of the orange and blue codes
PNG
media_image1.png
492
1276
media_image1.png
Greyscale
) based on a specified numerical value (
PNG
media_image2.png
114
628
media_image2.png
Greyscale
. Page 4 right column last paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Li to divide the plurality of feature points into a plurality of groups of feature points based on a specified numerical value in order to achieve efficient and effective entropy modeling.
Regarding claim 8, Wang in the combination teaches the encoding method according to claim 7, wherein determining the first image feature, the probability distribution, and the first hyper-prior feature of each feature point in a plurality of feature points of the to-be-encoded image comprises: determining first image features of the plurality of feature points (S101: Perform a forward transform on the input image to obtain the feature coefficient matrix representing the image information. [0035]); and determining the first hyper-prior features of the plurality of feature points (S102: Input the feature coefficient matrix into the prior analysis module and output the prior eigenvalue matrix representing the probability of the feature coefficients. [0036]).
Yao in the combination further teaches determining the probability distribution of each feature point in the plurality of feature points (Step 904: Input the quantization result of the first transform domain component into the probability estimation network, and determine the probability distribution of the quantization result of the first transform domain component through the probability estimation network. [0228]) in parallel (Of course, multiple image blocks can also be input into a transform convolutional neural network simultaneously. [0169]) based on the first image features of the plurality of feature points (Step 902: For the first image block among the multiple image blocks, based on the residual block of the first image block, feature transformation is performed through a transform convolutional neural network to obtain the first transform domain component corresponding to the first image block. [0223]).
Regarding claim 9, Li in the combination teaches the encoding method according to claim 8, wherein the plurality of feature points comprise a first feature point (The orange dot in Fig. 2(e).); and determining the probability distribution of the first feature point comprises: in response to determining that the first feature point is a non-initial feature point in the plurality of feature points (As shown in Fig. 2(e), the orange dot is in the middle of the 2D code block.
PNG
media_image3.png
492
1276
media_image3.png
Greyscale
); determining periphery information of the first feature point from the first image features of the plurality of feature points (
PNG
media_image1.png
492
1276
media_image1.png
Greyscale
); inputting the periphery information of the first feature point into a context model, to obtain a context feature of the first feature point (
PNG
media_image4.png
492
1276
media_image4.png
Greyscale
).
Wang in the combination further teaches determining a prior feature of the first feature point based on the first image feature of the first feature point (S102: Input the feature coefficient matrix into the prior analysis module and output the prior eigenvalue matrix representing the probability of the feature coefficients. [0036]); determining the probability distribution of the first feature point based on the prior feature of the first feature point (S104: Train the feature coefficients based on the quantized super-prior eigenvalue matrix in S103 to obtain a conditional probability model based on the Laplace distribution super-prior. [0038]) and the context feature of the first feature point (the entropy coding module is used for entropy coding. Abstract. It is common knowledge that entropy coding uses context feature of the feature point.).
Regarding claim 11, Li in the combination teaches the encoding method according to claim 9, wherein the specified numerical value is determined based on a size of a receptive field (It is common knowledge that the features a convolutional layer can process are strictly limited by its receptive field and, as a result, the specified numerical value depends on the size of the receptive field.) used by the context model (
PNG
media_image5.png
448
630
media_image5.png
Greyscale
); and dividing the plurality of feature points into the plurality of groups of feature points comprises: determining a slope based on the specified numerical value (A slope is determined by p+q=k, as is shown in Fig. 2(e).
PNG
media_image6.png
114
628
media_image6.png
Greyscale
. Page 4 right column last paragraph), wherein the slope indicates a tilt degree of a straight line on which feature points to be divided into a same group are located (Fig. 2(e) support set of the orange code
PNG
media_image7.png
492
1278
media_image7.png
Greyscale
); and dividing the plurality of feature points into the plurality of groups of feature points based on the slope (Fig. 2 (e) support sets of the orange and blue codes).
Regarding claim 12, Yao in the combination teaches the encoding method according to claim 11, wherein when the context model uses a plurality of receptive fields with different sizes (multiple image patches may be of different sizes. Abstract. It is common knowledge that a context model can use receptive fields with different sizes.), the specified numerical value is determined based on a size of a largest receptive field in the plurality of receptive fields with different sizes (It is common knowledge that the features a convolutional layer can process are strictly limited by its receptive field and, as a result, the size of the largest receptive field sets the boundary for the maximum area of input data the model can access to make a prediction.).
Regarding claim 1, Wang in the combination teaches a decoding method (Image encoding and decoding system and encoding and decoding method based on deep learning. Title), comprising: determining a prior feature of each feature point in a plurality of feature points of a to-be-decoded image based on a bit stream (S142: Based on the model number, use the corresponding super-prior eigenvalue entropy encoding model to decode the super-prior eigenvalue bitstream to obtain the super-prior eigenvalue matrix; based on the model number, construct and initialize the corresponding network model (including: the network parameters of the deep learning network used). [0101]); determining first image features of each group of feature points in the plurality of groups based on the prior features of the plurality of feature points (S143: Input the super-prior eigenvalue matrix obtained in step 142 into the super-prior reconstruction module, and output the conditional probabilities of the eigenvalue coefficients; [0102]. S144: Using the conditional probabilities of the feature coefficients in step 143, decode the feature coefficient matrix of the image from the feature coefficient bitstream. [0103]), wherein determining the first image features of each group of feature points in the plurality of groups comprises: determining a probability distribution of each feature point in a group of feature points (S143: Input the super-prior eigenvalue matrix obtained in step 142 into the super-prior reconstruction module, and output the conditional probabilities of the eigenvalue coefficients; [0102]), and parsing the bit stream to obtain a first image feature of each feature point in the group of feature points based on the probability distribution of each feature point in the group of feature points (S144: Using the conditional probabilities of the feature coefficients in step 143, decode the feature coefficient matrix of the image from the feature coefficient bitstream. [0103]); and reconstructing the to-be-decoded image based on the first image features of the plurality of feature points (S145: Input the feature coefficient matrix from step S144 into the inverse transform network module to reconstruct the image pixel values. [0104]).
Yao in the combination further teaches dividing the plurality of feature points into a plurality of groups of feature points (dividing the image to be encoded into multiple image blocks. Abstract); sequentially (It should be noted that when performing feature transformation on multiple image patches using a transform convolutional neural network, the multiple image patches can be input into the transform convolutional neural network sequentially, which is the implementation method described above. [0169]) determining first image features of each group of feature points in the plurality of groups (Step 1104: Based on the probability distribution corresponding to the first image block, decode the first bitstream to obtain the quantization result of the first transform domain component corresponding to the first image block. [0245]), wherein sequentially determining the first image features of each group of feature points in the plurality of groups comprises: determining a probability distribution of each feature point in a group of feature points (Furthermore, after obtaining the first bitstream of the first image block, the prediction information of the first image block can be determined based on the first bitstream of the first image block by using an inverse prediction convolutional neural network. [0237]. Step 1102: Decode the second bitstream to obtain the side information of the first image block. [0240]. Step 1103: Determine the probability distribution corresponding to the first image block based on the edge information. [0242]) in parallel (Of course, multiple image blocks can also be input into a transform convolutional neural network simultaneously. [0169]).
Li in the combination further teaches dividing the plurality of feature points into a plurality of groups of feature points (
PNG
media_image1.png
492
1276
media_image1.png
Greyscale
) based on a specified numerical value (
PNG
media_image2.png
114
628
media_image2.png
Greyscale
. Page 4 right column last paragraph).
Regarding claim 2, Li in the combination teaches the decoding method according to claim 1, wherein the group of feature points comprises a first feature point (The orange dot in Fig. 2(e).); and determining the probability distribution of each feature point in the group of feature points in parallel comprises: in response to determining that the first feature point (The orange dot in Fig. 2(e).) is a non-initial feature point in the plurality of feature points (As shown in Fig. 2(e), the orange dot is in the middle of the 2D code block.
PNG
media_image3.png
492
1276
media_image3.png
Greyscale
), determining periphery information of the first feature point from first image features of decoded feature points (Fig. 2(e) support sets of the orange code
PNG
media_image1.png
492
1276
media_image1.png
Greyscale
); inputting the periphery information of the first feature point into a context model, to obtain a context feature of the first feature point (
PNG
media_image4.png
492
1276
media_image4.png
Greyscale
).
Wang in the combination further teaches determining the probability distribution of the first feature point based on a prior feature of the first feature point (S143: Input the super-prior eigenvalue matrix obtained in step 142 into the super-prior reconstruction module, and output the conditional probabilities of the eigenvalue coefficients; [0102]. S144: Using the conditional probabilities of the feature coefficients in step 143, decode the feature coefficient matrix of the image from the feature coefficient bitstream. [0103]) and the context feature of the first feature point (the entropy decoding module is used for entropy decoding. Abstract. It is common knowledge that entropy coding uses context feature of the feature point.).
Regarding claim 3, Li in the combination teaches the decoding method according to claim 2, wherein the periphery information of the first feature point comprises first image features of decoded feature points in a neighborhood (Fig. 2(b) support set of the orange code) that uses the first feature point as a geometric center (Fig. 2(b) orange dot
PNG
media_image3.png
492
1276
media_image3.png
Greyscale
), a size of the neighborhood is determined based on a size of a receptive field used by the context model (
PNG
media_image5.png
448
630
media_image5.png
Greyscale
), the periphery information comprises at least first image features of n feature points around the first feature point (Fig. 2(b) support set of the orange code), and n is an integer greater than or equal to 4 (Fig. 2(b) n=12).
Regarding claim 5, Li in the combination teaches the decoding method according to claim 3, wherein the specified numerical value is determined based on the size of the receptive field (It is common knowledge that the features a convolutional layer can process are strictly limited by its receptive field and, as a result, the specified numerical value depends on the size of the receptive field.) used by the context model (
PNG
media_image5.png
448
630
media_image5.png
Greyscale
); and dividing the plurality of feature points into the plurality of groups of feature points comprises: determining a slope based on the specified numerical value (A slope is determined by p+q=k, as is shown in Fig. 2(e).
PNG
media_image6.png
114
628
media_image6.png
Greyscale
. Page 4 right column last paragraph), wherein the slope indicates a tilt degree of a straight line on which feature points to be divided into a same group are located (Fig. 2(e) support set of the orange code
PNG
media_image7.png
492
1278
media_image7.png
Greyscale
); and dividing the plurality of feature points into the plurality of groups of feature points based on the slope (Fig. 2 (e) support sets of the orange and blue codes).
Regarding claim 6, Yao in the combination teaches the decoding method according to claim 5, wherein when the context model uses a plurality of receptive fields with different sizes (multiple image patches may be of different sizes. Abstract. It is common knowledge that context model can use receptive fields with different sizes.), the specified numerical value is determined based on a size of a largest receptive field in the plurality of receptive fields with different sizes (It is common knowledge that the features a convolutional layer can process are strictly limited by its receptive field and, as a result, the size of the largest receptive field sets the boundary for the maximum area of input data the model can access to make a prediction.).
Apparatus claims 13-15 and 17-18 are drawn to the apparatus corresponding to the method of using same as claimed in claims 1-3 and 5-6. Therefore apparatus claims 13-15 and 17-18 correspond to method claims 1-3 and 5-6, and are rejected for the same reasons of obviousness as used above.
Claim 19 is drawn to a non-transitory computer-readable storage medium having executable instructions stored for carrying out the method of claim 1. Therefore, claim 19 corresponds to method claim 1, and is rejected for the same reasons of obviousness as used above.
Claims 4, 10 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Chinese Patent Pub. No.: CN110602494A), in view of Yao (Chinese Patent Pub. No.: CN111641832A), hereinafter Yao, further in view of Li (Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compression, arXiv:1906.10057v2 [eess.IV] 28 Mar 2020), hereinafter Li, further in view of Balaji (Bayesian Inference Concepts & Methods – Math Chapter 12, https://www.studocu.com/en-us/document/carnegie-mellon-university/foundation-probability/bayes-math/9374053, 2017/2018), hereinafter Balaji.
Regarding claim 4, Wang, Yao and Li teach all of the elements of the claimed invention as stated in claim 1 except for the following limitations as further recited. However, Balaji teaches wherein the plurality of feature points comprise a first feature point (Assumption I establishes a one-to-many correspondence between the input code block y and the output feature representation v(T). Page 3 left column 2nd paragraph); and determining the probability distribution of the first feature point comprises: in response to determining that the first feature point is an initial feature point in the plurality of feature points, determining the probability distribution of the first feature point based on a prior feature of the first feature point (We choose a probability density π(θ)— called the prior distribution — that expresses our beliefs about a parameter θ before we see any data (i.e., an initial feature point which does not have any data yet.). Page 2 2nd paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang, Yao and Li to incorporate the teachings of Balaji to determine the probability distribution of the first feature point based on a prior feature of the first feature point in response to determining that the first feature point is an initial feature point in the plurality of feature points in order to have the best estimation for the starting point.
Regarding claim 10, Balaji in the combination teaches the encoding method according to claim 8, wherein the plurality of feature points comprise a first feature point (Assumption I establishes a one-to-many correspondence between the input code block y and the output feature representation v(T). Page 3 left column 2nd paragraph); and determining the probability distribution of the first feature point comprises: in response to determining that the first feature point is an initial feature point in the plurality of feature points, determining the probability distribution of the first feature point based on a prior feature of the first feature point (We choose a probability density π(θ)— called the prior distribution — that expresses our beliefs about a parameter θ before we see any data (i.e., an initial feature point which does not have any data yet.). Page 2 2nd paragraph).
Apparatus claim 16 is drawn to the apparatus corresponding to the method of using same as claimed in claim 4. Therefore apparatus claim 16 corresponds to method claim 4, and is rejected for the same reasons of obviousness as used above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/
Supervisory Patent Examiner
Art Unit 2668