DETAILED ACTION
The present Office action is in response to the application filing on 20 JUNE 2024.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The Information Disclosure Statement (IDS) submitted on 06/20/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the Information Disclosure Statement is being considered by the Examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 1-12, 19, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2020/0145661 A1 (hereinafter “Jeon”) in view of U.S. Publication No. 2021/0125380 A1 (hereinafter “Lee”).
Regarding claim 1, Jeon discloses a method for decoding a bitstream to output one or more pictures for a video stream ([0158], “a decoder 160 configured to perform a process of decoding and thus reconstructing an image, and a bitstream obtainer 170 configured to obtain a bitstream including information associated with an encoded image”), the method comprising:
receiving a bitstream ([0158], “obtainer 170 configured to obtain a bitstream”); and
decoding, using coded information of the bitstream, one or more pictures ([0160], “the bitstream obtainer 170 may obtain a bitstream including information associated with an encoded image and various types of information used in the encoding, so as to allow the decoder 160 to use the bitstream in a decoding process”),
wherein the decoding comprises:
decompressing one or more compressed pictures comprised in the bitstream ([0169], “The decoder 160 has to perform an operation of decompressing a compressed residual signal obtained from a bitstream”); and
performing spatial upsampling on the one or more decompressed ([0190], “The residual signal increasing unit 925 may increase a resolution of a residual signal output from the autoencoder 924, and thus may restore the resolution of the residual signal which is decreased in an encoding process.” Note, the autoencoder 924 is part of the decoder, see FIG. 9, decoder 920),
wherein a total length of coding bits of parameters of the spatial upsampling model is less than a threshold that is pre-determined based on a desired quality of the reconstructed pictures ([0140], “when the compression information determined by the encoder 600 indicates a high degree of compression, the number of layers and the number of network nodes which constitute the autoencoder 604 may be increased.” [0145], “include a compression information generator 704 generating compression information used in a process of encoding an image at a degree of compression at which a subjective quality is changed […] decompressing the residual signal to its original state.” [0155], “the bitstream including compression information used when an image is compressed according to at least one degree of compression determined based on the subjective quality.” Note, the degree of compression represents the desired quality, which in turn determines the amount of data (i.e., “total length of coding bits”) transmitted in the bitstream as part of the compression information indicating neural-network parameters).
Jeon fails to expressly disclose performing spatial upsampling on the one or more decompressed pictures by a spatial upsampling model to obtain one or more reconstructed pictures, respectively.
However, Lee teaches performing spatial upsampling on the one or more decompressed pictures by a spatial upsampling model to obtain one or more reconstructed pictures, respectively ([0073], “During the AI decoding process, AI encoding data including AI data and image data, which are obtained as a result of AI encoding is received, a second image 135 is obtained via the first decoding 130, and a third image 145 is obtained by a decoding apparatus, receiving device, or the like performing AI up-scaling 140 on the second image 135.” FIG. 1 depicts first decoding 130 reconstructing a down-sampled picture 135 and up-sampling with AI up-scaling 140 and using AI data (i.e., “parameters of the spatial upsampling model”) signaled in the bitstream to produce a reconstructed up-sampled picture 145).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to have up-sampled a reconstructed picture, as taught by Lee (FIG. 1), in Jeon’s invention. One would have been motivated to modify Jeon’s invention, by incorporating Lee’s invention, to improve coding efficiency by achieving low bitrate for high resolution video (Lee: [0004-0005]).
Regarding claim 2, Jeon and Lee disclose every limitation of claim 1, as outlined above. Additionally, Lee discloses wherein the decoding further comprises: extracting from the bitstream parameters that comprise weights and bias for respective layers of the spatial upsampling model ([0010], “The obtained DNN setting information may include weights and biases of filter kernels in at least one convolution layer of the up-scaling DNN”). The same motivation of claim 1 applies equal to claim 2.
Regarding claim 3, Jeon and Lee disclose every limitation of claim 1, as outlined above. Additionally, Jeon discloses wherein the spatial upsampling model comprises 11 convolutional layers ([0091], “the deep convolutional neural network including any number of hidden layers”).
Regarding claim 10, the limitations are the same as those in claim 1; however, written from the perspective of the encoder, which performs the well-known inverse operations. Therefore, the same rationale of claim 1 applies to claim 10.
Regarding claim 11, the limitations are the same as those in claim 2; however, written from the perspective of the encoder, which performs the well-known inverse operations. Therefore, the same rationale of claim 2 applies to claim 11.
Regarding claim 12, the limitations are the same as those in claim 3; however, written from the perspective of the encoder, which performs the well-known inverse operations. Therefore, the same rationale of claim 3 applies to claim 12.
Regarding claim 19, the limitations are the same as those in claim 1. Therefore, the same rejection of claim 1 applies to claim 19. Note, this is an alternate rejection in which the non-transitory computer readable storage medium also includes a program executed by a processor to perform the recited operations. See 35 U.S.C. § 102 rejection below.
Regarding claim 20, the limitations are the same as those in claim 3. Therefore, the same rejection of claim 3 applies to claim 20.
Claim(s) 5 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2020/0145661 A1 (hereinafter “Jeon”) in view of U.S. Publication No. 2021/0125380 A1 (hereinafter “Lee”), and further in view of U.S. Publication No. 2025/0379978 A1 (hereinafter “Li”).
Regarding claim 5, Jeon and Lee disclose every limitation of claim 1, as outlined above. Jeon and Lee fail to expressly disclose wherein the spatial upsampling model comprises one or more Bottleneck Resblocks (BRes).
However, Li teaches wherein the spatial upsampling model comprises one or more Bottleneck Resblocks (BRes) ([0126], “each of the contextual encoder and the contextual decoder can include two bottleneck residual blocks”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to have used bottleneck residual blocks, as taught by Li ([0126]), in Jeon and Lee’s invention. One would have been motivated to modify Jeon and Lee’s invention, by incorporating Li’s invention, to reduce the complexity middle layers (Li: [0126]).
Regarding claim 14, the limitations are the same as those in claim 5; however, written from the perspective of the encoder, which performs the well-known inverse operations. Therefore, the same rationale of claim 5 applies to claim 14.
Claim(s) 8, 9, 17, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2020/0145661 A1 (hereinafter “Jeon”) in view of U.S. Publication No. 2021/0125380 A1 (hereinafter “Lee”), and further in view of U.S. Publication No. 2022/0231933 A1 (hereinafter “Mannor”).
Regarding claim 8, Jeon and Lee disclose every limitation of claim 1, as outlined above. Additionally, Lee discloses wherein the parameters of the spatial upsampling model are quantized into a pre-determined format ([0251], ““model compression” refers to compression techniques for reducing the amount of data while maintaining the highest possible accuracy by reducing the number and size of parameters in an artificial neural network model, to reduce the complexity of the artificial neural network model. Examples of ‘model compression’ include pruning and quantization”). The same motivation of claim 1 applies to claim 8.
Jeon and Lee fail to expressly disclose a pre-determined format.
However, Mannor teaches a pre-determined format ([0116], “As part of the quantization process, model weights may be quantized and stored in int8”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to have used an int8 format, as taught by Mannor ([0116]), in Jeon and Lee’s invention. One would have been motivated to modify Jeon and Lee’s invention, by incorporating Mannor’s invention, because it is an obvious application of a known technique for data formatting yielding predictable results. See MPEP § 2143(I)(D).
Regarding claim 9, Jeon, Lee and Mannor disclose every limitation of claim 8, as outlined above. Additionally, Mannor discloses wherein the pre-determined format is one of float16, int8, or binary ([0116], “As part of the quantization process, model weights may be quantized and stored in int8”). The same motivation of claim 8 applies to claim 9.
Regarding claim 17, the limitations are the same as those in claim 8; however, written from the perspective of the encoder, which performs the well-known inverse operations. Therefore, the same rationale of claim 8 applies to claim 17.
Regarding claim 18, the limitations are the same as those in claim 9; however, written from the perspective of the encoder, which performs the well-known inverse operations. Therefore, the same rationale of claim 9 applies to claim 18.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 19 and 20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by U.S. Publication No. 2014/0294063 A1 (hereinafter “Chen”).
Regarding claim 19, Chen discloses a non-transitory computer readable storage medium storing a bitstream of a video […] (MPEP § 2111.05(I)(A) states, “[t]o be given patentable weight, the printed matter and associated product must be in a functional relationship. A functional relationship can be found where the printed matter performs some function with respect to the product to which it is associated.” When a claimed “computer-readable medium merely serves as a support for information or data, no functional relationship exists. See MPEP § 2111.05(III). The storage medium storing the claimed bitstream of claim 19 merely services as a support for the storage of the data stream and provides no functional relationship between the stored bitstream and storage medium. Therefore, the bitstream structure, which scope is implied by the method steps, is non-functional descriptive material and given no patentable weight. See MPEP §2111.05(III). Thus, the claim scope is just a storage medium storing a bitstream and is anticipated by Chen, which recites, [0053], “encoded data may be output from interface 22 to a storage device”).
Regarding claim 20, Chen discloses every limitation of claim 19, as outlined above. Additionally, Chen discloses wherein the spatial upsampling model comprises 11 convolutional layers, or one or more Bottleneck Resblocks (BRes) (As per the rejection of claim 19, the subject matter is nonfunctional descriptive material. [0053], “encoded data may be output from interface 22 to a storage device”).
Allowable Subject Matter
Claims 4, 6, 7, 13, 15, and 16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Patent No. 11,582,481 B1 (hereinafter “Djokovic”) – Discloses removing numbers of layers, neurons, and other parameters for reducing complexity of a neural network based on a desired balance of cost function. See Djokovic, col. 5, ll. 58-67.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STUART D BENNETT whose telephone number is (571)272-0677. The examiner can normally be reached Monday - Friday from 9:00 AM - 5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at 571-272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STUART D BENNETT/Examiner, Art Unit 2481
Prosecution Insights