Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The 35 U.S.C. §112(b) rejection to claim 18 has been withdrawn in view of current amendments.
The 35 U.S.C. §112(b) rejection to claim 20 has been deemed moot in view of the cancellation of said claim.
Response to Arguments
Applicant's arguments filed March 9, 2026 have been fully considered but they are not persuasive. The Applicant argues US 2023/0100413 (hereinafter “Zhu”) fails to teach or suggest the limitations of claim 1. Specifically, the Applicant argues, in Zhu, the samples to be processed are samples of latent representation rather than samples of reconstructed latent representation. The Examiner respectfully disagrees. In Figure 3 and paragraphs 96-106 Zhu discloses an end-to-end neural network-based image and video coding (E2E-NNVC) system which is a combination of an autoencoder sub-network and a second sub-network that uses quantized latents used for entropy coding (paragraph 96). Figure 3 shows decoder 368 that includes a neural network that receives the quantized latents from Quantizer 364. Said quantized latents are being interpreted as the reconstructed latent representation.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, 7, 10, 12-13, 16-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Zhu et al. (US 2023/0100413).
Regarding claim 1 Zhu discloses a method for data processing, comprising:
processing, during a conversion between data and a bitstream of the data, a first set of samples of a reconstructed latent representation of the data and a second set of samples of the reconstructed latent representation by using a model, the first set of samples being associated with a first sample of the reconstructed latent representation and the second set of samples being associated with a second sample of the reconstructed latent representation; and performing the conversion based on a result of the processing (Zhu discloses an end-to-end neural network-based image and video coding (E2E-NNVC) system which is a combination of an autoencoder sub-network and a second sub-network that uses quantized latents used for entropy coding – [0096]; Figure 3 shows decoder 368 that includes a neural network that receives the quantized latents from Quantizer 364; it is noted the quantized latents are being interpreted as the reconstructed latent representation).
Regarding claim 2 Zhu discloses the method of claim 1, wherein the first set of samples and the second set of samples are processed in parallel (a transformer can process some or all of the sub-portions in parallel, such as when computing attention or self-attention. This parallelization can provide greater computational flexibility in comparison to, for example, RNNs, CNNs, or other neural networks trained to perform the same task – [0069]).
Regarding claim 3 Zhu discloses the method of claim 1, wherein the first set of samples does not comprise the second sample, and the second set of samples does not comprise the first sample (the encoder and decoder sub-networks operate over a series of patches; the series of patches are initially formed at the encoder sub-network as a non-overlapping segmentation of an input image – [0071]).
Regarding claim 4 Zhu discloses the method of claim 1, wherein the model comprises a neural subnetwork (sub-network in a neural network-based image and video coding – [0111]).
Regarding claim 5 Zhu discloses the method of claim 4, wherein the neural subnetwork is autoregressive (autoregressive – [0066-0067, 0070]).
Regarding claim 7 Zhu discloses the method of claim 1, wherein the first sample is at i-th row and j-th column of the reconstructed latent representation, the second sample is at p-th row and q-th column of the reconstructed latent representation, and each of i, j, p and q is an integer (obtaining a latent representation of an image – [0034-0035]; the encoder and decoder sub-networks operate over a series of patches; the series of patches are initially formed at the encoder sub-network as a non-overlapping segmentation of an input image – [0071]; a person with ordinary skill in the art would know the patches being processed by the encoder and decoder sub-networks are comprised of samples; each of the samples has a corresponding location identified by row and column).
Regarding claim 10 Zhu discloses the method of claim 7, wherein the first set of samples comprise a sample at (i+m)-th row and (j+n)-th column of the reconstructed latent representation, and each of m and n is an integer (obtaining a latent representation of an image – [0034-0035]; the encoder and decoder sub-networks operate over a series of patches; the series of patches are initially formed at the encoder sub-network as a non-overlapping segmentation of an input image – [0071]; a person with ordinary skill in the art would know a sample within a patch will have a coordinate/row and column/location within said patch).
Regarding claim 12 Zhu discloses the method of claim 10, wherein an absolute value of m is smaller than or equal to M, an absolute value of n is smaller than or equal to N, and each of M and N is a non-negative integer (obtaining a latent representation of an image – [0034-0035]; the encoder and decoder sub-networks operate over a series of patches; the series of patches are initially formed at the encoder sub-network as a non-overlapping segmentation of an input image – [0071]; a person with ordinary skill in the art would know a sample within a patch will have a coordinate/row and column/location within said patch; the claim limitations refer to coordinates of the samples within a patch of a frame in terms of rows and columns it is known that sample locations are described in terms of rows and columns).
Regarding claim 13 Zhu discloses the method of claim 12, wherein M is equal to 2 and N is equal to 2, or M is equal to 1 and N is equal to 1, or M is equal to 3 and N is equal to 3 (obtaining a latent representation of an image – [0034-0035]; the encoder and decoder sub-networks operate over a series of patches; the series of patches are initially formed at the encoder sub-network as a non-overlapping segmentation of an input image – [0071]; a person with ordinary skill in the art would know a sample within a patch will have a coordinate/row and column/location within said patch; the claim limitations refer to coordinates of the samples within a patch of a frame in terms of rows and columns it is known that sample locations are described in terms of rows and columns).
Regarding claim 16 Zhu discloses the method of claim 1, wherein information on at least one of the following is indicated in the bitstream: whether to process a plurality of samples of the reconstructed latent representation in parallel, or how to process the plurality of samples in parallel, or wherein information on the first set of samples is indicated in the bitstream, or wherein the reconstructed latent representation is partitioned into a plurality of regions, and the plurality of regions are processed independently, or wherein the reconstructed latent representation is a quantized latent representation of the data, or wherein the data comprise a picture of a video or an image (quantized latents – [0096]; the quantization codes (or data representing the quantized codes) can also be referred to as latent codes or as a latent (denoted as z) – [0104]).
Regarding claim 17 Zhu discloses the method of claim 1, wherein the conversion includes encoding the data into the bitstream, or wherein the conversion includes decoding the data from the bitstream (coding (e.g., encoding and/or decoding) media data using a transformer-based neural network architecture; obtaining a latent representation of a frame of encoded image data; and generating, by a plurality of decoder transformer layers of a decoder sub-network using the latent representation of the frame of encoded image data as input, a frame of decoded image data – [0004]).
Claim 18 corresponds to the apparatus performing the method of claim 1. Therefore, claim 18 is being rejected on the same basis as claim 1.
Claim 19 corresponds to the non-transitory computer-readable storage medium storing instructions that cause a processor to perform the acts of claim 1. Therefore, claim 19 is being rejected on the same basis as claim 1.
Claim 21 is being rejected on the same basis as claim 1. It is noted Zhu discloses storing the bitstream (the processor 304 is configured to send the output data 374 to at least one of the transmission medium 318 or the storage medium 314 – [0106]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (US 2023/0100413) in view of Nalci et al. (US 2023/0096567).
Regarding claim 6 Zhu discloses the method of claim 4. However, fails to explicitly disclose wherein the neural subnetwork comprises a context model subnetwork or a context subnetwork.
In his disclosure Nalci teaches wherein the neural subnetwork comprises a context model subnetwork or a context subnetwork (context based neural network module or sub-network – [0109]).
It would have been obvious to a person with ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the teachings of Nalci into the teachings of Zhu because the use of context models in neural networks/subnetworks is a well-known technique and would yield predictable results.
Claim(s) 8-9, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (US 2023/0100413) in view of Cricri et al. (WO 2020/012069A1).
Regarding claim 8 Zhu discloses the method of claim 7. However, fails to explicitly disclose wherein p=i+1 and q=j-T, and T represents a difference between column indexes of the first sample and the second sample and is a non-negative integer, or wherein p=1-T and q=j+1, and T represents a difference between row indexes of the first sample and the second sample and is a non-negative integer.
In his disclosure Cricri discloses wherein p=i+1 and q=j-T, and T represents a difference between column indexes of the first sample and the second sample and is a non-negative integer, or wherein p=1-T and q=j+1, and T represents a difference between row indexes of the first sample and the second sample and is a non-negative integer (the blocks (such as CTUs in HEVC) may be scanned in encoding and decoding tile-wise in the raster scan order of blocks, and tiles may be scanned in raster scan order along the tile grid. In wavefront parallel processing (WPP) each block row (such as CTU row in HEVC) of a slice can be encoded and decoded in parallel. When WPP is used, the state of the entropy codec at the beginning of a block row is obtained from the state of the entropy codec of the block row above after processing the second block of that row. Consequently, block rows can be processed in parallel with a delay of 2 blocks per each block row – p.22, 20-24).
It would have been obvious to a person with ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the teachings of Cricri into the teachings of Zhu because such incorporation improves the quality of the output video.
Regarding claim 9 Zhu discloses the method of claim 8. However, fails to explicitly disclose wherein T is equal to 0, 1, or 2, or wherein T is indicated in the bitstream.
In his disclosure Cricri teaches wherein T is equal to 0, 1, or 2, or wherein T is indicated in the bitstream (consequently, block rows can be processed in parallel with a delay of 2 blocks per each block row – p.22, 20-24).
It would have been obvious to a person with ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the teachings of Cricri into the teachings of Zhu because such incorporation improves the quality of the output video.
Regarding claim 14 Zhu discloses the method of claim 1. Zhu further discloses performing a predefined scan order (paragraph 94).
However, fails to explicitly disclose wherein if a third sample of the first set of samples is unavailable, a value of the third sample is obtained by means of padding or sample replication, or wherein samples of the reconstructed latent representation are processed row by row or column by column, or wherein a shape of a processing kernel for processing the first sample is symmetric around a diagonal axis passing through the first sample, or wherein the first set of samples are neighboring samples of the first sample.
In his disclosure Cricri teaches wherein if a third sample of the first set of samples is unavailable, a value of the third sample is obtained by means of padding or sample replication, or wherein samples of the reconstructed latent representation are processed row by row or column by column, or wherein a shape of a processing kernel for processing the first sample is symmetric around a diagonal axis passing through the first sample, or wherein the first set of samples are neighboring samples of the first sample (raster scan order of blocks – p.22, 20-24).
It would have been obvious to a person with ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the teachings of Cricri into the teachings of Zhu because such incorporation improves the quality of the output video.
Examiner’s Note: Claim 14 is written in an alternative form, the Examiner has chosen the alternative: “wherein samples of the reconstructed latent representation are processed row by row or column by column”.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (US 2023/0100413) in view of Ren et al. (US Patent No. 11,468,542).
Regarding claim 15 Zhu discloses the method of claim 14. However, fails to explicitly disclose wherein the value of the third sample is set to be one of: a fixed value, or a value of the nearest available sample of the third sample, or wherein the value of the third sample is generated by a mirror padding or a circular padding; wherein a transposing process is applied on the reconstructed latent representation before the processing; wherein the first set of samples are determined based on a position of the first sample.
In his disclosure Ren teaches wherein the value of the third sample is set to be one of: a fixed value, or a value of the nearest available sample of the third sample, or wherein the value of the third sample is generated by a mirror padding or a circular padding; wherein a transposing process is applied on the reconstructed latent representation before the processing; wherein the first set of samples are determined based on a position of the first sample (CS measurements are fused with a low-dimensional contextual latent vector to generate a high-frequency image residual, which is subsequently upsampled via a transposed CNN – col.19, 31-34).
It would have been obvious to a person with ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the teachings of Ren in to the teachings of Zhu because such incorporation improves the reconstruction performance.
Examiner’s Note: Claim 15 is written in an alternative form. The Examiner has elected the alternative: “wherein a transposing process is applied on the reconstructed latent representation before the processing”.
Allowable Subject Matter
Claim 11 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIA E VAZQUEZ COLON whose telephone number is (571)270-1103. The examiner can normally be reached M-F 7:30 AM-3:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CHRISTOPHER S KELLEY can be reached at (571)272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARIA E VAZQUEZ COLON/Examiner, Art Unit 2482
Prosecution Insights