DETAILED ACTION
This action is in response to the application filed on 11/29/2024.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The references listed on the Information Disclosure Statement submitted on 11/29/2024 has/have been considered by the examiner (see attached PTO-1449).
Claim Mapping Notation
In this office action, following notations are being used to refer to the paragraph numbers or column number and lines of portions of the cited reference.
In this office action, following notations are being used to refer to the paragraph numbers or column number and lines of portions of the cited reference.
[0005] (Paragraph number [0005])
C5 (Column 5)
Pa5 (Page 5)
S5 (Section 5)
Furthermore, unless necessary to distinguish from other references in this action, “et al.” will be omitted when referring to the reference.
Claim Interpretation
MPEP §2111.05(III): Non-functional Descriptive Material there is no functional relationship between the steps or elements that describe the generation of the bitstream and intended computer system of Claim 20.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 20 is rejected under 35 U.S.C. 102(a1) and (a2) as being anticipated by Karczewicz et al. (US20220103816).
20. A non-transitory computer storage medium, having stored a bitstream generated according to ,the coding method method according to claim 14.
“[0043] Video encoder 200 and video decoder 300 each may be implemented as any of a variety of suitable encoder and/or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof. When the techniques are implemented partially in software, a device may store instructions for the software in a suitable, non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure.”
Claim 20’s recitation of a CSM having stored a bit stream which is generated by the coding method applied to an encoder, the method comprising…” is a product by process claim limitation where the product is the bit stream and the process is the method steps to generate the bitstream. MPEP §2113 recites “Product-by-Process claims are not limited to the manipulations of the recited steps, only the structure implied by the steps”. Thus, the scope of the claim is the CRM storing the bitstream (with the structure implied by the method steps). The structure includes multiple tables, motion vector prediction tables, video blocks lock, motion blocks and other information manipulated by the steps.
To be given patentable weight, the CSM and the bitstream (i.e. descriptive material) must be in a functional relationship. A functional relationship can be found where the descriptive material performs some function with respect to the CSM to which it is associated. See MPEP §2111.05(I)(A). When a claimed “computer storage medium merely serves as a support for information or data, no functional relationship exists”. MPEP §2111.05(III). The storage medium storing the claimed bitstream in claim 20 merely services as a support for the storage of the bitstream and provides no functional relationship between the stored bitstream and storage medium. Therefor the structure bitstream, which scope is implied by the method steps, is non-functional descriptive material and given no patentable weight. MPEP §2111.05(III). Thus, the claim scope is just a storage medium storing data.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over Karczewicz et al. (US20220103816) in view of Agashe et al. (US 12057336 B2).
Regarding the claim 1, Karczewicz discloses the invention substantially as claimed.
Karczewicz discloses,
1. A decoding method, applied to a decoder, comprising:
decoding a bitstream, and determining first flag information;
“[0067] In this manner, video encoder 200 may generate a bitstream including encoded video data, e.g., syntax elements describing partitioning of a picture into blocks (e.g., CUs) and prediction and/or residual information for the blocks. Ultimately, video decoder 300 may receive the bitstream and decode the encoded video data.”
“[0208] In particular, initially, entropy decoding unit 302 may decode a value for a flag representing whether refinement filtering is to be applied.”
determining, in a case where the first flag information indicates that a scaling factor…is used for a current color component of a current frame of a current sequence, second flag information;
“[0208] In particular, initially, entropy decoding unit 302 may decode a value for a flag representing whether refinement filtering is to be applied.”
“[0132] vi) Video encoder 200 may separately signal scaling factors for each color component, and video decoder 300 may determine the scaling factors for each color component separately from the signaled values.”
“[0130] iv) Video encoder 200 may signal the values of the scaling factors in the bitstream as a syntax element at block level, e.g., at CTU level.”
determining, in a case where the second flag information indicates that the scaling factor… is used for a current block of the current color component, a scaling factor corresponding to the current block using a scaling factor generation…,
“[0139] Video encoder 200 may select one of the scaling factors based on an RD-cost, then signal the best scaling factor (i.e., the scaling factor having the best RD-cost) using a direct representation of the scaling factor or an index representing the scaling factor in data of the bitstream. The index can be coded as unary code or truncated binary code, or variable length coded based on the statistical properties of the scaling factor. When the scaling factor has a particular value, which is most probable across all input sequences, then unary code signaling can be chosen as a choice of signaling method, wherein, the least index value can be assigned to most a probable scaling factor value. [0140] vi. The value of scaling factors and offset may be derived at the encoding process and signaled in the bitstream as a syntax element at a block level, e.g., at a CTU level.”
wherein the scaling factor generation…is used to predict a scaling factor of a picture block; and
“[0130] iv) Video encoder 200 may signal the values of the scaling factors in the bitstream as a syntax element at block level, e.g., at CTU level. Video decoder 300 may determine the values of the scaling factors from the signaled block level syntax element(s).
“[0131] v) Video encoder 200 may signal a set of scaling factors in the bitstream, or the values of the scaling factors may be pre-defined at video encoder 200 and video decoder 300. Video encoder 200 may signal index values at block level, e.g., at CTU level, to specify the values of the scaling factors.”
determining a reconstructed block of the current block according to the scaling factor.
“[0247] Clause 16: A method of filtering decoded video data, the method comprising: decoding a block of video data to form a decoded block; applying a filter to the decoded block to form a filtered block; multiplying samples of the filtered block by a scaling factor to form a refined filtered block; and combining samples of the refined filtered block with corresponding samples of the decoded block.”
Karczewicz does not disclose,
…scaling factor generated by a neural network…
Agashe discloses.
…scaling factor generated by a neural network…
C6 “In an example, the trained model 120 may be understood as a model developed using machine learning or artificial intelligence-based techniques, such as using an artificial neural network (ANN). Thus, determining scaling factors to be used in estimating heights of defect of a wafer may be performed using an ANN. In said example, the scaling factor may be represented by a deep neural network layer.”
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to utilize the teachings of Agashe and apply them on the teachings of Karczewicz to incorporate deriving of the scaling factor by the ANN when performing encoding/decoding in Karczewicz.
One would have been motivated to utilize ANN in Karczewicz to derive scaling factor for the benefit of adapting to the different contents thus offering better reconstruction accuracy.
Unless stated otherwise, the same explanation for the rationale for the following dependent claims applies as given for the independent claim.
2. The method according to claim 1, wherein determining the scaling factor corresponding to the current block using the scaling factor generation network comprises: inputting a filtered picture block and a reconstructed picture block corresponding to the current block into the scaling factor generation network, and outputting the scaling factor.
Karczewicz “[0247] Clause 16: A method of filtering decoded video data, the method comprising: decoding a block of video data to form a decoded block; applying a filter to the decoded block to form a filtered block; multiplying samples of the filtered block by a scaling factor to form a refined filtered block; and combining samples of the refined filtered block with corresponding samples of the decoded block.”
Agashe C6 “In an example, the trained model 120 may be understood as a model developed using machine learning or artificial intelligence-based techniques, such as using an artificial neural network (ANN). Thus, determining scaling factors to be used in estimating heights of defect of a wafer may be performed using an ANN. In said example, the scaling factor may be represented by a deep neural network layer.”
3. The method according to claim 2, wherein a network structure of the scaling factor generation network comprises at least one of a convolutional layer, a nonlinear activation function PRELU layer, a concat layer, a max pooling layer, and a full connect layer.
Agashe C7 “In some cases, the trained model 120 may include a convolutional neural network (CNN). A CNN is a class of neural network that is commonly used in computer vision or image classification systems. In some cases, a CNN may enable processing of digital images with minimal pre-processing. A CNN may be characterized by the use of convolutional (or cross-correlational) hidden layers. These layers apply a convolution operation to the input before signaling the result to the next layer.”
4. The method according to claim 3, wherein a data form of the scaling factor is a two-dimensional matrix or a one-dimensional variable.
Karczewicz “[0130] iv) Video encoder 200 may signal the values of the scaling factors in the bitstream as a syntax element at block level, e.g., at CTU level. Video decoder 300 may determine the values of the scaling factors from the signaled block level syntax element(s).”
5. The method according to claim 4, wherein determining the reconstructed block of the current block according to the scaling factor comprises: determining the reconstructed block of the current block according to the scaling factor, the filtered picture block and the reconstructed picture block.
Karczewicz “[0209] Filter unit 312 may be configured to apply at least one of a neural network-based filter, a neural network-based loop filter, a neural network-based post loop filter, an adaptive in-loop filter, or a pre-defined adaptive in-loop filter to a decoded block of video data to form one or more filtered decoded blocks. Filter unit 312 may then refine the filtered decoded blocks by applying the one or more scaling factors to the filtered decoded blocks, assuming refinement filtering is enabled. For example, filter unit 312 may apply one of formulas (3)-(8) discussed above to refine the filtered decoded blocks.”
Karczewicz “[0247] Clause 16: A method of filtering decoded video data, the method comprising: decoding a block of video data to form a decoded block; applying a filter to the decoded block to form a filtered block; multiplying samples of the filtered block by a scaling factor to form a refined filtered block; and combining samples of the refined filtered block with corresponding samples of the decoded block.”
6. The method according to claim 5, wherein determining the reconstructed block of the current block according to the scaling factor, the filtered picture block and the reconstructed picture block comprises: determining a residual value corresponding to the current block according to the filtered picture block and the reconstructed picture block;
Karczewicz “[0209] Filter unit 312 may be configured to apply at least one of a neural network-based filter, a neural network-based loop filter, a neural network-based post loop filter, an adaptive in-loop filter, or a pre-defined adaptive in-loop filter to a decoded block of video data to form one or more filtered decoded blocks. Filter unit 312 may then refine the filtered decoded blocks by applying the one or more scaling factors to the filtered decoded blocks, assuming refinement filtering is enabled. For example, filter unit 312 may apply one of formulas (3)-(8) discussed above to refine the filtered decoded blocks.”
correcting the residual value according to the scaling factor to obtain a correction result corresponding to the current block; and
Karczewicz “[0247] Clause 16: A method of filtering decoded video data, the method comprising: decoding a block of video data to form a decoded block; applying a filter to the decoded block to form a filtered block; multiplying samples of the filtered block by a scaling factor to form a refined filtered block; and combining samples of the refined filtered block with corresponding samples of the decoded block.”
determining the reconstructed block of the current block according to the correction result and the reconstructed picture block.
Karczewicz “[0209] Filter unit 312 may be configured to apply at least one of a neural network-based filter, a neural network-based loop filter, a neural network-based post loop filter, an adaptive in-loop filter, or a pre-defined adaptive in-loop filter to a decoded block of video data to form one or more filtered decoded blocks. Filter unit 312 may then refine the filtered decoded blocks by applying the one or more scaling factors to the filtered decoded blocks, assuming refinement filtering is enabled. For example, filter unit 312 may apply one of formulas (3)-(8) discussed above to refine the filtered decoded blocks.”
7. The method according to claim 4, wherein determining the scaling factor corresponding to the current block using the scaling factor generation network comprises:
inputting the filtered picture block, the reconstructed picture block and feature information corresponding to the current block into the scaling factor generation network, and outputting the scaling factor,
Karczewicz “[0209] Filter unit 312 may be configured to apply at least one of a neural network-based filter, a neural network-based loop filter, a neural network-based post loop filter, an adaptive in-loop filter, or a pre-defined adaptive in-loop filter to a decoded block of video data to form one or more filtered decoded blocks. Filter unit 312 may then refine the filtered decoded blocks by applying the one or more scaling factors to the filtered decoded blocks, assuming refinement filtering is enabled. For example, filter unit 312 may apply one of formulas (3)-(8) discussed above to refine the filtered decoded blocks.”
Karczewicz “[0247] Clause 16: A method of filtering decoded video data, the method comprising: decoding a block of video data to form a decoded block; applying a filter to the decoded block to form a filtered block; multiplying samples of the filtered block by a scaling factor to form a refined filtered block; and combining samples of the refined filtered block with corresponding samples of the decoded block.”
Agashe C6 “In an example, the trained model 120 may be understood as a model developed using machine learning or artificial intelligence-based techniques, such as using an artificial neural network (ANN). Thus, determining scaling factors to be used in estimating heights of defect of a wafer may be performed using an ANN. In said example, the scaling factor may be represented by a deep neural network layer.”
Karczewicz “[0139] Video encoder 200 may select one of the scaling factors based on an RD-cost, then signal the best scaling factor (i.e., the scaling factor having the best RD-cost) using a direct representation of the scaling factor or an index representing the scaling factor in data of the bitstream. The index can be coded as unary code or truncated binary code, or variable length coded based on the statistical properties of the scaling factor. When the scaling factor has a particular value, which is most probable across all input sequences, then unary code signaling can be chosen as a choice of signaling method, wherein, the least index value can be assigned to most a probable scaling factor value. [0140] vi. The value of scaling factors and offset may be derived at the encoding process and signaled in the bitstream as a syntax element at a block level, e.g., at a CTU level.”
Agashe C 8 “As described herein, the system 102 may be configured to select the scaling factor based on the one or more attributes. For selecting the scaling factor, in an example embodiment, the system 102 may be configured to provide the one or more attributes as an input to the trained model 120. Based on the training, the trained model 120 may then determine the scaling factor based on the values of the one or more attributes.”
wherein the feature information comprises at least one of block partitioning information, prediction information and quantization parameter information.
Karczewicz “[0139] Video encoder 200 may select one of the scaling factors based on an RD-cost, then signal the best scaling factor (i.e., the scaling factor having the best RD-cost) using a direct representation of the scaling factor or an index representing the scaling factor in data of the bitstream. The index can be coded as unary code or truncated binary code, or variable length coded based on the statistical properties of the scaling factor. When the scaling factor has a particular value, which is most probable across all input sequences, then unary code signaling can be chosen as a choice of signaling method, wherein, the least index value can be assigned to most a probable scaling factor value. [0140] vi. The value of scaling factors and offset may be derived at the encoding process and signaled in the bitstream as a syntax element at a block level, e.g., at a CTU level.”
Note: One of ordinarily skilled in the art understands that the RD-cost calculation necessarily involves Qp.
8. The method according to claim 1, wherein the method further comprises: determining third flag information; and performing, in a case where the third flag information indicates that the scaling factor generated by the neural network is used for the current sequence, a determination process of the first flag information.
Karczewicz “[0208] In particular, initially, entropy decoding unit 302 may decode a value for a flag representing whether refinement filtering is to be applied. In the case that refinement filtering is to be applied, entropy decoding unit 302 may entropy decode data representing one or more scaling factors. For example, entropy decoding unit 302 may entropy decode the scaling factors directly or index values corresponding to the scaling factors in a lookup table (e.g., an index table or mapping table). Entropy decoding unit 302 may decode the flag and the scaling factors from a slice header, a picture header, a picture parameter set (PPS), an adaptive parameter set (APS), or other high level syntax structure.”
9. The method according to claim 8, wherein in a case where a value of the third flag information is a first value, the scaling factor generated by the neural network is used for the current sequence; and/or in a case where the value of the third flag information is a second value, the scaling factor generated by the neural network is not used for the current sequence.
Karczewicz “[0208] In particular, initially, entropy decoding unit 302 may decode a value for a flag representing whether refinement filtering is to be applied.”
10. The method according to claim 8, wherein in a case where a value of the first flag information is a first value, the scaling factor generated by the neural network is used for the current color component; and/or in a case where the value of the first flag information is a second value, the scaling factor generated by the neural network is not used for the current color component.
Karczewicz “[0132] vi) Video encoder 200 may separately signal scaling factors for each color component, and video decoder 300 may determine the scaling factors for each color component separately from the signaled values.”
11. The method according to claim 8, wherein in a case where a value of the second flag information is a first value, the scaling factor generated by the neural network is used for the current block; and/or in a case where the value of the second flag information is a second value, the scaling factor generated by the neural network is not used for the current block.
Karczewicz “[0130] iv) Video encoder 200 may signal the values of the scaling factors in the bitstream as a syntax element at block level, e.g., at CTU level. Video decoder 300 may determine the values of the scaling factors from the signaled block level syntax element(s). “
Karczewicz “[0139] Video encoder 200 may select one of the scaling factors based on an RD-cost, then signal the best scaling factor (i.e., the scaling factor having the best RD-cost) using a direct representation of the scaling factor or an index representing the scaling factor in data of the bitstream. The index can be coded as unary code or truncated binary code, or variable length coded based on the statistical properties of the scaling factor. When the scaling factor has a particular value, which is most probable across all input sequences, then unary code signaling can be chosen as a choice of signaling method, wherein, the least index value can be assigned to most a probable scaling factor value. [0140] vi. The value of scaling factors and offset may be derived at the encoding process and signaled in the bitstream as a syntax element at a block level, e.g., at a CTU level.”
12. The method according to claim 9, wherein the first value is 1, and the second value is 0; or the first value is set to true, and the second value is set to false.
Karczewicz “[0208] In particular, initially, entropy decoding unit 302 may decode a value for a flag representing whether refinement filtering is to be applied. In the case that refinement filtering is to be applied, entropy decoding unit 302 may entropy decode data representing one or more scaling factors. For example, entropy decoding unit 302 may entropy decode the scaling factors directly or index values corresponding to the scaling factors in a lookup table (e.g., an index table or mapping table). Entropy decoding unit 302 may decode the flag and the scaling factors from a slice header, a picture header, a picture parameter set (PPS), an adaptive parameter set (APS), or other high level syntax structure.”
13. The method according to claim 4, wherein the method further comprises: training the scaling factor generation network using a scaling factor set to determine a data label from the scaling factor set;
Agashe C6 “In an example, the trained model 120 may be understood as a model developed using machine learning or artificial intelligence-based techniques, such as using an artificial neural network (ANN). Thus, determining scaling factors to be used in estimating heights of defect of a wafer may be performed using an ANN. In said example, the scaling factor may be represented by a deep neural network layer.”
determining a loss of the scaling factor generation network according to output data of the scaling factor generation network and the data label; and updating a network parameter of the scaling factor generation network based on the loss.
Agashe C6 “During the training process, these weights are adjusted to improve the accuracy of the result (i.e., by minimizing a loss function which corresponds in some way to the difference between the current result and the target result).”
Regarding the claims 14-19, they recite elements that are at least included in the claims 1-5, 1 above but in a different claim form and/or encoding/decoding counterpart that are reciprocal. Therefore, the same rationale for the rejection of the claims 1-5, 1 applies.
Regarding the processor, memory and storage medium in the claims, see Karczewicz [0043].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. ZHAO et al. (20220116614) and Gallo et al. (US 20220153262 A1) disclose relevant art related to the subject matter of the present invention.
A shortened statutory period for reply to this action is set to expire THREE MONTHS from the mailing date of this action. An extension of time may be obtained under 37 CFR 1.136(a). However, in no event, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAE N. NOH whose telephone number is (571) 270-0686. The examiner can normally be reached on Mon-Fri 8:30AM-5PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached on (571) 272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAE N NOH/
Primary Examiner
Art Unit 2481