DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 5-6, 8-10, 13 and 16-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wan et al. (US 2021/0368211).
Regarding claim 1, Wan discloses:
A method for video processing, comprising:
applying, for a conversion between a current video unit of a video and a bitstream of the video, a neural network filter to the current video unit at least based on auxiliary information associated with the current video unit (Wan: Fig 1-2; video coding/decoding; convolutional neural network filter 201; [0033]-[0038]; convolutional neural network filter 201 fuses auxiliary information such as block dividing information and/or QP information with multiple input color components), the auxiliary information including at least one of:
prediction information of the current video unit (Wan: [0065]; first auxiliary information includes Motion Vector (MV) information and prediction direction information; [0102]),
partitioning information of the current video unit (Wan: [0064]-[0065]; first auxiliary information includes block dividing information, such as CU dividing information and/or CTU dividing information; [0102]), or
coding information of a previously coded video unit (Wan: [0064]-[0065]; first auxiliary information includes quantization parameter information, MV information, prediction direction information; [0102]); and
performing the conversion based on the applying (Wan: Fig 1-2; video coding/decoding; entropy encoding unit 105; [0033]-[0038]).
Regarding claim 5, Wan discloses:
The method of claim 1, wherein the prediction information of the current video unit comprises at least one of:
a prediction sample of the current video unit, or
a prediction mode of the current video unit (Wan: [0065]; Motion Vector (MV) information reflects inter prediction mode; prediction direction information; [0102]).
Regarding claim 6, Wan discloses:
The method of claim 1, wherein the partitioning information of the current video unit comprises a partitioning boundary of the current video unit (Wan: [0064]-[0065]; block dividing information, such as CU dividing information and/or CTU dividing information; [0102]).
Regarding claim 8, Wan discloses:
The method of claim 1, wherein a first color component and a second color component of the current video unit share the same auxiliary information, or the first and second color components are allowed to share the same auxiliary information (Wan: [0069]; [0075]).
Regarding claim 9, Wan discloses:
The method of claim 1, wherein a first color component and a second color component of the current video unit use different auxiliary information, or the first and second color components are allowed to use different auxiliary information (Wan: [0070]; [0086]).
Regarding claim 10, Wan discloses:
The method of claim 1, wherein a first chroma component and a second chroma component of the current video unit use or are allowed to use the same auxiliary information, and a luma component of the current video unit uses first auxiliary information different from second auxiliary information used by the first and second chroma components (Wan: [0086]).
Regarding claim 13, Wan discloses:
The method of claim 1, wherein the current video unit comprises a picture or a slice (Wan: Fig 1-2; [0033]-[0038]; [0064]; [0103]).
Regarding claim 16, Wan discloses:
The method of claim 1, wherein the conversion includes encoding the current video unit into the bitstream (Wan: Fig 1-2; video coding/decoding; convolutional neural network filter 201; [0033]-[0038]).
Regarding claim 17, Wan discloses:
The method of claim 1, wherein the conversion includes decoding the current video unit from the bitstream (Wan: Fig 1-2; video coding/decoding; convolutional neural network filter 201; [0033]-[0038]).
Regarding claims 18-20, Wan discloses the apparatus, CRM and bitstream as shown above with respect to claim 1 (Wan: abstract; [0019]-[0021]; [0146]-[0160]).The only portion of the language in claim 20 that is given patentable weight is the preamble limitation, because this is an article of manufacture. All of the other limitations of claim 20 body are not given patentable weight. This is because there is no functional relationship between the bitstream and any related computer.
According to MPEP 2111.05, “To be given patentable weight, the printed matter and associated product must be in a functional relationship. A functional relationship can be found where the printed matter performs some function with respect to the product to which it is associated.” Furthermore, for machine-readable media, “When determining the scope of a claim directed to a computer-readable medium containing certain programming, the examiner should first look to the relationship between the programming and the intended computer system. Where the programming performs some function with respect to the computer with which it is associated, a functional relationship will be found. For instance, a claim to computer-readable medium programmed with attribute data objects that perform the function of facilitating retrieval, addition, and removal of information in the intended computer system, establishes a functional relationship such that the claimed attribute data objects are given patentable weight. See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 1035. However, where the claim as a whole is directed to conveying a message or meaning to a human reader independent of the intended computer system, and/or the computer-readable medium merely serves as a support for information or data, no functional relationship exists. For example, a claim to a memory stick containing tables of batting averages, or tracks of recorded music, utilizes the intended computer system merely as a support for the information. Such claims are directed toward conveying meaning to the human reader rather than towards establishing a functional relationship between recorded data and the computer.”
The body limitations in claim 20 perform no function with respect to a computer with which it is associated. In other words, the body limitations in claim 20 do not perform a function or cause the associated computer to perform a function. An applying and generating method has already taken place. The bitstream does not perform an applying or generating method. The data stream also does not cause an associated computer to perform an applying or generating method.
Furthermore, the limitations in claim 20 convey a message or meaning to a human reader independent of the intended computer system. The message is a bitstream that is to be seen by a human reader after decoding. The bitstream in claim 20 merely serves as a support for information and data. Additionally, any computer-readable medium that stores such a bitstream is merely a support for data
While the body of claim 20 is not given patentable weight, the claim limitations are rejected using prior art in order to compact prosecution. In order for the body of claim 20 to be given patentable weight, Examiner suggest amending the claim to recite function, such as the function in claims 1-19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2-4, 7, 11-12 and 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wan et al. (US 2021/0368211) in view of SHAO et al. (SHAO et al., "AHG9: On auxiliary input and separate colour description in the neural- network post-filter characteristics SEI message", Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTCJ/SC29, 27th Meeting, by teleconference, 13-22 July 2022, Document: JVET-AA0J00-v3, July 14, 2022, 15 pages.).
Regarding claim 2,
Wan teaches:
The method of claim 1 (as shown above),
Wan fails to teach:
further comprising:
determining whether a condition for excluding the auxiliary information from an input of the neural network is satisfied based on at least one syntax element in the bitstream; and in accordance with a determination that the condition is satisfied, applying the neural network to the current video unit without inputting the auxiliary information to the neural network filter.
Shao teaches:
determining whether a condition for excluding the auxiliary information from an input of the neural network is satisfied based on at least one syntax element in the bitstream; and in accordance with a determination that the condition is satisfied, applying the neural network to the current video unit without inputting the auxiliary information to the neural network filter (Shao: pg 1; abstract, syntax elements for the neural-network post-filter characteristics SEI message; section 1; auxiliary input data may be present in only one configuration of a luma-chroma input tensor (nnpfc_inp_order_idc equal to 3) and not in any other luma-only, chroma-only, or luma-chroma configuration (nnpfc_inp_order_idc equal to 0, 1, and 2, respectively).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Shao with Wan. Using the specific coding techniques of Shao would enhance the efficiency, operability and/or customization options of Wan. Additionally, this is the application of a known technique, using specific coding techniques, to a known device ready for improvement, the Wan device, to yield predictable results.
Regarding claim 3,
Wan in view of Shao teaches:
The method of claim 2, wherein the at least one syntax element comprises:
a first syntax element for indicating a rule of ordering sample arrays of a cropped decoded output picture as an input to the neural network filter (Shao: pg 4, section 3.1.2; Cropped decoded output picture width and height in units of luma samples, denoted herein by InpPicWidthlnLumaSamples and InpPicHeightlnLumaSamples, respectively; Luma sample array CroppedYPic[ y ] [ x] and chroma sample arrays CroppedCbPic[ y ] [ x] and CroppedCrPic[ y ] [ x], when present, of the cropped decoded output picture for vertical coordinates y and horizontal coordinates x, where the top-left comer of the sample array has coordinates y equal to 0 and x equal to 0; pg 5, nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples section),
a second syntax element for specifying that a dimension in an input tensor to the neural network filter and an output tensor resulting from the neural network filter is used for a channel (Shao: pg 5, nnpfc_component_last_flag equal to 0 specifies that the second dimension in the input tensor inputTensor to the post-processing filter and the output tensor outputTensor resulting from the post-processing filter is used for the channel. nnpfc component_ last_ flag equal to 1 specifies that the last dimension in the input tensor inputTensor to the post-processing filter and the output tensor outputTensor resulting from the post-processing filter is used for the channel), and
a third syntax element for indicating whether the auxiliary information is present in an input tensor of the neural network filter (Shao: pg 6, nnpfc_auxiliary_inp_idc not equal to 0 specifies auxiliary input data is present in the input tensor of the neuralnetwork post-filter. nnpfc_auxiliary inp_idc equal to 0 indicates that auxiliary input data is not present in the input tensor. nnpfc auxiliary_ inp idc equal to 1 specifies that auxiliary input data is derived as specified in Table 23), and
wherein the condition is that the first syntax element is 3, the second syntax element is 0, and the third syntax element is 0 (Shao: pg 1; abstract, section 1; auxiliary input data may be present in only one configuration of a luma-chroma input tensor (nnpfc_inp_order_idc equal to 3) and not in any other luma-only, chroma-only, or luma-chroma configuration (nnpfc_inp_order_idc equal to 0, 1, and 2, respectively; pg 7; nnpfc_inp_order_idc section; nnpfc_inp_order_idc from 0 to 3; table 21; pg 12, nnpfc_out_order_idc section; nnpfc_out_order_idc from 0 to 3); pg 5, nnpfc_component_last_flag equal to 0; pg 6, nnpfc_auxiliary_inp_idc equal to 0).
Regarding claim 4,
Wan teaches:
The method of claim 1 (as shown above),
Wan fails to teach:
wherein the neural network filter comprises a neural network post-processing filter.
Shao teaches:
wherein the neural network filter comprises a neural network post-processing filter (Shao: pg 1, abstract, pg 4, section 3.1.2; SEI message specifies a neural network that may be used as a post-processing filter).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Shao with Wan. Using the specific coding techniques of Shao would enhance the efficiency, operability and/or customization options of Wan. Additionally, this is the application of a known technique, using specific coding techniques, to a known device ready for improvement, the Wan device, to yield predictable results.
Regarding claim 7,
Wan teaches:
The method of claim 1 (as shown above),
Wan fails to teaches:
wherein the coding information of the previously coded video unit comprises: a sample of at least one of a collocated block or a motion compensated block in the previously coded video unit, the collocated block being collocated with a current video block in the current video unit, the motion compensated block being associated with the current video block.
Shao teaches:
wherein the coding information of the previously coded video unit comprises: a sample of at least one of a collocated block or a motion compensated block in the previously coded video unit, the collocated block being collocated with a current video block in the current video unit, the motion compensated block being associated with the current video block (Shao: pg 4; SEI message with variables for luma and chroma samples).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Shao with Wan. Using the specific coding techniques of Shao would enhance the efficiency, operability and/or customization options of Wan. Additionally, this is the application of a known technique, using specific coding techniques, to a known device ready for improvement, the Wan device, to yield predictable results.
Regarding claim 11,
Wan teaches:
The method of claim 1, wherein at least one matrix associated with the neural network filter comprises at least one of: a luma component, a first chroma component, or a second chroma component (Wan: [0090]),
Wan fails to teaches:
wherein the at least one matrix comprises at least one of: an input matrix in an input tensor of the neural network filter, or an output matrix in an output tensor of the neural network filter, or
wherein the at least one matrix comprises a chroma matrix, the number of channels of an input tensor or an output tensor of the neural network filter is 1, or
wherein the at least one matrix comprises a chroma matrix and a luma matrix including a luma component, the number of channels of an input tensor or an output tensor of the neural network filter is 2, or
wherein the at least one matrix comprises a chroma matrix and four luma matrices including luma components, the number of channels of an input tensor or an output tensor of the neural network filter is 5, wherein a chroma format of the current video unit is 4:2:0.
Shao teaches:
The method of claim 1, wherein at least one matrix associated with the neural network filter comprises at least one of: a luma component, a first chroma component, or a second chroma component (Shao: pg 7, Table 21),
wherein the at least one matrix comprises at least one of: an input matrix in an input tensor of the neural network filter, or an output matrix in an output tensor of the neural network filter (Shao: pg 7, Table 21), or
wherein the at least one matrix comprises a chroma matrix, the number of channels of an input tensor or an output tensor of the neural network filter is 1, or
wherein the at least one matrix comprises a chroma matrix and a luma matrix including a luma component, the number of channels of an input tensor or an output tensor of the neural network filter is 2, or
wherein the at least one matrix comprises a chroma matrix and four luma matrices including luma components, the number of channels of an input tensor or an output tensor of the neural network filter is 5, wherein a chroma format of the current video unit is 4:2:0.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Shao with Wan. Using the specific coding techniques of Shao would enhance the efficiency, operability and/or customization options of Wan. Additionally, this is the application of a known technique, using specific coding techniques, to a known device ready for improvement, the Wan device, to yield predictable results.
Regarding claim 12,
Wan teaches:
The method of claim 1, wherein at least one type of visual quality improvement of the neural network filter is included in the bitstream (Wan: [0064]-[0065]; first auxiliary information includes quantization parameter information, MV information, prediction direction information; [0102]),
Wan fails to teaches:
wherein the at least one type is included in a neural network post-filter characteristics (NNPFC) supplemental enhancement information (SEI) message.
Shao teaches:
The method of claim 1, wherein at least one type of visual quality improvement of the neural network filter is included in the bitstream, wherein the at least one type is included in a neural network post-filter characteristics (NNPFC) supplemental enhancement information (SEI) message (Shao: pg 5, Table 20 and note 1; nnpfc_purpose with sei syntax; abstract).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Shao with Wan. Using the specific coding techniques of Shao would enhance the efficiency, operability and/or customization options of Wan. Additionally, this is the application of a known technique, using specific coding techniques, to a known device ready for improvement, the Wan device, to yield predictable results.
Regarding claim 14,
Wan teaches:
The method of claim 1 (as shown above),
Wan fails to teaches:
further comprising: receiving a patch width and a patch height from an external source; and determining a patch size for an input of the neural network filter based on the patch width and the patch height.
Shao teaches:
further comprising: receiving a patch width and a patch height from an external source; and determining a patch size for an input of the neural network filter based on the patch width and the patch height (Shao: pg 8; nnpfc _patch_ width_ minus 1 and nnpfc patch_ height_ minus 1 in various syntax and variables).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Shao with Wan. Using the specific coding techniques of Shao would enhance the efficiency, operability and/or customization options of Wan. Additionally, this is the application of a known technique, using specific coding techniques, to a known device ready for improvement, the Wan device, to yield predictable results.
Regarding claim 15,
Wan in view of Shao teaches:
The method of claim 14, wherein the patch width is a positive integer multiple of a sum of a fourth syntax element in the bitstream and a predefined value, and the patch height is a positive integer multiple of a sum of a fifth syntax element in the bitstream and the predefined value (Shao: pg 8; nnpfc _patch_ width_ minus 1 and nnpfc patch_ height_ minus 1 in various syntax and variables; positive integer multiple).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure is found in the attached Notice of References Cited.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES M PONTIUS whose telephone number is (571)270-7687. The examiner can normally be reached M-Th 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath V Perungavoor can be reached at (571)272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES M PONTIUS/Primary Examiner, Art Unit 2488
Prosecution Insights