DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDSs) submitted on 03/11/2025 and 07/09/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Interpretation
For purposes of examination and anyone reviewing the Office action and/or prosecution history, it is noted that the limitations of claim 63 were considered under 35 U.S.C. §112(f) and, as they use the term ‘means’, the presumption was met.
Claim Objections
Claim 62 is objected to because of the following informalities: On line 2, ‘decoder’ would appear to actually be ‘encoder’ based on context of this and claim 30. Appropriate correction is required if needed.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-5, 7-9 and 13 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by SHAO T., et al., "EE1-related: Further complexity reduction on the joint EE1-0 (LOP.2) unified filter", JVET-AF0071-V1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 32nd Meeting, Hannover, DE, 13-20 October 2023, pp. 1-6 referred to as SHAO hereinafter.
Regarding claim 1, SHAO shows a method of coding video data, the method comprising:
receiving a picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.);
reconstructing the picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.); and
performing a neural network (NN)-based filter process on one or more blocks of the reconstructed picture of video data using an NN-based filter (Figures 1, 3),
wherein the NN-based filter includes a pair of backbone blocks (Figure 3, NC and/or NY), each of the pair of backbone blocks comprising a three-component one-dimensional (1D) decomposition of a multi-dimensional convolution (Figure 3, wherein the Back Bone Block comprises 3 (and more) layers.), wherein the 1D decomposition includes at least one layer with feature channel reduction (Figure 3, wherein C21 has a reduced number of channels.).
Regarding claim 2, SHAO shows the limitations of claim 1 as applied above, and further shows wherein an input to the three-component 1D decomposition is defined by a height (h), width (w), and a number of channels (C), and wherein the three-component 1D decomposition of the multi-dimensional convolution includes: a 1×1 convolution having C output channels, a first separable convolution in a first direction having C1 output channels, wherein C1 is less than C, and a second separable convolution in a second direction having C output channels (See Figure 3, the Back Bone Block portion is self-explanatory and discloses all of the above.).
Regarding claim 3, SHAO shows the limitations of claim 2 as applied above, and further shows wherein the three-component 1D decomposition of the multi-dimensional convolution further includes an activation function (Figure 3, PReLU).
Regarding claim 4, SHAO shows the limitations of claim 3 as applied above, and further shows wherein the activation function is a parametric rectified linear unit (PReLU) (Figure 3, PReLU).
Regarding claim 5, SHAO shows the limitations of claim 1 as applied above, and further shows wherein the pair of backbone blocks includes a first backbone block and a second backbone block, wherein an input to the first backbone block is defined by a height (h), width (w), and a number of channels (C), and wherein the first backbone block includes: a first 1×1 convolution having C output channels, a first separable convolution in a first direction having C1 output channels, wherein C1 is less than C, and a second separable convolution in a second direction having C output channels, and wherein the second backbone block includes: a second 1×1 convolution having C output channels, a third separable convolution in the second direction having C1 output channels, wherein C1 is less than C, and a fourth separable convolution in the first direction having C output channels (See Figure 3, the Back Bone Block portion is self-explanatory and discloses all of the above. This would be covered by C1UV & C1Y OR NC OR NY.).
Regarding claim 7, SHAO shows the limitations of claim 5 as applied above, and further shows wherein the first direction is a vertical directional and the second direction is a horizontal direction (Figure 3, h,w.).
Regarding claim 8, SHAO shows the limitations of claim 5 as applied above, and further shows wherein the first 1×1 convolution is positioned before the first separable convolution (Figure 3, Back Bone Block for C1UV), and wherein the second 1×1 convolution is positioned before the third separable convolution (Figure 3, Back Bone Block for C1Y).
Regarding claim 9, SHAO shows the limitations of claim 5 as applied above, and further shows wherein the first 1×1 convolution is positioned after the second separable convolution (Figure 3, Back Bone Block for C1UV), and wherein the second 1×1 convolution is positioned before the fourth separable convolution (Figure 3, Back Bone Block for C1Y).
Regarding claim 13, SHAO shows the limitations of claim 1 as applied above, and further shows wherein the NN-based filter further includes a fusion block having two separable convolutions that each apply spatial downsampling (Figure 3, see fusion block.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 30, 32-36, 38-40, 44, 61 and 63-64 are rejected under 35 U.S.C. 103 as being unpatentable over SHAO in view of GALPIN et al., (US 20220141456 A1) referred to as GALPIN hereinafter.
Regarding claim 30, SHAO shows the limitations of claim 1 as applied above, however failing to but GALPIN further shows wherein coding comprises decoding (Figure 10, 1030, a Decoder inherently performs decoding operations in its normal and routine operation.), and wherein the method further comprises: displaying a decoded picture that includes the one or more blocks processed by the NN-based filter (Figure 10, 1030, a Display inherently performs displaying operations in its normal and routine operation.).
Both SHAO and GALPIN are analogous art to the claimed invention in that they both participate in video compression and more specifically using Neural Networks to increase coding efficiency.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in the spirit of GILPIN to combine the above prior art elements according to known methods to yield predictable results.
SHAO, in view of GALPIN, includes each element claimed, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. One of ordinary skill in the art would have combined the elements as claimed by known methods. Merely having a decoder as well as displaying a resultant decoded picture is a well-known technique in the art, and that in combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable in so far as that it is fundamental that decoders filter and decode and pictures are meant to be displayed. Both facts are beyond reproach and capable of instant and unquestioned support as needed.
Regarding claim 32, SHAO discloses an apparatus configured to code video data, the apparatus comprising the functionality of:
receive a picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.);
reconstruct the picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.); and
perform a neural network (NN)-based filter process on one or more blocks of the reconstructed picture of video data using an NN-based filter (Figures 1, 3), wherein the NN-based filter includes a pair of backbone blocks (Figure 3, NC and/or NY), each of the pair of backbone blocks comprising a three-component one-dimensional (1D) decomposition of a multi-dimensional convolution (Figure 3, wherein the Back Bone Block comprises 3 (and more) layers.), wherein the 1D decomposition includes at least one layer with feature channel reduction (Figure 3, wherein C21 has a reduced number of channels.).
However, SHAO fails to but GALPIN does show a memory (Figure 10, 1020); and
one or more processors in communication with the memory (Figure 10, 1010), the one or more processors configured to perform the above.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in the spirit of GILPIN to combine the above prior art elements according to known methods to yield predictable results.
Both SHAO and GALPIN are analogous art to the claimed invention in that they both participate in video compression and more specifically using Neural Networks to increase coding efficiency.
SHAO, in view of GALPIN, includes each element claimed, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. One of ordinary skill in the art would have combined the elements as claimed by known methods. Merely having a memory and processor is a well-known technique in the art for carrying out computer-related functions, and that in combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable in so far as that it is fundamental that computing devices have both memory and processor(s). Both facts are beyond reproach and capable of instant and unquestioned support as needed.
Regarding claim 33, SHAO shows the limitations of claim 32 as applied above, and further shows wherein an input to the three-component 1D decomposition is defined by a height (h), width (w), and a number of channels (C), and wherein the three-component 1D decomposition of the multi-dimensional convolution includes: a 1×1 convolution having C output channels, a first separable convolution in a first direction having C1 output channels, wherein C1 is less than C, and a second separable convolution in a second direction having C output channels (See Figure 3, the Back Bone Block portion is self-explanatory and discloses all of the above.).
Regarding claim 34, SHAO shows the limitations of claim 33 as applied above, and further shows wherein the three-component 1D decomposition of the multi-dimensional convolution further includes an activation function (Figure 3, PReLU).
Regarding claim 35, SHAO shows the limitations of claim 34 as applied above, and further shows wherein the activation function is a parametric rectified linear unit (PReLU) (Figure 3, PReLU).
Regarding claim 36, SHAO shows the limitations of claim 32 as applied above, and further shows wherein the pair of backbone blocks includes a first backbone block and a second backbone block, wherein an input to the first backbone block is defined by a height (h), width (w), and a number of channels (C), and wherein the first backbone block includes: a first 1×1 convolution having C output channels, a first separable convolution in a first direction having C1 output channels, wherein C1 is less than C, and a second separable convolution in a second direction having C output channels, and wherein the second backbone block includes: a second 1×1 convolution having C output channels, a third separable convolution in the second direction having C1 output channels, wherein C1 is less than C, and a fourth separable convolution in the first direction having C output channels (See Figure 3, the Back Bone Block portion is self-explanatory and discloses all of the above. This would be covered by C1UV & C1Y OR NC OR NY.).
Regarding claim 38, SHAO shows the limitations of claim 36 as applied above, and further shows wherein the first direction is a vertical directional and the second direction is a horizontal direction (Figure 3, h,w.).
Regarding claim 39, SHAO shows the limitations of claim 36 as applied above, and further shows wherein the first 1×1 convolution is positioned before the first separable convolution (Figure 3, Back Bone Block for C1UV), and wherein the second 1×1 convolution is positioned before the third separable convolution (Figure 3, Back Bone Block for C1Y).
Regarding claim 40, SHAO shows the limitations of claim 36 as applied above, and further shows wherein the first 1×1 convolution is positioned after the second separable convolution (Figure 3, Back Bone Block for C1UV), and wherein the second 1×1 convolution is positioned before the fourth separable convolution (Figure 3, Back Bone Block for C1Y).
Regarding claim 44, SHAO shows the limitations of claim 32 as applied above, and further shows wherein the NN-based filter further includes a fusion block having two separable convolutions that each apply spatial downsampling (Figure 3, see fusion block.).
Regarding claim 61, SHAO shows the limitations of claim 32 as applied above, however failing to but GALPIN further shows further shows wherein the apparatus is a video decoder (Figure 10, 1030), and wherein the apparatus further comprises: a display configured to display a decoded picture that includes the one or more blocks processed by the NN-based filter (Figure 10, 1100).
Both SHAO and GALPIN are analogous art to the claimed invention in that they both participate in video compression and more specifically using Neural Networks to increase coding efficiency.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in the spirit of GILPIN to combine the above prior art elements according to known methods to yield predictable results.
SHAO, in view of GALPIN, includes each element claimed, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. One of ordinary skill in the art would have combined the elements as claimed by known methods. Merely having a decoder as well as displaying a resultant decoded picture is a well-known technique in the art, and that in combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable in so far as that it is fundamental that decoders filter and decode and pictures are meant to be displayed. Both facts are beyond reproach and capable of instant and unquestioned support as needed.
Regarding claim 63, SHAO discloses an apparatus configured to code video data, the apparatus comprising:
receiving a picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.);
reconstructing the picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.); and
performing a neural network (NN)-based filter process on one or more blocks of the reconstructed picture of video data using an NN-based filter (Figures 1, 3), wherein the NN-based filter includes a pair of backbone blocks (Figure 3, NC and/or NY), each of the pair of backbone blocks comprising a three-component one-dimensional (1D) decomposition of a multi-dimensional convolution(Figure 3, wherein the Back Bone Block comprises 3 (and more) layers.), wherein the 1D decomposition includes at least one layer with feature channel reduction (Figure 3, wherein C21 has a reduced number of channels.).
However, SHAO fails to but GALPIN does show means (Figure 10, 1030) for carrying out the above functions.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in the spirit of GILPIN to combine the above prior art elements according to known methods to yield predictable results.
Both SHAO and GALPIN are analogous art to the claimed invention in that they both participate in video compression and more specifically using Neural Networks to increase coding efficiency.
SHAO, in view of GALPIN, includes each element claimed, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. One of ordinary skill in the art would have combined the elements as claimed by known methods. Merely having a memory and processor (Means) is a well-known technique in the art for carrying out computer-related functions, and that in combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable in so far as that it is fundamental that computing devices have both memory and processor(s). Both facts are beyond reproach and capable of instant and unquestioned support as needed.
Regarding claim 63, SHAO discloses instructions (Paragraph [0016]) configured to:
receive a picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.);
reconstruct the picture of video data (Figures 1, 3; RecEXTY, etc., wherein receiving a picture and reconstructing a picture are fundamentally integrated with the in-loop filtering procedure in the current coding schemes.); and
perform a neural network (NN)-based filter process on one or more blocks of the reconstructed picture of video data using an NN-based filter (Figures 1, 3), wherein the NN-based filter includes a pair of backbone blocks (Figure 3, NC and/or NY), each of the pair of backbone blocks comprising a three-component one-dimensional (1D) decomposition of a multi-dimensional convolution (Figure 3, wherein the Back Bone Block comprises 3 (and more) layers.), wherein the 1D decomposition includes at least one layer with feature channel reduction (Figure 3, wherein C21 has a reduced number of channels.).
However, SHAO fails to but GALPIN does show a non-transitory computer-readable storage medium storing instructions (Paragraph [0016]) that, when executed, cause one or more processors of a device configured to carry out the above functions.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in the spirit of GILPIN to combine the above prior art elements according to known methods to yield predictable results.
Both SHAO and GALPIN are analogous art to the claimed invention in that they both participate in video compression and more specifically using Neural Networks to increase coding efficiency.
SHAO, in view of GALPIN, includes each element claimed, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. One of ordinary skill in the art would have combined the elements as claimed by known methods. Merely having a non-transitory computer-readable storage medium storing instructions is a well-known technique in the art for carrying out computer-related functions, and that in combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable in so far as that it is fundamental that computing devices have a non-transitory computer-readable storage medium storing instructions. This fact is beyond reproach and capable of instant and unquestioned support as needed.
Claim(s) 31 and 62 are rejected under 35 U.S.C. 103 as being unpatentable over SHAO in view of GALPIN and further in view of OFFICIAL NOTICE, referred to as OFFICIAL NOTICE hereinafter.
Regarding claim 31, SHAO shows the limitations of claim 1 as applied above, however failing to but GALPIN further shows wherein coding comprises encoding (Figure 10, 1030, an Encoder inherently performs encoding operations in its normal and routine operation.).
Both SHAO and GALPIN are analogous art to the claimed invention in that they both participate in video compression and more specifically using Neural Networks to increase coding efficiency.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in the spirit of GILPIN to combine the above prior art elements according to known methods to yield predictable results.
SHAO, in view of GALPIN, includes each element claimed, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. One of ordinary skill in the art would have combined the elements as claimed by known methods. Merely having an encoder as well as a provided picture is a well-known technique in the art, and that in combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable in so far as that it is fundamental that encoders encode pictures and some form of source image is needed to do so. Both facts are beyond reproach and capable of instant and unquestioned support as needed.
Further however, SHAO in view of GALPIN fails to specifically but OFFICIAL NOTICE dictates capturing the picture of video data with a camera (OFFICIAL NOTICE).
The examiner is taking the position that using a camera to supply said picture would be a well-known, routine and conventional fact in the art of video compression.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in view of GALPIN in the spirit of the common understanding of the art so as to be able to effectively transmit pictures, videos and images captured by cameras. This is a fundamental necessity in today’s current climate of streaming media and the combination demonstrates unquestioned validity and obviousness.
Regarding claim 62, SHAO shows the limitations of claim 32 as applied above, however failing to but GALPIN further shows wherein coding comprises encoding (Figure 10, 1030, an Encoder inherently performs encoding operations in its normal and routine operation.).
Both SHAO and GALPIN are analogous art to the claimed invention in that they both participate in video compression and more specifically using Neural Networks to increase coding efficiency.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in the spirit of GILPIN to combine the above prior art elements according to known methods to yield predictable results.
SHAO, in view of GALPIN, includes each element claimed, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. One of ordinary skill in the art would have combined the elements as claimed by known methods. Merely having an encoder as well as a provided picture is a well-known technique in the art, and that in combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable in so far as that it is fundamental that encoders encode pictures and some form of source image is needed to do so. Both facts are beyond reproach and capable of instant and unquestioned support as needed.
Further however, SHAO in view of GALPIN fails to specifically but OFFICIAL NOTICE dictates capturing the picture of video data with a camera (OFFICIAL NOTICE).
The examiner is taking the position that using a camera to supply said picture would be a well-known, routine and conventional fact in the art of video compression.
Therefore, it would have been obvious to one possessing ordinary skill in the art before the effective filing date of the claimed invention to modify SHAO in view of GALPIN in the spirit of the common understanding of the art so as to be able to effectively transmit pictures, videos and images captured by cameras. This is a fundamental necessity in today’s current climate of streaming media and the combination demonstrates unquestioned validity and obviousness.
Allowable Subject Matter
Claims 6, 10-12, 14-29, 37, 41-43 and 45-60 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see the Notice of References Cited (PTO-892) for other references related to in-loop filtering and Neural Networks.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN W. RIDER whose telephone number is (571)270-1068. The examiner can normally be reached Monday-Friday, 7.00 am - 4.30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jamie J Atala can be reached at (571) 272-7384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JUSTIN W. RIDER
Primary Patent Examiner
Art Unit 2486
/Justin W Rider/Primary Patent Examiner, Art Unit 2486