DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 – 7, and 9 - 11 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1 – 5 and, 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Malakhov et al. (WO 2020/036502) (hereinafter Malakhov) in view of Li et al. (US 2016/0234492) (hereinafter Li) in view of Hu et al. (US 2020/0314424) (hereinafter Hu).
Regarding claims 1, and 11, Malakhov teaches a method for encoding an image, a method for decoding an image, and a non-transitory computer readable recording medium storing a bitstream generated by the method for encoding an image, the method comprising:
an importance derivation step of deriving importance of pixels included in a frame in a unit of a pixel, the frame being composed of a plurality of coding tree units, and each of the plurality of coding tree units being composed of a plurality of pixels (e.g. Fig. 8 and pg. 27, lines 9 – 22 and pg. 34, lines 11 – 17: depicting and describing that the system determines whether samples of the frame belong to a region of interest or a region of non-interest, the determination occurring on a coding tree unit (CTU) basis, wherein determining whether a sample belongs to a region of interest is the equivalent deriving the importance of pixels, and wherein the frame is divided into a plurality of CTUs [see, e.g. pg. 14, lines 18 – 29: describing that each picture is divided into a plurality of CTUs]);
a first classification step of first classifying the plurality of coding tree units into a coding tree unit important group or a coding tree unit unimportant group, based on whether an average value of the importance of pixels included in a coding tree unit is greater than or equal to a certain value, in a unit of the coding tree unit (e.g. Fig. 8 and pg. 27, lines 9 – 22 and pg. 34, lines 11 – 17: depicting and describing that the system determines whether samples of the frame belong to a region of interest or a region of non-interest, the determination occurring on a coding tree unit (CTU) basis, wherein classifying a CTU as a region of interest CTU or a region of non-interest CTU is the equivalent of classifying the CTU as a significant CTU or an insignificant CTU; pg. 25, line 14 – pg. 27, line 3: describing that the system determines values for pixels based on pixel motion, pixel texture, or both, the pixel values compared to a threshold value, the system classifying the pixel as a pixel belonging to a region of interest when the pixel value is greater than the threshold value and classifies the pixel as belonging to a region of non-interest when the pixel value is below the threshold value, the classification done based on units of a CTU [see, e.g. pg. 34, lines 11 – 17: describing that the system classifies based on pixels values within a CTU]),
a filter derivation step of obtaining filter sets, the filter sets being composed of filters for sub-blocks obtained by dividing the coding tree unit, each of the sub-blocks including at least one pixel (e.g. pg. 18, lines 1 – 11: describing that the system determines which filters and filter parameters are to be applied to each reconstructed block, the reconstructed blocks being sub-blocks of each CTU [see, e.g. par. 14, lines 18 – 29: describing that each CTU is further divided into sub-blocks], wherein determining which filters and filter parameters are to be applied to each CTU is the equivalent of obtaining filter sets for the plurality of CTUs), and
an encoding step of encoding a filter set group by grouping the filter sets based on results of the classification (e.g. pg. 28, lines 16 – 23: describing that the encoding parameters are based on groups of samples, the groups of samples being whether the CTU belongs to a region of interest or a region of non-interest [see, e.g. pg. 27, lines 20 – 22: describing that segmentation is the classification of samples into the region of interest or the region of non-interest], the encoding parameters including filtering parameters [pg. 18, lines 1 – 11: describing that the system adaptively determines filtering parameters], wherein obtaining coding parameters based whether a CTU belongs to a region of interest or a region of non-interest is the equivalent of obtaining a filter set group based on results of the first classification step),
wherein the filter set group includes 4 filter sets (e.g. pg. 17, lines 27 – 30: describing that the filtering includes a set of at least 4 filters [deblocking filter, SAO filter, sharpening/smoothing filter, and collaborative filter]).
Malakhov does not explicitly teach:
a second classification step of second classifying the plurality of coding tree units based on a direction and strength of edge of the coding tree unit, in a unit of the coding tree unit, wherein obtaining the filter set group is according to a result of the second classification step,
wherein the encoding step further encodes a filter set group syntax representing the filter set group, and
wherein the filter set group syntax is a filter set group index and a filter index, the filter set group index specifying one filter set group among a plurality of filter set groups for one of a coding tree unit important group and the coding tree unit unimportant group in a slice, the filter set group index encoded in a unit of the slice and the filter index encoded in a unit of the coding tree unit.
Li, however, teaches an image encoding method, an image decoding method, and a non-transitory computer readable recording medium storing a bitstream:
a second classification step of second classifying the plurality of coding tree units based on a direction and strength of edge of the coding tree unit, in a unit of the coding tree unit, wherein obtaining the filter set group is according to a result of the second classification step (e.g. pars. 51: describing that each CTU is divided into 4x4 blocks, the blocks then further classified based on a direction and activity, wherein activity is the equivalent of strength of an edge); and
wherein the encoding step further encodes a filter set group syntax representing the filter set group (e.g. par. 91: describing that the system encodes an index indicating filter parameters used in the filtering of the CTU).
Hu, however, teaches an image encoding method, an image decoding method, and a non-transitory computer readable recording medium storing a bitstream:
wherein the filter set group syntax is a filter set group index and a filter index, the filter set group index specifying one filter set group among a plurality of filter set groups for one of a coding tree unit important group and the coding tree unit unimportant group in a slice, the filter set group index encoded in a unit of the slice and the filter index encoded in a unit of the coding tree unit (e.g. pars. 125 – 128 and 180: describing that the system encodes an indication filter sets used within a slice at the slice level [system codes indication of APS’s used for a current slice at the slice level, the APSs containing filter parameter information of each filter], the system further encoding for each block within the slice an index to the particular APS including the filter parameters used for that respective block, wherein the block is the equivalent of the coding tree unit, wherein the indication of filter sets used within a slice is the equivalent of the filter set group index, and wherein the index of filter parameters within the filter set group used for a respective block is the equivalent of the filter index).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Malakhov by adding the teachings of Li in order to perform second classifying the plurality of coding tree units based on a direction and strength of edge of the coding tree unit, in a unit of the coding tree unit, wherein obtaining the filter set group is further based on the second classification step and for the encoding step further encodes a filter set group index representing the filter set group, and by adding the teachings of Hu in order for the filter set group syntax to be a filter set group index and a filter index, the filter set group index specifying one filter set group among a plurality of filter set groups for one of a coding tree unit important group and the coding tree unit unimportant group in a slice, the filter set group index encoded in a unit of the slice and the filter index encoded in a unit of the coding tree unit. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves picture quality, and because the modification improves coding efficiency (Hu, e.g. par. 6: describing a desire to improve coding efficiency).
Turning to claim 2, Malakhov, Li, and Hu teach all of the limitations of claim 1, as discussed above. Malakhov further teaches:
wherein the importance of pixels is a value representing in a unit of a pixel a degree which is referred to importantly to derive a result when performing neural network-based object detection, object segmentation or object tracking for the frame (pg. 25, line 14 – pg. 27, line 3: describing that the system determines values for pixels based on pixel motion, pixel texture, or both, the pixel value used by a neural network to perform object detection, object segmentation, and object tracking for the image).
Regarding claim 3, Malakhov, Li, and Hu teach all of the limitations of claims 1 and 2, as discussed above. Malakhov further teaches:
wherein the coding tree unit important group includes coding tree units among the plurality of coding tree units which an average value of the importance of pixels greater than or equal to a predetermined value, and wherein the coding tree unit unimportant group includes coding tree units among the plurality of coding tree units in which the average value of the importance of pixels in the coding tree unit in the frame is less than the predetermined value (pg. 25, line 14 – pg. 27, line 3: describing that the system determines values for pixels based on pixel motion, pixel texture, or both, the pixel values compared to a threshold value, the system classifying the pixel as a pixel belonging to a region of interest when the pixel value is greater than the threshold value and classifies the pixel as belonging to a region of non-interest when the pixel value is below the threshold value, the classification done based on units of a CTU [see, e.g. pg. 34, lines 11 – 17: describing that the system classifies based on pixels values within a CTU], wherein CTUs classified as a region of interest is the equivalent of the coding tree significant group and CTUs classified as a region of non-interest is the equivalent to the coding tree insignificant group).
Turning to claim 4, Malakhov, Li, and Hu teach all of the limitations of claims 1 - 3, as discussed above. Malakhov further teaches:
wherein a flag representing whether the coding tree unit is in the coding tree unit important group or the coding tree unit unimportant group is encoded in a unit of the coding tree unit (e.g. pg. 34, lines 1 – 17: describing that the system includes an indication of whether a CTU is classified as a region of interest or a non-region of interest, wherein the region of interest is the equivalent of the significant group and the non-region of interest is the equivalent of the insignificant group).
Regarding claim 5, Malakhov, Li, and Hu teach all of the limitations of claims 1 - 4, as discussed above. Malakhov further teaches:
wherein the filters for the sub-blocks obtained by dividing the coding tree unit are derived in different ways depending on whether the coding tree unit is in the coding tree unit important group or the coding tree unit unimportant group (e.g. Fig. 8 and pg. 27, line 9 – pg. 28, line 10: depicting and describing that the system derives coding parameters for the region of interest and region of non-interest separately, wherein the region of interest is equivalent to the coding tree important group and the region of non-interest is the equivalent of the coding tree unit unimportant group, and wherein deriving coding parameters separately based on whether a CTU is classified as a region of interest or region of non-interest reasonably suggests deriving the filters in different ways depending on the classification).
Claim(s) 6 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Malakhov et al. (WO 2020/036502) (hereinafter Malakhov) in view of Li et al. (US 2016/0234492) (hereinafter Li) in view of Hu et al. (US 2020/0314424) (hereinafter Hu) as applied to claim 5 above, and further in view of Jiang et al. (US 2022/0116633) (hereinafter Jiang).
Regarding claim 6, Malakhov, Li, and Hu teach all of the limitations of claims 1 – 5, as discussed above. Malakhov does not explicitly teach:
wherein the filters for the sub-blocks are derived by performing a feature domain minimum error method, and wherein a filter for a sub-block derived by performing the feature domain minimum error method has a smallest average pixel value error between a feature map of a filtered sub-block obtained through a convolution layer by filtering the sub-block with the filter and a feature map of an original sub-block obtained through the convolution layer in the sub-block compared to other filters.
Jiang, however, teaches a method for video encoding:
wherein the filters for the sub-blocks are derived by performing a feature domain minimum error method, and wherein a filter for a sub-block derived by performing the feature domain minimum error method has a smallest average pixel value error between a feature map of a filtered sub-block obtained through a convolution layer by filtering the sub-block with the filter and a feature map of an original sub-block obtained through the convolution layer in the sub-block compared to other filters (e.g. pars. 33 – 43: describing that the system trains a neural network loop filter by minimizing an error between a feature map of an original image block and a feature map the filtered image block [feature discrimination loss], wherein it is known to those of ordinary skill in the art that obtaining a feature map of an image block through a deep neural network necessarily includes obtaining the feature map through a convolutional layer).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Malakhov by adding the teachings of Jiang in order for the filters for the sub-blocks are derived by performing a feature domain minimum error method, and for a filter for a sub-block derived by performing the feature domain minimum error method has a smallest average pixel value error between a feature map of a filtered sub-block obtained through a convolution layer by filtering the sub-block with the filter and a feature map of an original sub-block obtained through the convolution layer in the sub-block compared to other filters. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves visual quality of the reconstructed frame (Jiang, e.g. par. 4: describing a desire to enhance visual quality of reconstructed images using deep neural network based models).
Turning to claim 7, Malakhov, Li, and Hu teach all of the limitations of claims 1 – 5, as discussed above. Malakhov does not explicitly teach:
wherein the filters for the sub-blocks are derived by performing a task error minimum error method, wherein a coefficient of the filters is updated by using a backpropagation method that improves a performance result of a neural network, and wherein a performance result of a neural network is a result of performing the neural network on a frame filtered with a specified initial value of the filter.
Jiang, however, teaches a method of video encoding:
wherein the filters for the sub-blocks are derived by performing a task error minimum error method, wherein a coefficient of the filters is updated by using a backpropagation method that improves a performance result of a neural network, and wherein a performance result of a neural network is a result of performing the neural network on a frame filtered with a specified initial value of the filter (e.g. par. 33 – 43: describing that a neural network loop filter is trained using back propagation to improve the performance of the neural network, the back propagation seeking to minimize a filtering error an original image and the image filtered by the neural network [see, e.g. pars. 34 and 40 - 42: describing that the system compares a loss value between the original image and the neural network filtered image in adjusting weight coefficients of the neural network loop filter], wherein minimizing a filtering error between the original image and the image filtered by the neural network is the equivalent of the task error minimum method, the neural network loop filtering having initial weight coefficients).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Malakhov by adding the teachings of Jiang in order for the filters for the sub-blocks are derived by performing a task error minimum error method, for a coefficient of the filters is updated by using a backpropagation method that improves a performance result of a neural network, and for a performance result of a neural network to be a result of performing the neural network on a frame filtered with a specified initial value of the filter. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves visual quality of the reconstructed frame (Jiang, e.g. par. 4: describing a desire to enhance visual quality of reconstructed images using deep neural network based models).
Claim(s) 9 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Malakhov et al. (WO 2020/036502) (hereinafter Malakhov) in view of Li et al. (US 2016/0234492) (hereinafter Li) in view of Hu et al. (US 2020/0314424) (hereinafter Hu) as applied to claim 1 above, and further in view of Chong et al. (US 2012/0051438) (hereinafter Chong).
Regarding claim 9, Malakhov, Li, and Hu teach all of the limitations of claim 1, as discussed above. Malakhov does not explicitly teach:
wherein the filter set group index is encoded in information encoded in a unit of a slice of the frame.
Chong, however, teaches an image encoding method:
wherein the filter set group index is encoded in information encoded in a unit of a slice of the frame (e.g. pars. 93-95: describing filter information is included in a slice of the frame to indicate filter set information, the filter information representing filter descriptions for filters associated with classified pixel activity in a block).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Malakhov by adding the teachings of Chong in order for the filter set group index is encoded in information encoded in a unit of a slice of the frame. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves visual quality of the reconstructed frame (Chong, e.g. par. 10: describing a desire to improve reconstructed video quality).
Turning to claim 10, Malakhov, Li, Hu, and Chong teach all of the limitations of claims 1 and 9, as discussed above. Malakhov does not explicitly teach:
wherein a maximum number of the filter set group indexes encoded in the information encoded in the unit of the slice is 4.
Chong, however, teaches an image encoding method:
wherein a maximum number of the filter set group indexes encoded in the information encoded in the unit of the slice is 4 (e.g. pars. 93 – 95: describing that the filter information includes a maximum number of filters in a filter set in a slice, the maximum number of filters being 4).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Malakhov by adding the teachings of Chong in order for a maximum number of the filter set group indexes encoded in the information encoded in the unit of the slice is 4. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves visual quality of the reconstructed frame (Chong, e.g. par. 10: describing a desire to improve reconstructed video quality).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHANIKA M BRUMFIELD whose telephone number is (571)270-3700. The examiner can normally be reached M-F 8:30 - 5 PM AWS.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Czekaj can be reached at 571-272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SHANIKA M. BRUMFIELD
Examiner
Art Unit 2487
/SHANIKA M BRUMFIELD/Examiner, Art Unit 2487
/Dave Czekaj/Supervisory Patent Examiner, Art Unit 2487
Prosecution Insights