DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 10 is objected to because of the following informalities:
Regarding claim 10; claim 10 recites the limitations “obtain the AI-encoded image.” However, claim 10 depends on claim 9 which is directed to AI decoding, and the context of the claim clearly requires “AI-decoded image.”
Appropriate correction(s) is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (KR102287947B1, copy of English Translation provided cited in PTO-892), hereinafter referred to as Lee, in view of Kuo et al. (US 2020/0014603 Al), hereinafter referred to as Kuo.
Regarding claim 1, Lee discloses a eelectronic device configured to process an image by using artificial intelligence (AI) encoding, comprising (p. 14: “AI downscaling unit 612 may store a plurality of settable DNN configuration information in the first DNN … and AI downscales the original image 105 through the first DNN set”):
memory storing a first neural network model that is trained; and at least one processor configured, wherein the memory stores instructions that, when executed by the at least one processor, cause the electronic device to (p. 1 “AI decoding apparatus according to an embodiment includes a memory for storing one or more instructions and a processor executing the one or more instructions stored in the memory, wherein the processor is configured to”):
a communication interface (p.1 “transmitted through a communication channel;” p.7 “communication unit 212;”)
obtain a second image based on a first image including pixel information (p.14 “The AI downscaler 612 may acquire the AI downscaled first image 115 from the original image 105 through the first DNN”)
input the second image into the first neural network model and obtain a first residual image (p. 14 “the AI data may include difference information between the original image 105 and the first image 115”),
Lee does not explicitly disclose including luminance information; including luminance residual information
However, Kuo from the same or similar endeavor of image processing discloses including luminance information (¶[0127] discloses that raw image is YUV420. See also ¶¶[0032]-[0040]); including luminance residual information (¶[0129] discloses that the X_y is the Y of the input image with YUV420 format. See also ¶¶[0032]-[0040])
It would have been obvious to the person of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings disclosed by Lee to add the teachings of Kuo as above, in order to enhance quality of media transmitted via network (Kuo, [0050]).
Furthermore, Lee discloses obtain a second residual image including pixel residual information based on the first residual image (p. 14 “the AI data may include difference information between the original image 105 and the first image 115”)
Examiner notes that Kuo discloses in ¶¶[0032]-[0040] that luminance (Y) output and chrominance (UV) output from neural network are combined to generate full image domain output. This supports forming a pixel domain based on luminance path processing.
Lee also discloses obtain an AI-encoded image based on the first image and the second residual image (p.11 “first encoded”), and
transmit, to an external device through the communication interface, a compressed image obtained by encoding the AI-encoded image (p. 6 “AI data may be transmitted together with image data in the form of a bitstream”).
Regarding claim 2, Lee and Kuo disclose all the limitations of claim 1, and is analyzed as previously discussed with respect to that claim.
Furthermore, Lee discloses the electronic device of claim 1, wherein the instructions, when executed by the at least one processor cause the electronic device to:
identify operation setting information of the first neural network model based on at least one of information about an image size of the second image, information about a network state, or information about a type of a codec, and input the second image into the first neural network model to which the identified operation setting information is applied (p. 22” determined based on at least one of a target bitrate, a bitrate type, and a codec type.”),
wherein the operation setting information comprises: at least one of information about a number of layers of the first neural network model, information about a number of channels for each layer, information about a filter size, information about stride, information about pulling, or information about a parameter (p. 12” DNN setting information (e.g., the number of convolution layers, the number of filter kernels for each convolutional layer, parameters of each filter kernel, etc.) is stored in the form of a lookup table”).
Regarding claim 3, Lee and Kuo disclose all the limitations of claim 2, and is analyzed as previously discussed with respect to that claim.
Furthermore, Lee discloses the electronic device of claim 2, wherein the instructions, when executed by the at least one processor cause the electronic device to:
identify the operation setting information of the first neural network model based on the information about the image size of the second image, the information about the network state, and the information about the type of the codec, and AI decoding information of the external device (p.12 “an identifier of mutually agreed DNN configuration information”),
wherein the AI decoding information of the external device comprises: operation setting information of a second neural network model used in the AI decoding in the external device, and wherein the first neural network model is trained in association with the operation setting information of the second neural network model (p.6 “the first DNN and the second DNN are jointly trained”).
Regarding claim 4, Lee and Kuo disclose all the limitations of claim 2, and is analyzed as previously discussed with respect to that claim.
Furthermore, Lee discloses electronic device of claim 2, wherein the instructions, when executed by the at least one processor cause the electronic device to: perform downscaling of the first image and obtain a third image (p.14 “The AI downscaler 612 may acquire the AI downscaled first image 115 from the original image 105 through the first DNN”) and
obtain the AI-encoded image based on the third image and the second residual image (p. 14 “the AI data may include difference information between the original image 105 and the first image 115”).
Regarding claim 5, Lee and Kuo disclose all the limitations of claim 4, and is analyzed as previously discussed with respect to that claim.
Furthermore, Lee discloses the electronic device of claim 4, wherein the first neural network model is a model trained to perform downsampling of an image through AI encoding (p. 14 “Each of the plurality of pieces of DNN configuration information may be trained to obtain the first image 115 having a predetermined resolution and/or a predetermined image quality”), and
wherein the instructions, when executed by the at least one processor cause the electronic device to: input the second image into the first neural network model and obtain the first residual image downsampled through the AI encoding (p. 14 “AI downscaler 612 may acquire the AI downscaled first image 115 from the original image 105 through the first DNN” ),
obtain the second residual image including pixel residual information based on the first residual image ((p. 14 “difference information between the original image 105 and the first image 115”)).
Lee does not explicitly disclose add pixel values included in the third image and pixel values included in the second residual image and obtain the AI-encoded image.
However, Kuo from the same or similar endeavor of image processing discloses the add pixel values included in the third image and pixel values included in the second residual image and obtain the AI-encoded image. (¶[0119] “Y=uint8 …+X2…).
The motivation for combining Lee and Kuo has been discussed in connection with claim 1, above.
Regarding claim 6, Lee and Kuo disclose all the limitations of claim 2, and is analyzed as previously discussed with respect to that claim.
Lee does not explicitly disclose electronic device of claim 2, wherein the luminance residual information includes YUV residual information, and wherein the instructions, when executed by the at least one processor cause the electronic device to: obtain an R value, a G value, and a B value by applying conversion gains to a Y value, a U value, and a V value included in the first residual image, and identify the obtained R value, G value, and B value as the pixel residual information.
However, Kuo from the same or similar endeavor of image processing discloses the electronic device of claim 2, wherein the luminance residual information includes YUV residual information, and wherein the instructions, when executed by the at least one processor cause the electronic device to: obtain an R value, a G value, and a B value by applying conversion gains to a Y value, a U value, and a V value included in the first residual image, and identify the obtained R value, G value, and B value as the pixel residual information. add pixel values included in the third image and pixel values included in the second residual image and obtain the AI-encoded image (¶[0128] If the input image is YUV420, and the output image is RGB).
The motivation for combining Lee and Kuo has been discussed in connection with claim 1, above.
Regarding claims 7-11, these claims are rejected based on the same art and evidentiary limitations applied to the device of claims 1-6, since they claim analogous subject matter in the form of a device for performing the same or equivalent functionality.
The Examiner notes that it is well-known in the art that video compression involves a complementary pair of systems: a encoder and a decoder. The encoder converts the source data into a compressed form, occupying a reduced number of bits prior to transmission or storage, while the decoder converts the compressed form back into a representation of the original video data by performing a reciprocal process to that of the encoder, decoding the encoded video data from the bitstream.
Regarding claims 12-17, these claims are rejected based on the same art and evidentiary limitations applied to the device of claims 1-6, since they claim analogous subject matter in the form of a method for performing the same or equivalent functionality.
Regarding claims 18-20, these claims are rejected based on the same art and evidentiary limitations applied to the device of claims 7, 9 and 10, since they claim analogous subject matter in the form of a method for performing the same or equivalent functionality.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892 for additional references.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FABIO S LIMA whose telephone number is (571)270-0625. The examiner can normally be reached on Monday through Friday, 7:30 AM - 4:00 PM (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JAMIE ATALA can be reached on (571)272-7384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FABIO S LIMA/Primary Examiner, Art Unit 2486