DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-5, 8, 10, 12-16, 21, 25-27, and 29 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jen et al., US 2020/0327864.
Regarding claim 1, Jen discloses a multimedia processing method (figs. 4-5; para 0005-0006 and 0025-0026; video processing system and method for image enhancement), comprising:
acquiring an input multimedia (figs. 4-5, elements IMG_1; para 0025-0026; receiving input frame IMG_1 from stream buffer), and
performing image quality enhancement processing on the input multimedia to obtain an enhanced multimedia (figs. 4-5; para 0025-0026; super resolution is performed at the AP processor 408 to generate the scaled frame IMG_3 with a resolution higher than that of the input frame IMG_1);
wherein a resolution corresponding to the enhanced multimedia is higher than a resolution corresponding to the input multimedia (figs. 4-5; para 0025-0026; the scaled frame IMG_3 with a resolution higher than that of the input frame IMG_1).
Regarding claim 2, the multimedia processing method according to claim 1, Jen discloses further comprising:
performing compensation processing on the enhanced multimedia to obtain a compensated multimedia (fig. 5, element IMG_2; para 0026-0027),
wherein a resolution corresponding to the compensated multimedia is higher than the resolution corresponding to the input multimedia (fig. 5, element IMG_2; para 0026-0027).
Regarding claim 3, the multimedia processing method according to claim 2, Jen discloses wherein the multimedia processing method is applied to a first display panel (fig. 4, element 412; para 0024; a screen mirroring panel),
the performing compensation processing on the enhanced multimedia to obtain a compensated multimedia (fig. 5, element IMG_2; para 0026-0027), comprises:
selecting a compensation parameter corresponding to the first display panel from a plurality of compensation parameters (figs. 4-5; para 0024 and 0027), and
performing compensation processing on the enhanced multimedia based on the compensation parameter corresponding to the first display panel to obtain the compensated multimedia (figs. 4-5; para 0024 and 0027).
Regarding claim 4, the multimedia processing method according to claim 3, Jen discloses wherein the selecting a compensation parameter corresponding to the first display panel from a plurality of compensation parameters (figs. 4-5; para 0024 and 0027), comprises:
acquiring a compensation input parameter corresponding to the first display panel, wherein the compensation input parameter comprises a panel parameter corresponding to the first display panel and/or an environmental parameter of an environment where the first display panel is located (figs. 4-5; para 0024 and 0027), and
based on the compensation input parameter, selecting a compensation parameter corresponding to the compensation input parameter from the plurality of compensation parameters as the compensation parameter corresponding to the first display panel (figs. 4-5; para 0024 and 0027).
Regarding claim 5, the multimedia processing method according to claim 4, Jen discloses wherein the acquiring a compensation input parameter corresponding to the first display panel (figs. 4-5; para 0024 and 0027), comprises:
generating a first array, wherein the first array corresponds to the first display panel, the first array comprises a plurality of first array elements, the panel parameter is represented by at least one first array element, and the environmental parameter is represented by at least one first array element (figs. 4-6; para 0024 and 0027-0028), and
determining the compensation input parameter corresponding to the first display panel based on the first array (figs. 4-5; para 0024 and 0027);
wherein the panel parameter comprises a type of the first display panel, a size of the first display panel, and a display mode of the first display panel (fig. 6; para 0028); and
the type of the first display panel comprises an organic light-emitting display panel and a liquid crystal display panel, and the display mode of the first display panel comprises direct display and cast screen display (fig. 6; para 0028).
Regarding claim 8, the multimedia processing method according to claim 4, Jen discloses wherein the environmental parameter comprises an ambient light parameter (figs. 3 and 7; para 0022 and 0029),
the ambient light parameter is determined based on a brightness value of ambient light of the environment where the first display panel is located (figs. 3 and 7; para 0022 and 0029);
the compensation processing comprises at least one of the following: brightness adjustment, contrast adjustment, and saturation adjustment (figs. 5-7; para 0022 and 0027-0029).
Regarding claim 10, the multimedia processing method according to claim 3, Jen discloses further comprising:
generating the plurality of compensation parameters, wherein the generating the plurality of compensation parameters (fig. 5; element 504; para 0026-0027), comprises:
acquiring a color card image obtained by photographing a standard color card (figs. 4-5; para 0023-0027);
performing image quality enhancement processing on the color card image to obtain an enhanced color card image (figs. 4-5; para 0023-0027);
performing compensation processing on the enhanced color card image based on an initial compensation parameter to obtain a compensated color card image (figs. 4-5; para 0023-0027);
displaying the compensated color card image on a second display panel, and determining whether a display of the compensated color card image meets a preset requirement (figs. 4-5; para 0023-0027);
in response to the display of the compensated color card image not meeting the preset requirement, adjusting the initial compensation parameter to obtain an adjusted compensation parameter, and performing compensation processing on the enhanced color card image again based on the adjusted compensation parameter until the display of the compensated color card image meets the preset requirement, and taking a compensation parameter corresponding to the compensated color card image meeting the preset requirement as one of the plurality of compensation parameters (figs. 4-5; para 0023-0027);
determining a compensation input parameter corresponding to the second display panel, wherein the compensation input parameter corresponding to the second display panel comprises a panel parameter corresponding to the second display panel and/or an environmental parameter of an environment where the second display panel is located (figs. 4-5; para 0023-0027); and
establishing a mapping relationship between the compensation parameter corresponding to the compensated color card image meeting the preset requirement and the compensation input parameter corresponding to the second display panel (figs. 4-5; para 0023-0027).
Regarding claim 12, the multimedia processing method according to claim 1, Jen discloses wherein the image quality enhancement processing comprises at least one of the following processing: frame interpolation processing (fig. 3, element 306; para 0022), super-resolution processing (fig. 5; para 0026), noise-reduction processing (para 0020), color adjustment processing (figs. 4-5; para 0023-0027), high dynamic range up-conversion processing (figs. 4-5; para 0025-0026), and detail restoration processing (fig. 5; para 0027),
performing image quality enhancement processing on the input multimedia to obtain the enhanced multimedia (figs. 4-5; para 0025-0026), comprises:
acquiring an image quality enhancement parameter (figs. 4-5; para 0025-0027); and
performing image quality enhancement processing on the input multimedia according to the image quality enhancement parameter to obtain the enhanced multimedia (figs. 4-5; para 0025-0027).
Regarding claim 13, the multimedia processing method according to claim 12, Jen discloses wherein the image quality enhancement parameter comprises at least one of following parameters: a frame interpolation algorithm name and/or a frame interpolation parameter corresponding to the frame interpolation processing (fig. 3, element 306; para 0022), a super-resolution algorithm name and/or a resolution parameter corresponding to the super-resolution processing (fig. 5; para 0026), a color adjustment algorithm name and/or a color adjustment parameter corresponding to the color adjustment processing (figs. 4-5; para 0023-0027), a noise-reduction algorithm name and/or a noise-reduction parameter corresponding to the noise-reduction processing (para 0020), a high dynamic range up-conversion algorithm name and/or a high dynamic range up-conversion parameter corresponding to the high dynamic range up-conversion processing figs. 4-5; para 0025-0026), and a detail restoration algorithm name and/or a detail restoration parameter corresponding to the detail restoration processing (fig. 5; para 0027).
Regarding claim 14, the multimedia processing method according to claim 13, Jen discloses wherein the acquiring an image quality enhancement parameter (figs. 4-5; para 0025-0027), comprises:
generating a second array, wherein the second array comprises a plurality of second array elements, the frame interpolation parameter corresponding to the frame interpolation processing is represented by at least one second array element (fig. 3, element 306; para 0022), the resolution parameter corresponding to the super-resolution processing is represented by at least one second array element (fig. 5; para 0026), the color adjustment parameter corresponding to the color adjustment processing is represented by at least one second array element (figs. 4-5; para 0023-0027), the noise-reduction parameter corresponding to the noise-reduction processing is represented by at least one second array element (para 0020), the high dynamic range up-conversion parameter corresponding to the high dynamic range up-conversion processing is represented by at least one second array element (figs. 4-5; para 0025-0026), and the detail restoration parameter corresponding to the detail restoration processing is represented by at least one second array element (fig. 5; para 0027); and
determining the image quality enhancement parameter based on the second array (figs. 4-5; para 0025-0027).
Regarding claim 15, the multimedia processing method according to claim 13, Jen discloses wherein the acquiring an image quality enhancement parameter (figs. 4-5; para 0025-0027), comprises:
acquiring an algorithm string, wherein the algorithm string comprises at least one of following algorithms: a frame interpolation algorithm (fig. 3, element 306; para 0022), a super-resolution algorithm (fig. 5; para 0026), a color adjustment algorithm (figs. 4-5; para 0023-0027), a noise-reduction algorithm (para 0020), a high dynamic range up-conversion algorithm (figs. 4-5; para 0025-0026), and a detail restoration algorithm (fig. 5; para 0027), the frame interpolation algorithm comprises the frame interpolation algorithm name and the frame interpolation parameter (fig. 3, element 306; para 0022), the super-resolution algorithm comprises the super-resolution algorithm name and the resolution parameter (fig. 5; para 0026), the color adjustment algorithm comprises the color adjustment algorithm name and the color adjustment parameter (figs. 4-5; para 0023-0027), the noise-reduction algorithm comprises the noise-reduction algorithm name and the noise-reduction parameter (para 0020), the high dynamic range up-conversion algorithm comprises the high dynamic range up-conversion algorithm name and the high dynamic range up-conversion parameter (figs. 4-5; para 0025-0026), and the detail restoration algorithm comprises the detail restoration algorithm name and the detail restoration parameter (fig. 5; para 0027);
determining the image quality enhancement parameter based on the algorithm string (figs. 4-5; para 0025-0027).
Regarding claim 16, the multimedia processing method according to claim 12, Jen discloses wherein a multimedia on which the frame interpolation processing (fig. 3, element 306; para 0022) is used to be performed comprises a video (figs. 4-5, elements IMG_1; para 0025-0026),
the frame interpolation processing is implemented based on a first deep learning model (fig. 3, element 306; para 0020 and 0022), and the first deep learning model is configured to add at least one transition image frame between every two image frames in the video (figs. 1 and 3, element 306; para 0020 and 0022),
the super-resolution processing is implemented based on a second deep learning model, and the second deep learning model is configured to perform super-resolution processing on a multimedia on which the super-resolution processing is used to be performed to improve a spatial resolution of the multimedia on which the super-resolution processing is used to be performed (fig. 5; para 0020 and 0026),
the color adjustment processing is implemented based on a third deep learning model (figs. 4-5; para 0020 and 0023-0027); and
the noise-reduction processing is implemented based on a fourth deep learning model, and the fourth deep learning model is configured to perform noise-reduction processing on a multimedia on which the noise-reduction processing is used to be performed (para 0020).
Regarding claim 21, the multimedia processing method according to claim 12, Jen discloses wherein the input multimedia comprises a video (figs. 4-5, elements IMG_1; para 0025-0026), and the enhanced multimedia comprises an enhanced video corresponding to the video (figs. 4-5, elements IMG_3; para 0025-0026),
performing image quality enhancement processing on the input multimedia to obtain the enhanced multimedia (figs. 4-5, elements IMG_3; para 0025-0026), comprises:
performing frame interpolation processing on the video to obtain a video after frame interpolation (fig. 3, element 306; para 0022);
performing color adjustment processing on the video after frame interpolation to obtain a color-adjusted video (figs. 4-5; para 0023-0027);
performing noise-reduction processing on the color-adjusted video to obtain a noise-reduced video (para 0020);
performing super-resolution processing on the noise-reduced video to obtain the enhanced video (fig. 5; para 0026),
wherein a resolution of the enhanced video is higher than a resolution of the noise-reduced video (figs. 4-5; para 0025-0026);
a color depth corresponding to the color-adjusted video is higher than a color depth corresponding to the video after frame interpolation, and/or a color gamut corresponding to the color-adjusted video is higher than a color gamut corresponding to the video after frame interpolation (figs. 4-5; para 0023-0027).
Regarding claim 25, the multimedia processing method according to claim 2, Jen discloses wherein a color depth corresponding to the compensated multimedia is higher than a color depth corresponding to the input multimedia, and/or a color gamut corresponding to the compensated multimedia is higher than a color gamut corresponding to the input multimedia (figs. 4-5; para 0023-0027).
Regarding claim 26, this claim recites substantially the same limitations that are performed by claim 1 above, and it is rejected for the same reasons.
Regarding claim 27, the multimedia processing apparatus according to claim 26, Jen discloses further comprising an input apparatus (fig. 4, element 406; para 0023-0024; wireless communications device (e.g. mobile device-cellular phone or tablet)),
wherein in response to the multimedia processing method comprising acquiring a compensation input parameter corresponding to a first display panel and/or acquiring an image quality enhancement parameter (figs. 4-5; para 0024 and 0027),
the compensation input parameter and/or the image quality enhancement parameter are/is input through the input apparatus (figs. 4-5; para 0024 and 0027);
the input apparatus comprises at least one selected from a group comprising a touch screen, a touch panel, a keyboard, a mouse, and a microphone (fig. 4, element 406; para 0023-0024; a cellular phone or tablet is capable of having a touch screen, a touch panel, a keyboard, and a microphone).
Regarding claim 29, this claim recites substantially the same limitations that are performed by claim 1 above, and it is rejected for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 23-24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jen et al., US 2020/0327864 in view of Wang et al., US 2020/0036983.
Regarding claim 23, the multimedia processing method according to claim 1, Jen discloses the acquiring an input multimedia (figs. 4-5, elements IMG_1; para 0025-0026).
Jen discloses claim 23 as enumerated above, but Jen does not explicitly disclose acquiring an original multimedia file; decoding the original multimedia file to obtain an original multimedia; and performing format conversion processing on the original multimedia to obtain the input multimedia, wherein a pixel format of the input multimedia comprises an RGB format as claimed.
However, Wang discloses obtain first stream data generated from a first image of an image file; the decoding apparatus decodes the first stream data according to the first video decoding mode, to generate first YUV data of the first image; converts the first YUV data into the RGB data of the first image (fig. 10; para 0217 and 0222-0223).
Therefore, taking the combined disclosures of Jen and Wang as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate obtain first stream data generated from a first image of an image file; the decoding apparatus decodes the first stream data according to the first video decoding mode, to generate first YUV data of the first image; converts the first YUV data into the RGB data of the first image as taught by Wang into the invention of Jen for the benefit of improving loading speed and significantly reducing bandwidth and storage costs (Wang: para 0003).
Regarding claim 24, the multimedia processing method according to claim 2, Jen does not explicitly disclose wherein a pixel format of the compensated multimedia is an RGB format, the multimedia processing method further comprises: performing format conversion on the compensated multimedia to obtain an output multimedia, wherein a pixel format of the output multimedia is a YUV format; encoding the output multimedia to obtain an output multimedia file as claimed.
However, Wang discloses converting the RGB data of the first image into first YUV data; and encoding the first YUV data according to the first video encoding mode, to generate the first stream data (para 0321).
Therefore, taking the combined disclosures of Jen and Wang as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate obtain first stream data generated from a first image of an image file; the decoding apparatus decodes the first stream data according to the first video decoding mode, to generate first YUV data of the first image; converts the first YUV data into the RGB data of the first image as taught by Wang into the invention of Jen for the benefit of improving loading speed and significantly reducing bandwidth and storage costs (Wang: para 0003).
Allowable Subject Matter
Claim 17 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
The prior art made of record and considered pertinent to the applicant's disclosure, taken individually or in combination, does not teach the claimed invention having the following limitations, in combination with the remaining claimed limitations.
Regarding dependent claim 17, the prior art does not teach or suggest the claimed invention having “wherein the third deep learning model comprises a regression sub-model and a plurality of lookup table sub-model groups, and each lookup table sub-model group comprises at least one lookup table sub-model; the color adjustment processing comprises: preprocessing the multimedia on which the color adjustment processing is used to be performed to obtain normalized data, wherein the preprocessing comprises normalization processing; processing the normalized data by using the regression sub-model to obtain at least one weight parameter; acquiring a color adjustment parameter; selecting a lookup table sub-model group corresponding to the color adjustment parameter from the plurality of lookup table sub-model groups according to the color adjustment parameter; determining a target lookup table sub-model based on the at least one weight parameter and the lookup table sub-model group; and processing the normalized data by using the target lookup table sub-model to generate an output of the color adjustment processing; the first deep learning model comprises a real-time intermediate flow estimation algorithm model; the second deep learning model comprises a residual feature distillation network model; the fourth deep learning model comprises an Unet network model”, and a combination of other limitations thereof as recited in the claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Nayak et al., US 2022/0374714 discloses real time content enhancement can be provided using a solution that is lightweight enough to operate on client devices, even for high resolution, high bitrate content.
Wang et al., US 2020/0327702 discloses techniques related to accelerated video enhancement using deep learning selectively applied based on video codec information.
Wang, et al., US 2022/0207680 discloses an image processing method includes obtaining a plurality of frames of raw images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN D HUYNH whose telephone number is (571)270-1937. The examiner can normally be reached 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN D HUYNH/Primary Examiner, Art Unit 2665
Prosecution Insights