DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1-2 are canceled. Claims 21-22 are added. Claims 3-22 are pending in this application.
Election/Restrictions
Applicant’s election without traverse of Group II (claims 3-20) in the reply filed on 04/28/2026 is acknowledged. Claims 1-2 are withdrawn from further consideration pursuant to 37 CFR 1.142(b), as being drawn to a nonelected invention. Since the restriction requirement properly made, the restriction requirement is now made final.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 3-22 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mironica, US 2022/0270209.
Regarding claim 3, Mironica discloses a system (fig. 1; para 0047; an image artifact removal system), comprising: at least one processor operatively coupled to non-transitory computer-readable memory, the non-transitory computer-readable memory storing instructions which, when executed (para 0129; the image artifact removal system 106 includes one or more instructions stored on a computer-readable storage medium and executable by processors. When executed by the one or more processors, the computer-executable instructions of the image artifact removal system 106 causes a computing device to perform the feature learning methods), cause the at least one processor to:
receive a stream of video data (para 0021, 0039-0040, 0056, 0098, and 0102; a digital image (e.g., a compressed digital image) can include one or more frames in a video);
determine a category of the stream of video data based at least partially upon a compression level of the stream of video data (para 0021, 0041, 0056, 0064-0065, 0098, and 0135; the image artifact removal system determines and provides the compression ratio of the image; …images may be compressed by different amounts as measured by a compression ratio. For example, in some implementations, the amount that an image is compressed signals the number and/or magnitude of compression artifacts in the image; …converts the compression ratio into a compression ratio parameter. For example, for a compression ratio of 10 (e.g., 90% compressed), the conditional layer 314 outputs the compression ratio parameter of 01, and/or for a compressed digital image with a poor compression ratio, the conditional layer 314 outputs a compression ratio parameter of 09. In some implementations, the conditional layer 314 includes two values that represent the ratio between the minimum quality factor and the maximum quality factor (e.g., the compression ratio));
select weights for a machine learning model based upon the category (para 0020, 0022-0023, 0043-0044, 0057, 0066-0067, 0099, and 0137; the image artifact removal system 106 modifies the generator (i.e., generator neural network) based on the compression ratio; For example, based on the compression ratio parameter, the generator 310 learns which components of the neural network layers to apply and/or the appropriate weights to assign to each component to effectively and efficiently remove compression artifacts; the image artifact removal system 106 utilizes the compression ratio to adjust the generator 410. For example, based on the compression ratio, the generator 410 utilizes a particular path of components and/or weights within the generator 410 to generate the improved image 424. If the compression ratio changes, the image artifact removal system 106 utilizes different components and/or weights within the generator 410); and
execute the machine learning model with the selected weights to remove compression artifacts in the stream of video data and upscale a resolution of the stream of video data (para 0021, 0045, 0057-0058, 0071, 0099-0101, and 0136-0137; The image artifact removal system utilizes the generator (i.e., generative adversarial network (GAN)) to remove complex compression artifacts from compressed digital images; Based on the compression ratio, the generator outputs an improved image that accurately removes the complex compression artifacts, such as blurring, blocking, and ringing artifacts; the image artifact removal system 106 utilizes the compression ratio to adjust the generator 410. For example, based on the compression ratio, the generator 410 utilizes a particular path of components and/or weights within the generator 410 to generate the improved image 424).
Regarding claim 4, the system of claim 3, Mironica further discloses wherein the determination and selection are performed by a machine learning model with an input vector representing at least one slice of the stream of video data and with an output vector including data representing at least a degree of compression of the stream of video data (fig. 4B; para 0101-0109).
Regarding claim 5, the system of claim 3, Mironica further discloses wherein the non-transitory computer-readable memory stores further instructions which, when executed, further cause the at least one processor to:
receive metadata representing at least one aspect of the stream of video data (para 0056, 0098, 0101-0102, and 0135); and
determine the category of the stream of video data based at least partially upon the metadata (para 0021, 0041, 0056, 0064-0065, 0098, and 0135).
Regarding claim 6, the system of claim 5, Mironica further discloses wherein the metadata includes at least one of a quantization parameter (para 0040-0041, 0071, and 0092) or a video genre (para 0101-0109).
Regarding claim 7, the system of claim 3, Mironica further discloses wherein the non-transitory computer-readable memory stores further instructions which, when executed, further cause the at least one processor to:
detect one or more visual boundary edge strengths within at least one slice of the stream of video data (para 0026, 0042, and 0112-0113); and
select the weights at least partially based upon the detected visual boundary edge strengths (para 0020, 0022-0023, 0043-0044, 0057, 0066-0067, 0099, and 0137).
Regarding claim 8, the system of claim 3, Mironica further discloses wherein the at least one processor comprises a first processor and a neural network accelerator comprising accelerator circuitry, wherein the accelerator circuitry is configured to perform multiplication and accumulation operations at a higher rate than the first processor of the at least one processor is capable of, and wherein the accelerator circuitry is employed to perform at least a portion of the artifact removal (figs. 8 and 10; para 0129, 0148, and 0158-0159).
Regarding claim 9, the system of claim 3, Mironica further discloses wherein the non-transitory computer-readable memory stores further instructions which, when executed, further cause the at least one processor to:
analyze one or more frames of the stream of video data (para 0021, 0039-0040, 0056, 0098, and 0102); and
determine metadata about the stream of video data based upon a content of the stream of video data (para 0056, 0098, 0101-0102, and 0135), wherein the metadata includes at least one of a video genre, one or more individuals or objects in the one or more frames, a contrast value, a saturation value, a cast listing, or a cinematographic style (para 0101-0109).
Regarding claim 10, the system of claim 3, Mironica further discloses wherein the determination includes generating a histogram of quantization parameter values associated with the stream of video data, and wherein the category is determined based at least partially upon the histogram (para 0026, 0075, and 0077-0078).
Regarding claim 11, this claim recites substantially the same limitations that are performed by claim 3 above, and it is rejected for the same reasons.
Regarding claim 12, this claim recites substantially the same limitations that are performed by claim 4 above, and it is rejected for the same reasons.
Regarding claim 13, this claim recites substantially the same limitations that are performed by claim 5 above, and it is rejected for the same reasons.
Regarding claim 14, this claim recites substantially the same limitations that are performed by claim 6 above, and it is rejected for the same reasons.
Regarding claim 15, this claim recites substantially the same limitations that are performed by claim 7 above, and it is rejected for the same reasons.
Regarding claim 16, this claim recites substantially the same limitations that are performed by claim 9 above, and it is rejected for the same reasons.
Regarding claim 17, this claim recites substantially the same limitations that are performed by claim 10 above, and it is rejected for the same reasons.
Regarding claim 18, the method of claim 11, Mironica further discloses wherein the determining comprises:
determining metadata with a machine learning model (para 0056, 0098, 0101-0102, and 0135); and
performing rules-based analysis to categorize the stream of video data based at least in part upon the determined metadata (para 0021, 0041, 0056, 0064-0065, 0098, and 0135).
Regarding claim 19, the method of claim 11, Mironica further discloses wherein the determining includes performing motion analysis on the stream of video data (para 0021, 0039-0040, 0056, 0098, and 0102).
Regarding claim 20, the method of claim 11, Mironica further discloses wherein the selecting includes outputting a model index indicative of the weights (para 0020, 0022-0023, 0043-0044, 0057, 0066-0067, 0099, and 0137).
Regarding claim 21, this claim recites substantially the same limitations that are performed by claim 3 above, and it is rejected for the same reasons.
Regarding claim 22, the method of claim 21, Mironica further discloses comprising:
selecting second weights for the stream of video data (para 0113 and 0143); and
executing the machine learning model with the second weights to remove compression artifacts in a per-frame region of frames of the stream of video data and upscale a resolution of the per-frame region of the frames of the stream of video data concurrently with the executing the machine learning model with the selected weights (para 0107 and 0143-0144).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kim et al., US 2023/0102895 discloses a method and/or device for predicting a compression quality of an image during image correction (e.g., image quality enhancement) in an electronic device, and/or processing the image, based on at least the prediction.
Kennett et al., US 2021/0274224 discloses the method (500) involves decompressing compressed digital video content to generate (510) decompressed digital video content includes multiple decoded video frames.
Morzos, US 2022/0148146 discloses systems and methods for detecting and remediating compression artifacts in multimedia items.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN D HUYNH whose telephone number is (571)270-1937. The examiner can normally be reached 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN D HUYNH/Primary Examiner, Art Unit 2665
Prosecution Insights