Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to the amendment filed on November 13th, 2025. Claims 1, 4, 5, 7, 10, 11, 13, 16, and 18 have been amended and claims 2, 3, 8, 9, 14, and 15 have been cancelled. The amended claims limitations have been fully considered but are not persuasive. Claims 1, 4-7, 10-13, and 16-18 remain rejected in the application.
Response to Arguments
In response to applicant’s arguments regarding the 101 rejections, argument was fully considered and is persuasive and amendments overcome the 101 rejections.
In response to applicant’s arguments regarding the 102 rejections, argument was fully considered and is persuasive and amendments overcome the 102 rejections.
In response to applicant’s arguments regarding Cohen failing to disclose downsample luminance to match chrominance size, arguments fully considered but is not persuasive. Cohen explicitly discloses this (Cohen: Col. 8, Lines 45-50 “Once the high-resolution signal input module 400 has received a high-resolution image, it provides that image to a signal downsampling module 440 which downsamples the high-resolution image to a lower resolution image that has a low enough resolution to be processed using available computing power and memory.”)(teaches that the entire image, including its luminance information is fed to module 440 and downsampled, therefore the Y channel is downsampled in accordance with the chosen sampling layout).
In response to applicant’s arguments regarding Cohen failing to obtain coefficients by linear transformation on chrominance based on downsampled luminance, arguments fully considered but is not persuasive. Cohen explicitly teaches this (Cohen: Col. 8, Lines 45-50 “Once the high-resolution signal input module 400 has received a high-resolution image, it provides that image to a signal downsampling module 440 which downsamples the high-resolution image to a lower resolution image that has a low enough resolution to be processed using available computing power and memory.”)(teaches that the entire image, including its luminance information is fed to module 440 and downsampled, therefore the Y channel is downsampled in accordance with the chosen sampling layout).
In response to applicant’s arguments regarding Cohen failing to disclose extracting luminance and chrominance of the initial text image, arguments fully considered but is not persuasive. Cohen explicitly teaches this (Cohen: Col. 12, Lines 61-67 “the Joint Bilateral Upsampler converts the output of the colorization algorithms into the YIQ color space (or to any other desired color space separating luminance (Y) from the two chrominance channels (I and Q)). In other words, the low-resolution colorization Solution set is split using conventional techniques to provide into two separate chrominance solution sets”)(teaches that the system separates luminance from chrominance).
In response to applicant’s arguments regarding Cohen failing to disclose the guided filter set up, arguments fully considered but is not persuasive. Cohen explicitly teaches this (Cohen: Col. 9, Lines 17-21 “constructs a high-resolution solution set as a joint bilateral function of a distance measure (spatial filter) between data points of the low-resolution solution set, and a difference (range filter) between data points of the original high-resolution input signal.”).
In response to applicant’s arguments regarding Budagavi failing to disclose the coefficient limitations, argument is fully considered but is not persuasive. Budagavi explicitly teaches this (Budagavi: Abstract “computing parameters C. and B of a linear model”).
In response to applicant’s arguments regarding Westerman failing to cure the deficiencies, arguments is fully considered but is not persuasive. Cohen explicitly teaches this (Cohen: Col. 9, Lines 44-46 “processed version of the original input signal that is provided for storage and/or display via a high-resolution signal output module 480.”) (teaches generating a displayable output signal/image).
In response to applicants argument that He fails to disclose the coefficient upsampling, argument is fully considered but is not persuasive. He explicitly teaches this (He: Page 2 “The two coefficient maps ¯a and ¯ b are bilinearly upsampled to the original size.”).
In response to applicant’s arguments regarding allowing the dependent claims, argument has been fully considered but is not persuasive. Due to the Examiner maintaining the rejection for the independent claims, rejections for the dependent claims are maintained. Claims 1, 4-7, 10-13, and 16-18 remain rejected in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 7, 10, 13, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Cohen et. al (U.S. Patent No. 7,889,949), in view of Budagavi (U.S. Patent No. 9,723,327).
Regarding claim 1, Cohen discloses a method for processing text images, comprising: obtaining an initial text image (interpreted as the system gets an image frame that will later be processed)(Cohen: Col. 14, Lines 36-39 “Upsampler begins operation by receiving 500 one or more high-resolution signals, i, from any of a plurality of signal input sources”) (teaches that the first step is to receive an input image); extracting luminance and chrominance of the initial text image (interpreted as breaking the input image into its brightness (Y) channel and its color channels)(Cohen: Col. 12, Lines 61-67 “the Joint Bilateral Upsampler converts the output of the colorization algorithms into the YIQ color space (or to any other desired color space separating luminance (Y) from the two chrominance channels (I and Q)). In other words, the low-resolution colorization Solution set is split using conventional techniques to provide into two separate chrominance solution sets”)(teaches that the system separates luminance from chrominance); obtaining coefficients (interpreted as one of the numeric weights in the local linear model that links brightness to color) of chrominance transformation by performing a linear transformation on the chrominance based on the luminance (interpreted as compute local linear weights (a,b) that map luma values to chroma values)(Cohen: Col. 2, Lines 3-8 “Unlike traditional bilateral filters which operate at a single resolution, the Joint Bilateral Upsampler described herein operates as a joint bilateral function of a distance measure (spatial filter) between data points of a low-resolution solution set, and a difference (range filter) between data points of the original high-resolution input signal.”)(Cohen: Col. 11, Lines 31-49 “The upsampled solution S is then obtained as illustrated by Equation 3, where: W 1 W. W. E ion 3 S = XI, f(pl.-q1Dg(II – III) As noted above, the output of Equation 3 is a high-resolution solution set that is either saved for later use, or applied to the original high-resolution input signal. However, it should also be noted that in various embodiments, further upsampling or downsampling of either the combination of the original high-resolution input signal and the high-resolution signal set, or the high-resolution output signal is enabled in order to achieve a desired final resolution for either the final high resolution output signal or the high-resolution solution set.”)(Equation 3 is the linear model who’s spatial/range kernels deliver a per-pixel value that acts as the sought after coefficient linking Y to each chroma channel. The reference teaches a linear combination that reconstructs each high resolution sample from the luminance guided range kernel; the weights act as coefficients of a chroma-reconstruction transform); and obtaining a to-be-displayed text image based on the coefficients of chrominance transformation (interpreted as using those coefficients to rebuild a full resolution color image for display) (Cohen: Col. 9, Lines 33-46 “Once the joint bilateral upsampling module 460 has constructed the high-resolution Solution set, that Solution set is either saved for later use or applied to the original input signal to produce a high-resolution output signal…The resulting high-resolution output signal is a processed version of the original input signal that is provided for storage and/or display via a high-resolution signal output module 480.”)(teaches that the computed coefficients solution set is combined with the original image to generate a high resolution output ready for display), wherein the obtaining the coefficients of chrominance transformation by performing the linear transformation on the chrominance based on the luminance comprises (interpreted as a preamble for defining how the coefficients are found/determined): obtaining a way of sampling of the initial text image (interpreted as deciding what color sample scheme with be used (e.g., which channels and how they are arranged))(Cohen: Col. 12, Lines 61-67 “to address the colorization problem, the Joint Bilateral Upsampler converts the output of the colorization algorithms into the YIQ color space (or to any other desired color space separating luminance (Y) from the two chrominance channels (I and Q)). In other words, the low-resolution colorization Solution set is split using conventional techniques to provide into two separate chrominance Solution sets.”)(converting to YIQ explicitly selects a sampling/encoding scheme that separates one luminance channel from two chrominance channels corresponding to the claimed limitation); downsampling the luminance based on the way of sampling (interpreted as reducing the resolution of the Y brightness channel according to that scheme)(Cohen: Col. 8, Lines 45-50 “Once the high-resolution signal input module 400 has received a high-resolution image, it provides that image to a signal downsampling module 440 which downsamples the high-resolution image to a lower resolution image that has a low enough resolution to be processed using available computing power and memory.”)(teaches that the entire image, including its luminance information is fed to module 440 and downsampled, therefore the Y channel is downsampled in accordance with the chosen sampling layout); and obtaining the coefficients of chrominance transformation by performing the linear transformation on the chrominance based on the downsampled luminance, wherein a size of the downsampled luminance is the same as a size of the chrominance (interpreted as using the low resolution Y map (it’s the same grid as each chroma plane), compute local linear weights (a,b) that relate Y to each chroma value)(Cohen: Col. 13, Lines 3-6 “the Joint Bilateral Upsampler separately applies the upsampling techniques described with respect to Equation 3 to each of the two chrominance channels to produce two high resolution chrominance solutions”)(Cohen: Col. 11, Lines 25-42 “the general idea is to apply a spatial filter f() (such as, for example, a truncated Gaussian or other distribution) to the low-resolution solution, S. jointly with a similar range filter g() on the full resolution image, i. Thus, denoting p and q as the integer coordinates of pixels ini, and denoting p and q as the corresponding (possibly fractional) pixel coordinates in the low-resolution solution S. The upsampled solution S is then obtained as illustrated by Equation 3, where: W 1 W. W. E ion 3 S = XI, f(pl.-q1Dg(II – III) As noted above, the output of Equation 3 is a high-resolution solution set that is either saved for later use, or applied to the original high resolution input signal”) (teaches that equation 3 is a linear combination whose weights are estimated with the low resolution chrominance solution set (same pixel grid as low resolution Y). Further teaches those techniques are run per chrominance channel, yielding exactly the sought after linear ‘coefficients of the chrominance transformation’ because both Y and chroma come from the same downsampled image, their sizes inherently match meeting the size clause), wherein the obtaining the coefficients of chrominance transformation by performing the linear transformation on the chrominance based on the downsampled luminance comprises: constructing a guided filter (interpreted as building a filter whose weights are directed by another image)(Cohen: Col. 2, Lines 4-9 “the Joint Bilateral Upsampler described herein operates as a joint bilateral function of a distance measure (spatial filter) between data points of a low-resolution solution set, and a difference (range filter) between data points of the original high-resolution input signal.”)(Cohen: Col. 10, Lines 47-50 “the Joint Bilateral Upsampler described herein provides a novel bilateral filter that operates on two different resolutions to directly address the problem of upsampling.”)(the joint bilateral function is a form of guided filter (spatial + range)); calculating linear coefficients by using the guided filter based on the guidance image and the chrominance (interpreted as computing the local linear weights from that filter)(Cohen: Col. 11, Lines 31-33 “The upsampled solution S is then obtained as illustrated by Equation 3, where:”)(teaches that equation 3 is a linear combination, the weights produced by the joint bilateral (guided) filter are exactly the claimed ‘linear coefficients’); and obtaining the coefficients of chrominance transformation by performing transformation on the chrominance based on the linear coefficients (interpreted as applying those weights to generate full resolution chroma)(Cohen: Col. 13, Lines 6-10 “The resulting high-resolution chrominance Solutions are then recombined to produce a single high-resolution colorization Solution which is then applied to the original high resolution image to be colorized.”)(teaches that once the linear weights have been applied, the chrominance transformation is finished and its coefficients embodied in the high resolution chroma planes) but fails to explicitly disclose taking the downsampled luminance as a guidance image of the guided filter and taking the chrominance as an input image of the guided filter.
However, Budagavi discloses taking the downsampled luminance as a guidance image of the guided filter and taking the chrominance as an input image of the guided filter (interpreted as using the low resolution Y channel to guide up sampling of the chroma planes) (Budagavi: Col. 2, Lines 1-15 “a method for luma-based chroma intra-prediction in a video encoder or a video decoder is provided that includes filtering reconstructed neighboring samples of a reconstructed down sampled luma block, computing parameters C. and B of a linear model using the filtered, reconstructed neighboring samples of the recon structed down sampled luma block and reconstructed neigh boring samples of a corresponding chroma block, wherein the linear model is Pred-x.y=C. Rec'xy+B, wherein X and y are sample coordinates, Pred, is predicted chroma samples, and Rec' is samples of the reconstructed down sampled luma block, and computing samples of a predicted chroma block from corresponding samples of the recon structed down sampled luma block using the linear model and the parameters”) (teaches using down sampled luminance block as the predictor for chroma reconstruction).
Cohen and Budagavi are both considered to be analogous to the claimed invention because they are in the same field of luminance guided chrominance reconstruction for real time image and video processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Cohen to incorporate Budagavi’s teachings of using down sampled luminance block as the guidance image for calculating chroma coefficients. The motivation for such a combination would provide the benefit of reducing memory bandwidth and computation while still achieving sharp chroma reconstruction.
Regarding claim 4, Cohen discloses the method of claim 1, but fails to explicitly disclose wherein the calculating the linear coefficients based on the guidance image and the chrominance comprises: performing a linear transformation on the guidance image to obtain a guided output image; obtaining a target function based on the guided output image and the chrominance; and optimizing the target function to find an optimal solution and obtaining the linear coefficients.
However, Budagavi discloses wherein the calculating the linear coefficients based on the guidance image and the chrominance comprises: performing a linear transformation on the guidance image to obtain a guided output image (interpreted as applying a linear model to the guidance image to create an intermediate output)(Budagavi: Col. 2, Lines 5-12 “computing parameters C. and B of a linear model using the filtered, reconstructed neighboring samples of the recon structed down sampled luma block and reconstructed neigh boring samples of a corresponding chroma block, wherein the linear model is Pred-x.y=C. Rec'xy+B, wherein X and y are sample coordinates, Pred, is predicted chroma samples, and Rec' is samples of the reconstructed down sampled luma block”)(teaches a linear transform of the down sampled luminance to yield a predicted output); obtaining a target function based on the guided output image and the chrominance (interpreted as formulating an error/cost function that compares the linear model output with actual chroma data)(Budagavi: Col. 3, 57-59 “The ordinary least square (OLS) technique, also referred to as the linear least squares technique, is used to derive the parameters C. and B:”) (teaches the OLS technique which corresponds to the ‘target function’ built from model output and chroma); and optimizing the target function to find an optimal solution and obtaining the linear coefficients (interpreted as minimize that function to produce the best coefficients)(Col. 10, Lines “C. and B used to compute the predicted chroma samples. These parameters are derived using the OLS technique.”)(The OLS technique provides an optimal solution for obtaining the coefficients).
Cohen and Budagavi are both considered to be analogous to the claimed invention because they are in the same field of luminance guided chrominance reconstruction for real time image and video processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Cohen to incorporate Budagavi’s teachings of obtaining optimal coefficients and output. The motivation for such a combination would provide the benefit of reducing memory bandwidth and computation while ensuring chroma sharpness.
Claims 7 and 13 are apparatus and computer-readable storage medium claims corresponding to claim 1 above without any additional limitations. Thus, claims 7 and 13 are rejected for the same reason as claim 1.
Claims 10 and 16 are apparatus and computer-readable storage medium claims corresponding to claim 4 above without any additional limitations. Thus, claims 10 and 16 are rejected for the same reason as claim 4.
Claims 5, 11, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Cohen et. al (U.S. Patent No. 7,889,949), in view of Budagavi (U.S. Patent No. 9,723,327), in further view of Westerman (U.S. Patent No. 6,510,242).
Regarding claim 5, Cohen in view of Budagavi disclose the method of claim 1, wherein the obtaining the to-be-displayed text image based on the coefficients of chrominance transformation comprises: upsampling the coefficients of chrominance transformation (interpreted as enlarging the low resolution chroma mapping data to a higher resolution)(Cohen: Col. 13, Lines 3-10 “the Joint Bilateral Upsampler separately applies the upsampling techniques described with respect to Equation 3 to each of the two chrominance channels to produce two high resolution chrominance solutions. The resulting high-resolution chrominance Solutions are then recombined to produce a single high-resolution colorization Solution which is then applied to the original high resolution image to be colorized.”)(teaches the low resolution chrominance ‘solution sets’ (which serve as the chroma transformation data) are upsampled to high resolution, these solution sets correspond as the claimed coefficients); but fail to explicitly disclose obtaining an output image based on the upsampled coefficients of chrominance transformation and the luminance; and converting an encoding mode of the output image and obtaining the to-be-displayed text image.
However, Westerman discloses obtaining an output image based on the upsampled coefficients of chrominance transformation and the luminance (interpreted as combining the upsampled chroma data with luminance data to create an output image)(Westerman: Col. 2, Lines 33-37 “Generally, the present invention provides a method for upsampling a received YCbCr Signal, by generating the missing chrominance values. By operating only on the YCbCr signal, the method of the invention does not require computations on three color channels.”)(Westerman: Col. 2, Lines 13-18 “After upsampling, the image data can be converted to color data in a different color Space. A typical color Space is the red, green, blue (RGB) color space, also used by color television. For Some applications, a high color contrast output image is desirable for presentation to the viewer, as for example in a television receiver”)(teaches that the YCbCr signal inherenrly contains both luminance (Y) and upsampled chrominance (CbCr) coefficients that together form the complete image data for the output image); and converting an encoding mode of the output image and obtaining the to-be-displayed text image (interpreted as converting the format/encoding of the output image for display purposes) (Westerman: Col. 2, Lines 13-18 “After upsampling, the image data can be converted to color data in a different color Space. A typical color Space is the red, green, blue (RGB) color space, also used by color television. For Some applications, a high color contrast output image is desirable for presentation to the viewer, as for example in a television receiver”)(Westerman: Col. 3, Lines 9-11 “The method is particularly useful for Systems that receive a Signal in the luminance chrominance color Space, and con vert it to the RGB color space.”)(teaches converting the encoding mode from YCbCr color space to RGB color space for display).
Cohen, Budagavi, and Westerman are considered to be analogous to the claimed invention because they are in the same field of luminance guided chrominance reconstruction for real time image processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Cohen and Budagavi’s to incorporate Westerman’s teachings of upsampling the coefficients and obtaining an output image. The motivation for such a combination would provide the benefit of enabling proper display on RGB devices and complete the image processing pipeline from upsampling to display.
Claims 11 and 17 are apparatus and computer-readable storage medium claims corresponding to claim 5 above without any additional limitations. Thus, claims 11 and 17 are rejected for the same reason as claim 5.
Claims 6, 12, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Cohen et. al (U.S. Patent No. 7,889,949), in view of Budagavi (U.S. Patent No. 9,723,327), in view of Westerman (U.S. Patent No. 6,510,242), in further view of He et. al (K. He and J. Sun “Fast Guided Filter” arXiv:1505.00996v1 [cs.CV], published May 5th, 2015, 2 pages).
Regarding claim 6, Cohen in view of Budagavi and Westerman disclose the method of claim 5, but fail to explicitly disclose wherein the upsampling the coefficients of chrominance transformation comprises: upsampling the linear coefficients corresponding to the chrominance transformation and enlarging sizes of the linear coefficients to a same size as the luminance.
However, Chang discloses wherein the upsampling the coefficients of chrominance transformation comprises: upsampling the linear coefficients corresponding to the chrominance transformation and enlarging sizes of the linear coefficients to a same size as the luminance (interpreted as enlarge the transforms coefficient maps to luma resolution) (He: Introduction “This method subsamples the filtering input image and the guidance image, computes the local linear coefficients, and upsamples these coefficients. The upsampled coefficients are adopted on the original guidance image to produce the output.”) (He: Page 2 “The two coefficient maps a¯ and ¯b are bilinearly upsampled to the original size. Finally, the output q is still computed by q = ¯aI + ¯b. In this last step, the image I is the full-resolution guidance that is not downsampled”) (teaches upsampling coefficients to image I (which is the guidance) which corresponds to upsampling the coefficients to the size of the luminance).
Cohen, Budagavi, Westerman and He are considered to be analogous to the claimed invention because they are in the same field of luminance guided chrominance reconstruction for image processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Cohen Budagavi’s and Westerman to incorporate He’s teachings of upsampling the coefficients to the size of the luminance. The motivation for such a combination would provide the benefit of pixel aligned application of chrominance transformations at the luminance resolution to reduce the color bleeding and improve text-image clarity.
Claims 12 and 18 are apparatus and computer-readable storage medium claims corresponding to claim 6 above without any additional limitations. Thus, claims 12 and 18 are rejected for the same reason as claim 6.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED TAHA whose telephone number is (571)272-6805. The examiner can normally be reached 8:30 am - 5 pm, Mon - Fri. Examiner interviews are available via telephone, in person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571)272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786- 9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED TAHA/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613