DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on February 05, 2025, February 18, 2025, and February 27, 2025, are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the Examiner.
Response to Amendment
Applicant’s Preliminary Amendment to the Claims and the Abstract, filed on May 24, 2024, has been entered and made of record.
Currently pending Claim(s) 45-64
Independent Claim(s) 45, 57, and 64
Newly Added Claim(s) 45-64
Canceled Claim(s) 1-44
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 53 and 62 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 53, the meaning of the term “preset rule” is not defined. The Specification in [0058] describes a “first preset rule” for splitting a picture into local pictures and a “second preset rule” for obtaining visual sensory experience parameters, and the “preset rule” includes the “first preset rule” and the “second preset rule”. However, the claim language does not mention the “first preset rule” or the “second preset rule,” and the steps contained within these rules are not clearly defined.
Regarding claim 62, the limitation "third preset rule" is recited, but no “second preset rule” was defined in claim 62 or its base claim 57. Thus, there is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 45-46, 48-53, 55-58, 60-62, and 64 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Chen (US 2021/0272238 A1).
Regarding claim 45, Chen teaches a method, comprising:
obtaining a to-be-processed picture (Fig. 3A shows Chen’s method. A source image 313 is the “to-be-processed picture”.);
obtaining a zoom-in operation instruction, wherein the zoom-in operation instruction indicates a to-be-zoomed-in region of the to-be-processed picture (In Fig. 3A, when a user performs a gesture (such as a pinch or double-tap [0034]), a zoom region 317 is specified. Fig. 3B shows a zoomed-in region 317a-b from the to-be-processed picture 313.), and
the to-be-zoomed-in region corresponds to one or more local pictures (Figs. 5-6 show that the zoom region is segmented into segments or textures (local pictures), and each segment is processed and enhanced individually [0066]. Also see [0065-0068], which describes methods for segmenting the zoom region.);
obtaining one or more groups of visual sensory experience parameters corresponding to the one or more local pictures (In steps 303 of Fig. 3A, Chen teaches determining if detail enhancement of the zoomed region is necessary. [0042-0047] teaches various methods of analyzing the zoomed region to determine if the detail should be increased. For example, areas with high-frequency data may need detail enhancement while low-frequency data may not since the texture appearance may not become too pixelated when zoomed-in. Furthermore, [0055] teaches that segments of the image (corresponding local images), are corrected to better match the zoom region. This involves detail correction by replacing the segment with a higher-resolution texture, and that texture may be modified to match the zoom region and surrounding region of the source image (to-be-processed image) by adjusting brightness, luminance, opacity, hue, etc.); and
separately processing a corresponding local picture based on the one or more groups of visual sensory experience parameters to obtain a processed local picture ([0054-0056] teaches processing a local picture by replacing textures or merging existing textures with new textures. Furthermore, corrections to brightness, luminance, hue, etc. are made, so the enhanced local picture is coherent with the to-be-processed-picture.).
Regarding claim 46, Chen teaches the method according to claim 45, wherein the one or more groups of visual sensory experience parameters comprise at least one of a luminance parameter, a contrast parameter, a color parameter, or a detail parameter (In steps 303 of Fig. 3A, Chen teaches determining if detail enhancement of the zoomed region is necessary. [0042-0047] teaches various methods of analyzing the zoomed region to determine if the detail should be increased. For example, areas with high-frequency data may need detail enhancement while low-frequency data may not since the texture appearance may not become too pixelated when zoomed-in. Furthermore, when a texture segment (local image) of the zoomed region is corrected, the new texture’s luminance, contrast, and color is adjusted to match the surrounding region of the to-be-processed picture. [0055] “Any characteristic of the substitute texture may be altered to better match a characteristic of the source image before substitution. For example, the color or chrominance of the texture from the texture database may be altered to match the source image region being replaced.”);
wherein the separately processing the corresponding local picture based on the one or more groups of visual sensory experience parameters to obtain the processed local picture comprises at least one of: when the one or more groups of the visual sensory experience parameters comprise the luminance parameter, performing luminance adjustment on the corresponding local picture; when the one or more groups of the visual sensory experience parameters comprise the contrast parameter, performing contrast adjustment on the corresponding local picture; when the one or more groups of the visual sensory experience parameters comprise the color parameter, performing color adjustment on the corresponding local picture (When a texture segment (local picture) of the zoomed region is corrected, the new texture’s luminance, contrast, and color is adjusted to match the surrounding region of the to-be-processed picture. [0055] “Any characteristic of the substitute texture may be altered to better match a characteristic of the source image before substitution. For example, the color or chrominance of the texture from the texture database may be altered to match the source image region being replaced. The replacement texture may be scaled, rotated, skewed, or otherwise dimensionally transformed to match one or more dimensions of the zoom region better. Other such parameters of the substitute texture may be modified or altered to match the source image region such as brightness, luminance, opacity, hue, saturation, tint, shade, or other image parameters.”); or
when the one or more groups of the visual sensory experience parameters comprise the detail parameter, performing detail adjustment on the corresponding local picture ([0055] teaches that texture segments of the picture (corresponding local pictures), are corrected to better match the zoom region. This involves detail correction by replacing the segment with a higher-resolution texture, and that texture may be modified to match the zoom region and surrounding region of the source picture (to-be-processed picture) by adjusting brightness, luminance, opacity, hue, etc.); and
wherein the corresponding local picture is one of the one or more local pictures (Figs. 5-6 show that the zoom region is comprised of multiple textures (local pictures).
Regarding claim 48, Chen teaches the method according to claim 46, wherein the performing the detail adjustment on the corresponding local picture comprises: obtaining a plurality of reference pictures, wherein the plurality of reference pictures and the to-be-processed picture are obtained by shooting a same scene by a plurality of cameras; and performing the detail adjustment on the corresponding local picture based on the plurality of reference pictures (Fig. 8 teaches an example embodiment of the process described in Fig. 3A. In this example, the texture database is obtained from the multiple pictures of the same scene. In [0076] several images of the same chain-link fence are captured with different focal distances. Thus, various references for the same image segment in the same scene are available for use during detail adjustment. See 0074-0078.).
Regarding claim 49, Chen teaches the method according to claim 46, wherein the performing the detail adjustment on the corresponding local picture comprises: obtaining a plurality of historical pictures whose similarities to the to-be-processed picture exceed a preset threshold; and performing the detail adjustment on the corresponding local picture based on the plurality of historical pictures (Fig. 3A describes the overall method. In step 307, a database of historical pictures (texture database 319) is searched for to find a similar image for the detail adjustment step (generate enhanced zoom 309). [0048-0053] explains methods for determining the textures present in the to-be-processed image, searching the texture database for similar images to the zoom region, and using the similar textures for enhancing the detail of the zoom region.).
Regarding claim 50, Chen teaches the method according to claim 45, wherein a picture presented by the processed local picture is used to simulate visual sensory experience for a real scene that is of the to-be- zoomed-in region and that is perceived by human eyes (317b of Fig. 3B shows the zoomed in image. In [0039-0042], Chen explains the importance of realism in zoomed-in regions across different platforms (viewing broadcasts, VR/AR, etc.), and the image enhancement is used to increase the realism of the zoomed-in region.)
Regarding claim 51, Chen teaches the method according to claim 45, wherein: the zoom-in operation instruction is generated through an outward sliding operation of two fingers of a user on the to-be-zoomed-in region; or the zoom-in operation instruction is generated through a tapping operation of the two fingers of the user on the to-be-zoomed-in region (Using gestures, such as sliding two-fingers or double-tapping, for zoom-in is a common feature of touch-screen devices. [0034] “Touch-screen displays such as tablet computers may present intuitive interfaces for viewers to indicate a desire to zoom in on an image. With these interactive displays, a user may indicate a zoom region by a two-finger gesture such as a pinch gesture.”).
Regarding claim 52, Chen teaches the method according to claim 45, wherein the obtaining the one or more groups of visual sensory experience parameters corresponding to the one or more local pictures comprises: decoding an obtained bitstream to obtain the one or more groups of visual sensory experience parameters (In steps 303 of Fig. 3A, Chen teaches determining if detail enhancement of the zoomed region is necessary. [0042-0047] teaches various methods of analyzing the zoomed region to determine if the detail should be increased. For example, areas with high-frequency data may need detail enhancement while low-frequency data may not since the texture appearance may not become too pixelated when zoomed-in. Furthermore, [0055] teaches that segments of the image (corresponding local images), are corrected to better match the zoom region. This involves detail correction by replacing the segment with a higher-resolution texture, and that texture may be modified to match the zoom region and surrounding region of the source image (to-be-processed image) by adjusting brightness, luminance, opacity, hue, etc.).
Regarding claim 53, Chen teaches the method according to claim 45, wherein the obtaining the one or more groups of visual sensory experience parameters corresponding to the one or more local pictures comprises: obtaining the one or more groups of visual sensory experience parameters according to a preset rule (Step 303 of Fig. 3A includes evaluating the zoom region to determine necessary if detail corrections are necessary for the zoom region and the corresponding texture regions (local pictures). Chen provides examples of some methods, such as determining detail correction based on a “pixilation” of the local images [0042], using a Discrete Cosine Transform to analyze if high frequency information is present in a region [0045], etc.).
Regarding claim 55, Chen teaches the method according to claim 45, after the separately processing the corresponding local picture based on the one or more groups of visual sensory experience parameters to obtain the processed local picture, the method further comprising:
displaying the processed local picture; or storing the processed local picture (Fig. 3B shows the relationship between the zoomed image 317b and the to-be-processed image 313. When the user gestures to zoom in the to-be-processed image, the method of Fig. 3A is performed, and the enhanced zoomed image 317b is shown on the display device. See Fig. 0039. Additionally, Fig. 2 shows the device with a display, such as a smartphone, computer, tablet, etc.).
Regarding claim 56, Chen teaches the method according to claim 45, wherein the method further comprises: obtaining a zoom-in termination instruction, wherein the zoom-in termination instruction is generated through an inward sliding operation of two fingers of a user on the processed local picture, or the zoom-in termination instruction is generated through a tapping operation of a single finger of the user on the processed local picture; and displaying the to-be-processed picture based on the zoom-in termination instruction (Using gestures, such as sliding two-fingers or double-tapping, for zoom-in is a common feature of touch-screen devices. [0034] “Touch-screen displays such as tablet computers may present intuitive interfaces for viewers to indicate a desire to zoom in on an image. With these interactive displays, a user may indicate a zoom region by a two-finger gesture such as a pinch gesture.”).
Regarding claim 57, Chen teaches a method, comprising:
obtaining a to-be-processed picture (Fig. 3A shows Chen’s method. A source image 313 is the “to-be-processed picture”.);
splitting the to-be-processed picture to obtain a plurality of candidate local pictures (At Fig. 8, Chen teaches a method similar to the method of Fig. 3A, but here, the to-be-processed picture is split into segments or textures before the zoom in operation. Figs. 5-6 show that a zoomed region involves multiple textures (local pictures).);
obtaining a plurality of groups of visual sensory experience parameters, wherein the plurality of groups of visual sensory experience parameters correspond to the plurality of candidate local pictures (In steps 303 of Fig. 3A, Chen teaches determining if detail enhancement of the zoomed region is necessary. [0042-0047] teaches various methods of analyzing the zoomed region to determine if the detail should be increased. For example, areas with high-frequency data may need detail enhancement while low-frequency data may not since the texture appearance may not become too pixelated when zoomed-in. Furthermore, [0055] teaches that segments of the image (corresponding local images), are corrected to better match the zoom region. This involves detail correction by replacing the segment with a higher-resolution texture, and that texture may be modified to match the zoom region and surrounding region of the source image (to-be-processed image) by adjusting brightness, luminance, opacity, hue, etc.);
separately processing a corresponding candidate local picture based on the plurality of groups of visual sensory experience parameters to obtain a plurality of processed candidate local pictures ([0054-0056] teaches processing a local picture by replacing textures or merging existing textures with new textures. Furthermore, corrections to brightness, luminance, hue, etc. are made, so the enhanced local picture is coherent with the to-be-processed-picture.); and
encoding the to-be-processed picture and the plurality of processed candidate local pictures ([0049] “And finally, at step 309 the substitute texture may be integrated into the zoom region, producing an enhanced zoom image.” Chen teaches accessing video and/or picture data over a bitstream, processing textures over a bitstream, and displaying a video and/or picture with enhanced textures on a user end device. Thus, the video and/or image (to-be processed picture) and the textures (candidate local pictures) must undergo encoding/decoding for transmission.).
Regarding claim 58, Chen teaches the method according to claim 57,
wherein the plurality of groups of visual sensory experience parameters comprise at least one of a luminance parameter, a contrast parameter, a color parameter, or a detail parameter (In steps 303 of Fig. 3A, Chen teaches determining if detail enhancement of the zoomed region is necessary. [0042-0047] teaches various methods of analyzing the zoomed region to determine if the detail should be increased. For example, areas with high-frequency data may need detail enhancement while low-frequency data may not since the texture appearance may not become too pixelated when zoomed-in. Furthermore, when a texture segment (local image) of the zoomed region is corrected, the new texture’s luminance, contrast, and color is adjusted to match the surrounding region of the to-be-processed picture. [0055] “Any characteristic of the substitute texture may be altered to better match a characteristic of the source image before substitution. For example, the color or chrominance of the texture from the texture database may be altered to match the source image region being replaced.”);
wherein the separately processing the corresponding candidate local picture based on the plurality of groups of visual sensory experience parameters to obtain the plurality of processed candidate local pictures comprises at least one of:
when the plurality of groups of visual sensory experience parameters comprise the luminance parameter, performing luminance adjustment on the corresponding local picture; when the plurality of groups of visual sensory experience parameters comprise the contrast parameter, performing contrast adjustment on the corresponding local picture; when the plurality of groups of visual sensory experience parameters comprise the color parameter, performing color adjustment on the corresponding local picture (When a texture segment (local picture) of the zoomed region is corrected, the new texture’s luminance, contrast, and color is adjusted to match the surrounding region of the to-be-processed picture. [0055] “Any characteristic of the substitute texture may be altered to better match a characteristic of the source image before substitution. For example, the color or chrominance of the texture from the texture database may be altered to match the source image region being replaced. The replacement texture may be scaled, rotated, skewed, or otherwise dimensionally transformed to match one or more dimensions of the zoom region better. Other such parameters of the substitute texture may be modified or altered to match the source image region such as brightness, luminance, opacity, hue, saturation, tint, shade, or other image parameters.”); or
when the plurality of groups of visual sensory experience parameters comprise the detail parameter, performing detail adjustment on the corresponding local picture ([0055] teaches that texture segments of the picture (corresponding local pictures), are corrected to better match the zoom region. This involves detail correction by replacing the segment with a higher-resolution texture, and that texture may be modified to match the zoom region and surrounding region of the source picture (to-be-processed picture) by adjusting brightness, luminance, opacity, hue, etc.); and
wherein the corresponding local picture is one of the plurality of candidate local pictures (Figs. 5-6 show that the zoom region is comprised of multiple textures (local pictures).
Regarding claim 60, Chen teaches the method according to claim 58, wherein the performing the detail adjustment on the corresponding local picture comprises: obtaining a plurality of reference pictures, wherein the plurality of reference pictures and the to-be-processed picture are obtained by shooting a same scene by a plurality of cameras; and performing the detail adjustment on the corresponding local picture based on the plurality of reference pictures (Fig. 8 teaches an example embodiment of the process described in Fig. 3A. In this example, the texture database is obtained from the multiple pictures of the same scene. In [0076] several images of the same chain-link fence are captured with different focal distances. Thus, various references for the same image segment in the same scene are available for use during detail adjustment. See 0074-0078.).
Regarding claim 61, Chen teaches the method according to claim 58, wherein the performing the detail adjustment on the corresponding local picture comprises: obtaining a plurality of historical pictures whose similarities to the to-be-processed picture exceed a preset threshold; and performing the detail adjustment on the corresponding local picture based on the plurality of historical pictures (Fig. 3A describes the overall method. In step 307, a database of historical pictures (texture database 319) is searched for to find a similar image for the detail adjustment step (generate enhanced zoom 309). [0048-0053] explains methods for determining the textures present in the to-be-processed image, searching the texture database for similar images to the zoom region, and using the similar textures for enhancing the detail of the zoom region.).
Regarding claim 62, Chen teaches the method according to claim 57, wherein the obtaining the plurality of groups of visual sensory experience parameters comprises: obtaining the plurality of groups of visual sensory experience parameters according to a third preset rule (The meaning of a “third preset rule” is not clear. See the 35 USC 112(b) rejection. Here, the Examiner provides the same rejection as to claim 53. Step 303 of Fig. 3A includes evaluating the zoom region to determine necessary if detail corrections are necessary for the zoom region and the corresponding texture regions (local pictures). Chen provides examples of some methods, such as determining detail correction based on a “pixilation” of the local images [0042], using a Discrete Cosine Transform to analyze if high frequency information is present in a region [0045], etc.).
Regarding claim 64, Chen teaches a device, comprising: one or more processors, and a non-transitory computer-readable storage medium, coupled to the one or more processors and storing a program executed by the one or more processors, wherein when the program is executed by the one or more processors (See Figs. 1-2 showing a user end-device (tablet, smartphone, TV, etc. and servers for implementing the method of Fig. 3A. Chen’s methods can be processed on a server or on a user-end device as shown in different embodiments.), the device is enabled to perform operations including:
obtaining a to-be-processed picture (Fig. 3A shows Chen’s method. A source image 313 is the “to-be-processed picture”.);
splitting the to-be-processed picture to obtain a plurality of candidate local pictures (In steps 303 of Fig. 3A, Chen teaches determining if detail enhancement of the zoomed region is necessary. [0042-0047] teaches various methods of analyzing the zoomed region to determine if the detail should be increased. For example, areas with high-frequency data may need detail enhancement while low-frequency data may not since the texture appearance may not become too pixelated when zoomed-in. Furthermore, [0055] teaches that segments of the image (corresponding local images), are corrected to better match the zoom region. This involves detail correction by replacing the segment with a higher-resolution texture, and that texture may be modified to match the zoom region and surrounding region of the source image (to-be-processed image) by adjusting brightness, luminance, opacity, hue, etc.);
obtaining a plurality of groups of visual sensory experience parameters, wherein the plurality of groups of visual sensory experience parameters correspond to the plurality of candidate local pictures (In steps 303 of Fig. 3A, Chen teaches determining if detail enhancement of the zoomed region is necessary. [0042-0047] teaches various methods of analyzing the zoomed region to determine if the detail should be increased. For example, areas with high-frequency data may need detail enhancement while low-frequency data may not since the texture appearance may not become too pixelated when zoomed-in. Furthermore, [0055] teaches that segments of the image (corresponding local images), are corrected to better match the zoom region. This involves detail correction by replacing the segment with a higher-resolution texture, and that texture may be modified to match the zoom region and surrounding region of the source image (to-be-processed image) by adjusting brightness, luminance, opacity, hue, etc.);
separately processing a corresponding candidate local picture based on the plurality of groups of visual sensory experience parameters to obtain a plurality of processed candidate local pictures ([0054-0056] teaches processing a local picture by replacing textures or merging existing textures with new textures. Furthermore, corrections to brightness, luminance, hue, etc. are made, so the enhanced local picture is coherent with the to-be-processed-picture.); and
encoding the to-be-processed picture and the plurality of processed candidate local pictures ([0049] “And finally, at step 309 the substitute texture may be integrated into the zoom region, producing an enhanced zoom image.” Chen teaches accessing video and/or picture data over a bitstream, processing textures over a bitstream, and displaying a video and/or picture with enhanced textures on a user end device. Thus, the video and/or image (to-be processed picture) and the textures (candidate local pictures) must undergo encoding/decoding for transmission.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 47 and 59 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (US 2021/0272238 A1), and further in view of Gijsenij et al. (Color Constancy for Multiple Light Sources. IEEE Transactions on Image Processing, vol. 21, no. 2, pp. 697-707), hereafter Gijsenij.
Regarding claim 47, Chen teaches the method according to claim 45, wherein the separately processing the corresponding local picture based on the one or more groups of visual sensory experience parameters to obtain the processed local picture comprises:
when the corresponding local picture corresponds to a dark region of the to-be-processed picture, performing at least one of luminance improvement, contrast improvement, color adaptation for the dark region, or underexposure detail increase on the corresponding local picture; when the corresponding local picture corresponds to a bright region of the to-be- processed picture, performing at least one of luminance reduction, contrast reduction, color adaptation for the bright region, and overexposure detail increase on the corresponding local picture (When a texture segment (local picture) of the zoomed region is corrected, the new texture is adjusted to match the surrounding region of the to-be-processed picture. [0055] “Any characteristic of the substitute texture may be altered to better match a characteristic of the source image before substitution. For example, the color or chrominance of the texture from the texture database may be altered to match the source image region being replaced. The replacement texture may be scaled, rotated, skewed, or otherwise dimensionally transformed to match one or more dimensions of the zoom region better. Other such parameters of the substitute texture may be modified or altered to match the source image region such as brightness, luminance, opacity, hue, saturation, tint, shade, or other image parameters.”); and
wherein the corresponding local picture is one of the one or more local pictures (Figs. 5-6 show that the zoom region is comprised of multiple textures (local pictures)).
Although Chen teaches adjusting the luminance, contrast, and hue of a local picture based on the surrounding region [0055], Chen does not teach the specific methods used for performing these adjustments. Therefore, Chen fails to teach when the corresponding local picture corresponds to a picture subject region, performing color adaptation for a subject on the corresponding local picture.
However, Gijsenij teaches when the corresponding local picture corresponds to a picture subject region, performing color adaptation for a subject on the corresponding local picture (Gijsenij teaches methods for color adaptation of an image; there are many well-known color adaption algorithms, such as the von Kris transform [Section 2]. Gikseni teaches performing color adaptation for images with multiple light sources. Giksenij teaches that an image can be broken down into many patches, color adaptation can be performed on the patches, the illuminant of each patch can be estimated, and the patches with the same (or a similar) illuminant can be grouped to perform color adaptation for the image [Section 3].).
Chen and Gijsenij are analogous in the art, because both teach methods of enhancing local regions of an image to maintain consistent luminance and balancing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply a color adaptation method, such as Gijsenij’s method, for correcting the color of a local picture. Gijsenij shows that color adaptation methods are well-known and well-established in the art. Furthermore, color adaptation methods are applied to local (patch-based) areas for maintaining color constancy under different lighting sources across a subject area ([Section 1] “To construct color constant images from scenes that are recorded under multiple sources, the proposed methodology makes use of local image patches, rather than the entire image. These image patches are assumed to have (local) uniform spectral illumination and can be selected by any sampling method. In this paper, grid-based sampling, key-point-based sampling, and segmentation-based sampling are evaluated. After sampling of the patches, illuminant estimation techniques are applied to obtain local illuminant estimates, and these estimates are combined into more robust estimations.”).
Regarding claim 59, Chen teaches the method according to claim 57, wherein the separately processing the corresponding candidate local picture based on the plurality of groups of visual sensory experience parameters to obtain the plurality of processed candidate local pictures comprises:
when the corresponding local picture corresponds to a dark region of the to-be- processed picture, performing at least one of luminance improvement, contrast improvement, color adaptation for the dark region, or underexposure detail increase on the corresponding local picture; when the corresponding local picture corresponds to a bright region of the to-be- processed picture, performing at least one of luminance reduction, contrast reduction, color adaptation for the bright region, and overexposure detail increase on the corresponding local picture (When a texture segment (local picture) of the zoomed region is corrected, the new texture is adjusted to match the surrounding region of the to-be-processed picture. [0055] “Any characteristic of the substitute texture may be altered to better match a characteristic of the source image before substitution. For example, the color or chrominance of the texture from the texture database may be altered to match the source image region being replaced. The replacement texture may be scaled, rotated, skewed, or otherwise dimensionally transformed to match one or more dimensions of the zoom region better. Other such parameters of the substitute texture may be modified or altered to match the source image region such as brightness, luminance, opacity, hue, saturation, tint, shade, or other image parameters.”); and
wherein the corresponding local picture is one of the plurality of candidate local pictures (Figs. 5-6 show that the zoom region is comprised of multiple textures (local pictures)).
Although Chen teaches adjusting the luminance, contrast, and hue of a local picture based on the surrounding region [0055], Chen does not teach the specific methods used for performing these adjustments. Therefore, Chen fails to teach when the corresponding local picture corresponds to a picture subject region, performing color adaptation for a subject on the corresponding local picture.
However, Gijsenij teaches when the corresponding local picture corresponds to the to-be-processed picture subject region, performing color adaptation for a subject on the corresponding local picture (Gijsenij teaches methods for color adaptation of an image; there are many well-known color adaption algorithms, such as the von Kris transform [Section 2]. Gikseni teaches performing color adaptation for images with multiple light sources. Giksenij teaches that an image can be broken down into many patches, color adaptation can be performed on the patches, the illuminant of each patch can be estimated, and the patches with the same (or a similar) illuminant can be grouped to perform color adaptation for the image [Section 3].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply a color adaptation method, such as Gijsenij’s method, for correcting the color of a local picture. Gijsenij shows that color adaptation methods are well-known and well-established in the art. Furthermore, color adaptation methods are applied to local (patch-based) areas for maintaining color constancy under different lighting sources across a subject area ([Section 1] “To construct color constant images from scenes that are recorded under multiple sources, the proposed methodology makes use of local image patches, rather than the entire image. These image patches are assumed to have (local) uniform spectral illumination and can be selected by any sampling method. In this paper, grid-based sampling, key-point-based sampling, and segmentation-based sampling are evaluated. After sampling of the patches, illuminant estimation techniques are applied to obtain local illuminant estimates, and these estimates are combined into more robust estimations.”).
Claims 54 and 63 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (US 2021/0272238 A1), and further in view of Poynton (Digital Video and HD. The Morgan Kaufmann Series in Computer Graphics. DOI: 10.1016/C2010-0-68987-5).
Regarding claim 54, Chen teaches the method according to claim 45. Furthermore, Chen teaches transmitting video and/or picture data over a network between a user end-device (tablet, smartphone, computer, TV, etc.) and content-providing servers (See Fig. 1). Chen’s method involves receiving a bitstream from a server, and a server may perform the enhancement method described in Fig. 3A when a user intends to zoom-in. Chen teaches receiving the bitstream on the user-end device and viewing the video and/or picture, which would require a decoding or decompression method, but Chen fails to teach wherein the obtaining the to-be-processed picture comprises: performing scalable video decoding on an obtained bitstream to obtain the to-be-processed picture; or performing picture decompression on an obtained picture file to obtain the to-be-processed picture.
However, Poynton teaches wherein the obtaining the to-be-processed picture comprises: performing scalable video decoding on an obtained bitstream to obtain the to-be- processed picture; or performing picture decompression on an obtained picture file to obtain the to-be- processed picture ([Page 547, Section titled Scalable video coding (SVC)] “Scalable video coding– defined in Annex G of H.264– allows conveyance of information structured in a hierarchical manner to allow portions of the bitstream to be extracted at lower bit rate than the complete sequence to enable decoding of pictures with multiple image structures (for sequences encoded with spatial scalability), pictures at multiple picture rates (for sequences encoded with temporal scalability), and/or pictures with multiple levels of image quality (for sequences encoded with SNR/quality scalability).”).
Chen and Poynton are analogous in the art, because both teach methods of transmitting a bitstream of video and/or image data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Chen’s invention by utilizing scalable video coding, because this would implement a well-known standard of video transmission and compression (such as H.264) for receiving the video and/or image. Furthermore, utilizing scalable video allows for flexibility with different network conditions and device capabilities ([Page 547, Section titled Scalable video coding (SVC)] “Different layers can be separated into different bitstreams. All decoders access the base stream; more capable decoders can access enhancement streams.”).
Regarding claim 63, Chen teaches the method according to claim 57. Furthermore, Chen teaches transmitting video and/or picture data over a network between a user end-device (tablet, smartphone, computer, TV, etc.) and content-providing servers (See Fig. 1). Chen’s method involves receiving a bitstream from a server, and a server may perform the enhancement method described in Fig. 3A when a user intends to zoom-in. Chen teaches receiving the bitstream on the user-end device and viewing the video and/or picture, which would require a decoding or decompression method, but Chen fails to teach wherein the obtaining the to-be-processed picture comprises: performing scalable video decoding on an obtained bitstream to obtain the to-be-processed picture; or performing picture decompression on an obtained picture file to obtain the to-be-processed picture.
However, Poynton teaches wherein the encoding the to-be-processed picture and the plurality of processed candidate local pictures comprises: performing scalable video encoding on the to-be-processed picture and the plurality of processed candidate local pictures to obtain a bitstream; or performing picture compression on the to-be-processed picture and the plurality of processed candidate local pictures to obtain a picture file ([Page 547, Section titled Scalable video coding (SVC)] “Scalable video coding– defined in Annex G of H.264– allows conveyance of information structured in a hierarchical manner to allow portions of the bitstream to be extracted at lower bit rate than the complete sequence to enable decoding of pictures with multiple image structures (for sequences encoded with spatial scalability), pictures at multiple picture rates (for sequences encoded with temporal scalability), and/or pictures with multiple levels of image quality (for sequences encoded with SNR/quality scalability).”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Chen’s invention by utilizing scalable video coding, because this would implement a well-known standard of video transmission and compression (such as H.264) for receiving the video and/or image. Furthermore, utilizing scalable video allows for flexibility with different network conditions and device capabilities ([Page 547, Section titled Scalable video coding (SVC)] “Different layers can be separated into different bitstreams. All decoders access the base stream; more capable decoders can access enhancement streams.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Watts et al. (US 9,142,010 B2) teaches a method for performing image enhancement using multiple related images. This method may involve using images of the same scene, but the captured images use different focal lengths.
Grandin et al. (US 9,443,335 B2) teaches a method for producing a zoomed image by merging a color image and an achromatic image with a narrower field of view.
Nashizawa (US 11,521,306 B2) teaches a method for adjusting the luminance and brightness when changing the display area of an image.
Wu et al. (US 2024/0265489 A1) teaches a method for a user to view zoomed-in regions of an image using a magnifying glass effect.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC JAMES SHOEMAKER whose telephone number is (571)272-6605. The examiner can normally be reached Monday through Friday from 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner' s supervisor, JENNIFER MEHMOOD, can be reached at (571)272-2976. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Eric Shoemaker/
Patent Examiner
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights