DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 9-14 are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Dorner (US 9,633,448 B1, hereinafter referenced “Dorner”).
In regards to claim 9. Dorner discloses a system (Dorner, Abstract) comprising:
-one or more memory devices (Dorner, Column 3, lines 37-43; Reference discloses the image processing service 102 may be implemented by one or more computing devices. For example, the image processing service 102 may be implemented by computing devices that include one or more processors to execute one or more instructions stored in memory, and communication devices to transmit and receive data over the network 120);
-and one or more processors coupled to the one or more memory devices that cause the system to perform operations (Dorner, Column 3, lines 37-43; Reference discloses the image processing service 102 may be implemented by one or more computing devices. For example, the image processing service 102 may be implemented by computing devices that include one or more processors to execute one or more instructions stored in memory, and communication devices to transmit and receive data over the network 120) comprising:
-determining hue values of a plurality of colors within a digital image (Dorner, Fig. 3; reference illustrates method in which at step 304 extracting of hue values from is performed from color image data);
-determining significance metrics for the hue values by comparing colors comprising the hue values within the digital image with colors comprising one or more respectively complementary hue values or respectively adjacent hue values within the digital image (Dorner, Column 9, lines 23-27 and lines 30-39; Column 9 discloses at block 308, a distribution of hue values is generated. For example, the distribution can be a histogram representing a number of pixels corresponding to distinct hue values extracted from the color image data, with or without binning (i.e. determining significance metrics for the hue values by comparing colors comprising the hue values within the digital image). Lines 30-39 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image. Various clustering methods, such as k-means, mean-shift, expectation-maximization or the like, can be utilized by the image processing service 102 to group the hues. When applied to certain employed clustering methods, in some embodiments, distance between any two hue values can be defined as the shortest arc or the smallest angle between the two hue values on a corresponding color circle (i.e. with colors comprising one or more respectively complementary hue values or respectively adjacent hue values within the digital image));
-and providing, for display via a user interface on a client device and based on the significance metrics (Dorner, Column 6, lines 50-63; Reference discloses the interface module 212 can be configured to facilitate generating one or more user interfaces through which an image source provider 130, a third party consumer 140 or a third party data provider 150 utilizing a compatible computing device (i.e. client device), may send to, or receive from, the image processing service 102 image data, palette data, instruction data, metadata, color or hue related data….including obtaining color image data, determining representative hue, determining representative colors, identifying corresponding color names…..the user interface can be implemented as a graphical user interface (GUI) (i.e. providing, for display via a user interface and based on the significance distribution)), one or more dominant hue values for the digital image (Dorner, Column 8, lines 57-61 and Column 9, lines 30-32; Column 8 discloses in some embodiments, the color image data obtained is in the form of a color palette. As described earlier, a color palette includes a collection of color values and their associated weights. The color values may also be vectors in a color space, such as an RGB space. Column 9 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image (i.e. representative hues interpreted as one or more dominant hue values in the digital image)).
In regards to claim 10. Dorner discloses the system of claim 9.
Dorner further discloses
-wherein determining the significance metrics for the hue values further comprises: determining a first respectively adjacent hue value for a particular hue value by adding a predetermined constant to the particular hue value; and determining a second respectively adjacent hue value for the particular hue value by subtracting the predetermined constant from the particular hue value (Dorner, Column 9, lines 30-48; Reference discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image…..For example, the distance between HUE 1 at 10 degrees and HUE 2 at 350 degrees is 20 degrees. Upon grouping or clustering the hues included in the distribution, each group of hue values may be represented by a respective group hue. The group hue may be a merged, single hue value (e.g., a weighted average of the group of hues.) Alternatively or in addition, the group hue may be a range of hue values. The range can be calculated based on various statistics of the respective group of hue values to cover a portion of a corresponding color circle. For example, for each respective group, a corresponding group hue may be a range of two standard deviations centered at the mean hue value of the group (i.e. the range of values interpreted as entailing addition or subtracting of hue values which would be indicated in the various clustering methods)).
In regards to claim 11. Dorner discloses the system of claim 9.
Dorner further discloses
-the operations further comprising weighting the significance metrics for the hue values based on a respective proximity of colors comprising the hue values within the digital image to a center of the digital image or a focal point of the digital image (Dorner, Column 9, lines 30-48; Reference discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image…..For example, the distance between HUE 1 at 10 degrees and HUE 2 at 350 degrees is 20 degrees. Upon grouping or clustering the hues included in the distribution, each group of hue values may be represented by a respective group hue. The group hue may be a merged, single hue value (e.g., a weighted average of the group of hues.) Alternatively or in addition, the group hue may be a range of hue values. The range can be calculated based on various statistics of the respective group of hue values to cover a portion of a corresponding color circle. For example, for each respective group, a corresponding group hue may be a range of two standard deviations centered at the mean hue value of the group (i.e. weighting the significance metrics for the hue values based on a respective proximity of colors comprising the hue values within the digital image to a center of the digital image or a focal point of the digital image)).
In regards to claim 12. Dorner discloses the system of claim 9.
Dorner further discloses
-the operations further comprising weighting the significance metrics for the hue values based on one or more of a chroma value, a lightness value, or a saturation value of colors comprising the hue values within the digital image (Dorner, Column 7, lines 30-43 and Column 9, lines 30-48; Reference at Column 7 discloses FIG. 3 is a flow diagram illustrating an embodiment of a representative hue determination routine implemented by the image processing service 102….For example, the color image data can correspond to an equation or table mapping illumination to x-y coordinates, a pixelized image, or other formats…The pixels can be associated with a value, which can be a vector based on a primary color model (e.g., RGB) or a luminance-chrominance model (e.g., Y'UV, YUV, YCbCr or YPbPr) (i.e. based on one or more of a chroma value, a lightness value, or a saturation value of colors comprising the hue values within the digital image). Column 9 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image…..For example, the distance between HUE 1 at 10 degrees and HUE 2 at 350 degrees is 20 degrees. Upon grouping or clustering the hues included in the distribution, each group of hue values may be represented by a respective group hue. The group hue may be a merged, single hue value (e.g., a weighted average of the group of hues.) Alternatively or in addition, the group hue may be a range of hue values. The range can be calculated based on various statistics of the respective group of hue values to cover a portion of a corresponding color circle. For example, for each respective group, a corresponding group hue may be a range of two standard deviations centered at the mean hue value of the group (i.e. weighting the significance metrics for the hue values)).
In regards to claim 13. Dorner discloses the system of claim 9.
Dorner further discloses
-wherein determining the significance metrics for the hue values further comprises filtering the plurality of colors within the digital image to exclude colors having a lightness value or a chroma value below a predetermined threshold (Dorner, Column 7, lines 56-60 and Column 8, lines 8-15; Reference at Column 7 discloses in some embodiments, obtained color image data may need to be manually, semi-manually, semi-automatically, or automatically assessed and filtered so as to only retain color data relevant to determination of representative hues (i.e. filtering the plurality of colors within the digital image to exclude colors having a lightness value or a chroma value below a predetermined threshold). Column 8 discloses in some embodiments, the image processing service 102 can develop or derive, from the obtained data and/or metadata, contextual criteria applicable to the determination of representative hues, identification of corresponding colors or color names, or other processes disclosed herein. The criteria may indicate various preferences, factors, parameters, thresholds, or requirements that facilitate or control the representative hue determination routine of FIG. 3).
In regards to claim 14. Dorner discloses the system of claim 9.
Dorner further discloses
-the operations further comprising: generating a significance distribution of the hue values for the digital image based on the significance metrics (Dorner, Column 9, lines 23-27 and lines 30-32; Column 9 discloses at block 308, a distribution of hue values is generated. For example, the distribution can be a histogram representing a number of pixels corresponding to distinct hue values extracted from the color image data, with or without binning…At block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image);
-selecting the one or more dominant hue values for the digital image based on the significance distribution (Dorner, Fig. 3; Reference at step 310 discloses grouping and merging of hue values included in the distribution. Step 312 discloses determining a representative hue from the merged hues);
-and providing, for display via the user interface on the client device (Dorner, Column 6, lines 50-63; Reference discloses the interface module 212 can be configured to facilitate generating one or more user interfaces through which an image source provider 130, a third party consumer 140 or a third party data provider 150 utilizing a compatible computing device (i.e. client device), may send to, or receive from, the image processing service 102 image data, palette data, instruction data, metadata, color or hue related data….including obtaining color image data, determining representative hue, determining representative colors, identifying corresponding color names…..the user interface can be implemented as a graphical user interface (GUI) (i.e. providing, for display via a user interface and based on the significance distribution)), the one or more dominant hue values for the digital image in relation to a spectrum of hue values (Dorner, Column 8, lines 57-61 and Column 9, lines 30-32; Column 8 discloses in some embodiments, the color image data obtained is in the form of a color palette. As described earlier, a color palette includes a collection of color values and their associated weights. The color values may also be vectors in a color space, such as an RGB space. Column 9 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image (i.e. representative hues interpreted as one or more dominant hue values in the digital image in relation to the distribution of hue values i.e. spectrum of hue values)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 8, 15-17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Dorner (US 9,633,448 B1) in view of Tan (2018 “Palette-based image decomposition, harmonization, and color transfer”, hereinafter referenced “Tan”).
In regards to claim 1. Dorner discloses a computer-implemented method (Dorner, Abstract) comprising:
-generating a significance distribution of hue values for a digital image based on comparisons of colors comprising the hue values within the digital image (Dorner, Column 7, lines 53-60 and Column 8, lines 25-32; Column 7 discloses illustratively, the image processing service 102 can obtain the color image data by receiving the data from image source providers 130 or by locating and retrieving color palette data from the palette data store 110. In some embodiments, obtained color image data may need to be manually, semi-manually, semi-automatically, or automatically assessed and filtered so as to only retain color data relevant to determination of representative hues (i.e. generating a significance distribution of hue values for a digital image based on comparisons of colors). Column 8 discloses at block 304, the image processing service 102 extracts hue values from the color image data. Illustratively, if an original or pre-processed color image is obtained, hue values can be extracted from color values associated with each pixel of the image (i.e. comprising the hue values within the digital image). The color values may be vectors in a color space as discussed earlier. For example, the color values can be vectors in an RGB, Y'UV, YUV, YCbCr or YPbPr based color space);
-providing, for display via a user interface on a client device and based on the significance distribution (Dorner, Column 6, lines 50-63; Reference discloses the interface module 212 can be configured to facilitate generating one or more user interfaces through which an image source provider 130, a third party consumer 140 or a third party data provider 150 utilizing a compatible computing device (i.e. client device), may send to, or receive from, the image processing service 102 image data, palette data, instruction data, metadata, color or hue related data….including obtaining color image data, determining representative hue, determining representative colors, identifying corresponding color names…..the user interface can be implemented as a graphical user interface (GUI) (i.e. providing, for display via a user interface and based on the significance distribution)), one or more dominant hue values for the digital image in relation to a spectrum of hue values (Dorner, Column 8, lines 57-61 and Column 9, lines 30-32; Column 8 discloses in some embodiments, the color image data obtained is in the form of a color palette. As described earlier, a color palette includes a collection of color values and their associated weights. The color values may also be vectors in a color space, such as an RGB space. Column 9 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image (i.e. representative hues interpreted as one or more dominant hue values in the digital image in relation to the distribution of hue values i.e. spectrum of hue values));
-
Dorner does not explicitly disclose but Tan teaches
-and adjusting, in response to a user interaction with a target hue value of the one or more dominant hue values, a plurality of colors comprising the target hue value within the digital image (Tan, Fig. 10 and “4. Color Harmonization” section, page 8; Reference at Fig. 10 illustrates and describes GUI that allows users to edit palettes and see the resulting layer decomposition in real-time. Videos of live palette editing can be found in the supplemental materials. In this example, the automatic palette (right) becomes sparser as a result of interactive editing. The user edits the automatically generated palette to ensure that the background and hair colors are directly represented (i.e. interpreted as the adjustment based on user interaction with a selected or target hue value of the one or more dominant hue values expressed in the palette with regards to the image shown) As a result, editing the purple haze and hair no longer affects the background color).
Dorner and Tan are combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner to include the palette based color harmonization and transfer features of Tan in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations, applicable to representative hue determination systems such as those taught in Dorner.
In regards to claim 2. Dorner in view of Tan teach the computer-implemented method of claim 1.
Dorner further discloses
-wherein presenting the one or more dominant hue values for the digital image in relation to the spectrum of hue values comprises providing, for display via the user interface on the client device, the one or more dominant hue values in relation to a color wheel comprising the spectrum of hue values (Dorner, Column 9, lines 30-39; Reference discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image. Various clustering methods, such as k-means, mean-shift, expectation-maximization or the like, can be utilized by the image processing service 102 to group the hues. When applied to certain employed clustering methods, in some embodiments, distance between any two hue values can be defined as the shortest arc or the smallest angle between the two hue values on a corresponding color circle (i.e. providing display of the one or more dominant hue values in relation to a color wheel comprising the spectrum of hue values))
In regards to claim 3. Dorner in view of Tan teach the computer-implemented method of claim 2.
Dorner does not explicitly disclose but Tan teaches
-further comprising generating indications of the one or more dominant hue values as one or more axes originating at a center of the color wheel comprising the spectrum of hue values (Tan, fig. 11 and “Template Fitting” section continued, page 9; Reference discloses Figure 11 shows different harmonic templates enforced over the same input image. Additional examples can be found in the supplementary material. Users can control the strength of harmonization via an interpolation parameter, where β = 0 leaves the palette unchanged and β = 1 fully rotates each palette color to lie on its matched axis (Figure 12). In the LCh color space, this affects hue alone. The reference illustrates the dominant hue values in relation to the rotation angles of the templates with the axes (which vary by template) shown extending from a center point of the color wheel).
Dorner and Tan are combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner to include the palette based color harmonization and transfer features of Tan in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations, applicable to representative hue determination systems such as those taught in Dorner.
In regards to claim 4. Dorner in view of Tan teach the computer-implemented method of claim 1.
Dorner further discloses
-further comprising providing, for display within the digital image via the user interface on the client device, one or more color dots respectively associated with the one or more dominant hue values within the digital image with one or more pixel coordinates corresponding to the one or more dominant hue values (Dorner, Column 9, lines 40-48; Reference discloses upon grouping or clustering the hues included in the distribution, each group of hue values may be represented by a respective group hue. The group hue may be a merged, single hue value (e.g., a weighted average of the group of hues.) Alternatively or in addition, the group hue may be a range of hue values. The range can be calculated based on various statistics of the respective group of hue values to cover a portion of a corresponding color circle (i.e. hue weight averages indicated pixel coordinates as representation of respective hue group in a color circle interpreted as display within the digital image via the user interface on the client device, one or more color dots respectively associated with the one or more dominant hue values within the digital image with one or more pixel coordinates corresponding to the one or more dominant hue values)).
In regards to claim 8. Dorner in view of Tan teach the computer-implemented method of claim 1.
Dan does not disclose but Tan teaches
-wherein adjusting the target hue value comprises fixing two or more selected hue values relative to one another within the spectrum of hue values while altering colors comprising the target hue value within the digital image (Tan, Fig. 10 and “4. Color Harmonization” section, page 8; Reference at Fig. 10 illustrates and describes GUI that allows users to edit palettes and see the resulting layer decomposition in real-time. Videos of live palette editing can be found in the supplemental materials. In this example, the automatic palette (right) becomes sparser as a result of interactive editing. The user edits the automatically generated palette to ensure that the background and hair colors are directly represented (i.e. interpreted as the adjustment based on a selected or target hue value of the one or more hue values expressed in the palette with regards to the image shown) As a result, editing the purple haze and hair no longer affects the background color).
Dorner and Tan are combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner to include the palette based color harmonization and transfer features of Tan in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations, applicable to representative hue determination systems such as those taught in Dorner.
In regards to claim 15. Dorner discloses a non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor (Dorner, Column 13, lines 57-67), cause the at least one processor to perform operations comprising:
-generating a significance distribution of hue values for a digital image based on comparisons of colors comprising the hue values within the digital image (Dorner, Column 7, lines 53-60 and Column 8, lines 25-32; Column 7 discloses illustratively, the image processing service 102 can obtain the color image data by receiving the data from image source providers 130 or by locating and retrieving color palette data from the palette data store 110. In some embodiments, obtained color image data may need to be manually, semi-manually, semi-automatically, or automatically assessed and filtered so as to only retain color data relevant to determination of representative hues (i.e. generating a significance distribution of hue values for a digital image based on comparisons of colors). Column 8 discloses at block 304, the image processing service 102 extracts hue values from the color image data. Illustratively, if an original or pre-processed color image is obtained, hue values can be extracted from color values associated with each pixel of the image (i.e. comprising the hue values within the digital image). The color values may be vectors in a color space as discussed earlier. For example, the color values can be vectors in an RGB, Y'UV, YUV, YCbCr or YPbPr based color space);
-providing, for display via a user interface on a client device and based on the significance distribution (Dorner, Column 6, lines 50-63; Reference discloses the interface module 212 can be configured to facilitate generating one or more user interfaces through which an image source provider 130, a third party consumer 140 or a third party data provider 150 utilizing a compatible computing device (i.e. client device), may send to, or receive from, the image processing service 102 image data, palette data, instruction data, metadata, color or hue related data….including obtaining color image data, determining representative hue, determining representative colors, identifying corresponding color names…..the user interface can be implemented as a graphical user interface (GUI) (i.e. providing, for display via a user interface and based on the significance distribution)), one or more dominant hue values for the digital image in relation to a spectrum of hue values (Dorner, Column 8, lines 57-61 and Column 9, lines 30-32; Column 8 discloses in some embodiments, the color image data obtained is in the form of a color palette. As described earlier, a color palette includes a collection of color values and their associated weights. The color values may also be vectors in a color space, such as an RGB space. Column 9 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image (i.e. representative hues interpreted as one or more dominant hue values in the digital image in relation to the distribution of hue values i.e. spectrum of hue values));
Dorner does not disclose but Tan teaches
-and adjusting, in response to a user interaction with a target hue of the one or more dominant hue values, a plurality of colors comprising the target hue within the digital image (Tan, Fig. 10 and “4. Color Harmonization” section, page 8; Reference at Fig. 10 illustrates and describes GUI that allows users to edit palettes and see the resulting layer decomposition in real-time. Videos of live palette editing can be found in the supplemental materials. In this example, the automatic palette (right) becomes sparser as a result of interactive editing. The user edits the automatically generated palette to ensure that the background and hair colors are directly represented (i.e. interpreted as the adjustment based on user interaction with a selected or target hue value of the one or more dominant hue values expressed in the palette with regards to the image shown) As a result, editing the purple haze and hair no longer affects the background color).
Dorner and Tan are combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner to include the palette based color harmonization and transfer features of Tan in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations, applicable to representative hue determination systems such as those taught in Dorner.
In regards to claim 16. Dorner in view of Tan teach the non-transitory computer-readable medium of claim 15.
Dorner further discloses
-wherein generating the significance distribution of hue values for the digital image comprises determining significance metrics for the hue values by comparing colors comprising the hue values within the digital image with colors comprising one or more respectively complementary hue values or respectively adjacent hue values within the digital image (Dorner, Column 9, lines 23-27 and lines 30-39; Column 9 discloses at block 308, a distribution of hue values is generated. For example, the distribution can be a histogram representing a number of pixels corresponding to distinct hue values extracted from the color image data, with or without binning (i.e. determining significance metrics for the hue values by comparing colors comprising the hue values within the digital image). Lines 30-39 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image. Various clustering methods, such as k-means, mean-shift, expectation-maximization or the like, can be utilized by the image processing service 102 to group the hues. When applied to certain employed clustering methods, in some embodiments, distance between any two hue values can be defined as the shortest arc or the smallest angle between the two hue values on a corresponding color circle (i.e. with colors comprising one or more respectively complementary hue values or respectively adjacent hue values within the digital image)).
In regards to claim 17. Dorner in view of Tan teach the non-transitory computer-readable medium of claim 16.
Dorner further discloses
-the operations further comprising adjusting the significance metrics for the hue values based on one or more of a chroma, a lightness, a saturation, or a relative proximity to a center or a focal point of colors comprising the hue values within the digital image (Dorner, Column 7, lines 30-43 and Column 9, lines 30-48; Reference at Column 7 discloses FIG. 3 is a flow diagram illustrating an embodiment of a representative hue determination routine implemented by the image processing service 102….For example, the color image data can correspond to an equation or table mapping illumination to x-y coordinates, a pixelized image, or other formats…The pixels can be associated with a value, which can be a vector based on a primary color model (e.g., RGB) or a luminance-chrominance model (e.g., Y'UV, YUV, YCbCr or YPbPr) (i.e. based on one or more of a chroma value, a lightness value, or a saturation value of colors comprising the hue values within the digital image). Column 9 discloses at block 310, hue values included in the distribution are grouped and merged for determining a hue or range of hues representative of the color image…..For example, the distance between HUE 1 at 10 degrees and HUE 2 at 350 degrees is 20 degrees. Upon grouping or clustering the hues included in the distribution, each group of hue values may be represented by a respective group hue. The group hue may be a merged, single hue value (e.g., a weighted average of the group of hues.) Alternatively or in addition, the group hue may be a range of hue values. The range can be calculated based on various statistics of the respective group of hue values to cover a portion of a corresponding color circle. For example, for each respective group, a corresponding group hue may be a range of two standard deviations centered at the mean hue value of the group (i.e. adjusting the significance metrics for the hue values based on a respective proximity of colors comprising the hue values within the digital image to a center of the digital image or a focal point of the digital image)).
In regards to claim 19. Dorner in view of Tan teach the non-transitory computer-readable medium of claim 15.
Dorner further discloses
-wherein generating the significance distribution comprises: determining significance metrics for the hue values based on the comparisons of colors comprising the hue values within the digital image (Dorner, Column 7, lines 53-60 and Column 8, lines 25-32; Column 7 discloses illustratively, the image processing service 102 can obtain the color image data by receiving the data from image source providers 130 or by locating and retrieving color palette data from the palette data store 110. In some embodiments, obtained color image data may need to be manually, semi-manually, semi-automatically, or automatically assessed and filtered so as to only retain color data relevant to determination of representative hues (i.e. generating a significance distribution of hue values for a digital image based on comparisons of colors). Column 8 discloses at block 304, the image processing service 102 extracts hue values from the color image data. Illustratively, if an original or pre-processed color image is obtained, hue values can be extracted from color values associated with each pixel of the image (i.e. comprising the hue values within the digital image). The color values may be vectors in a color space as discussed earlier. For example, the color values can be vectors in an RGB, Y'UV, YUV, YCbCr or YPbPr based color space); and generating a histogram of the significance metrics spanning the spectrum of hue values (Dorner, Column 9, lines 23-27; Column 9 discloses at block 308, a distribution of hue values is generated. For example, the distribution can be a histogram representing a number of pixels corresponding to distinct hue values extracted from the color image data, with or without binning)).
In regards to claim 20. Dorner in view of Tan teach the non-transitory computer-readable medium of claim 15.
Dorner does not disclose but Tan teaches
-wherein providing the one or more dominant hue values for the digital image comprises providing, for display via the user interface, one or more axes respectively corresponding to the one or more dominant hue values in a color wheel representing the spectrum of hue values with one or more selectable preset configurations of the one or more axes (Tan, Fig. 11 and “Template Fitting” section continued, page 9; Reference discloses Figure 11 shows different harmonic templates enforced over the same input image. Additional examples can be found in the supplementary material. Users can control the strength of harmonization via an interpolation parameter, where β = 0 leaves the palette unchanged and β = 1 fully rotates each palette color to lie on its matched axis (Figure 12). In the LCh color space, this affects hue alone. The reference illustrates the dominant hue values in relation to the rotation angles of the templates with the axes (which vary by template) shown extending from a center point of the color wheel).
Dorner and Tan are combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner to include the palette based color harmonization and transfer features of Tan in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations, applicable to representative hue determination systems such as those taught in Dorner.
Claims 5-7 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Dorner (US 9,633,448 B1) in view of Tan (2018 “Palette-based image decomposition, harmonization, and color transfer”) as applied to claim 1 above, and further in view of Sayre III (US 2015/0379732 A1, hereinafter referenced “Sayre III”).
In regards to claim 5. Dorner in view of Tan teach the computer-implemented method of claim 1.
Dorner and Tan does not disclose but Sayre III teaches
-further comprising providing, for display via the user interface on the client device, one or more suggestions for adjustment of the one or more dominant hue values within the digital image (Sayre III, para [0032]; Reference discloses the color palette can be built by adding an affiliated color to the colors in the palette and then updating the list of affiliated colors to suggest new affiliated colors to add to the updated palette. The resulting color palette can be configured to contain a combination of colors that is visually appealing or preferable because each affiliated color used in generating the color palette has been determined by the community of people to be an appropriate or preferable color companion to the color or colors already in the palette. The palettes generated using the affiliated color process may be used to provide color-related recommendations for colors or colored items that would go well with another color or colored item).
Dorner and Tan are combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner to include the palette based color harmonization and transfer features of Tan in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations, applicable to representative hue determination systems such as those taught in Dorner.
Dorner and Sayre III are also combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner, in view of the palette based color harmonization and transfer features of Tan, to include the image based color recommendation features of Sayre III in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations. Further incorporating the image based color recommendation features of Sayre III allows for use of techniques that recommend images, items, and/or metadata based at least in part on a reference color palette or reference color name to improve visual impressions and a desired look for the content presented to the user, the applicable to representative hue determination and color naming systems such as those taught in Dorner and Tan.
In regards to claim 6. Dorner in view of Tan teach the computer-implemented method of claim 5.
Dorner does not disclose but Tan teaches
-wherein the one or more suggestions for adjustment of the one or more dominant hue values comprise one or more harmonic templates comprising predetermined relationships between colors within the spectrum of hue values (Tan, “7 Color Transfer” section, page 15; Reference discloses this method achieves results where I is recolored so it is harmonic with respect to R, taking into account the overall relevance of each color of the palette. Figure 24 shows results of this approach. We found that this method is good for matching dominant colors, which works better for content without real reference colors (e.g. graphics design or man-made objects)).
Dorner and Sayre III are also combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner, in view of the palette based color harmonization and transfer features of Tan, to include the image based color recommendation features of Sayre III in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations. Further incorporating the image based color recommendation features of Sayre III allows for use of techniques that recommend images, items, and/or metadata based at least in part on a reference color palette or reference color name to improve visual impressions and a desired look for the content presented to the user, the applicable to representative hue determination and color naming systems such as those taught in Dorner and Tan.
In regards to claim 7. Dorner in view of Tan teach the computer-implemented method of claim 5.
Dorner does not disclose but Tan teaches
-wherein at least one of the one or more suggestions for adjustment of the one or more dominant hue values comprise an alignment of at least one of the one or more dominant hue values with a complementary hue value within the spectrum of hue values (Tan, “4.1 Template Fitting” section, page 8; Reference discloses use of axis-based templates where given and input image and its extracted color palette the systems finds the template that is closest to the colors in the extracted color palette in the chroma plane so that the closest axis to each color is determined and the global rotation and additional angles that define the template are solved for (i.e. an alignment of at least one of the one or more dominant hue values with a complementary hue value within the spectrum of hue values via the templates)).
Dorner and Sayre III are also combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner, in view of the palette based color harmonization and transfer features of Tan, to include the image based color recommendation features of Sayre III in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations. Further incorporating the image based color recommendation features of Sayre III allows for use of techniques that recommend images, items, and/or metadata based at least in part on a reference color palette or reference color name to improve visual impressions and a desired look for the content presented to the user, the applicable to representative hue determination and color naming systems such as those taught in Dorner and Tan.
Claim 18 is rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Dorner (US 9,633,448 B1) in view of Tan (2018 “Palette-based image decomposition, harmonization, and color transfer”) as applied to claim 15 above, and further in view of Kim (US 2017/0278246 A1, hereinafter referenced “Kim”).
In regards to claim 18. Dorner in view of Tan teach the non-transitory computer-readable medium of claim 16.
Dorner and Tan does not disclose but Kim teaches
-the operations further comprising adjusting the significance metrics for the hue values based on respective locations of colors comprising the hue values within the digital image according to a saliency map of the digital image (Kim, para [0038] and [0044]; Reference at para [0038] discloses the local saliency-map represents uniformly highlighted regions with well-defined boundaries of high frequencies in the CIELab color space, while the global saliency-map represents spatial distance and color distance through image segmentation in the RGB color space. Para [0044] discloses when the shortest distance pixel is a pixel identical to the foreground mean color value and located in the unknown region, the adaptive tri-map generator 130 may generate an adaptive tri-map by replacing the shortest distance pixel and the pixel which is adjacent to the shortest distance pixel (e.g., 410 in FIG. 4) with a tri-map's foreground mean color value as 4630 in FIG. 4. Here, 402 in FIG. 4 is a tri-map, 404 is an adaptive tri-map, 450 is a foreground region, 460 is an unknown region, and the rest region is a background region).
Dorner and Tan are combinable because they are in the same field of endeavor regarding color composition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner to include the palette based color harmonization and transfer features of Tan in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations, applicable to representative hue determination systems such as those taught in Dorner.
Dorner and Kim are also combinable because they are in the same field of endeavor regarding image analysis techniques. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the hue-based color naming system of Dorner, in view of the palette based color harmonization and transfer features of Tan, to include the object extraction features of Kim in order to provide the user with a system that allows for use of systems and methods for determining and associating representative hues with a color image based on hue as taught by Dorner, while incorporating the palette based color harmonization and transfer features of Tan to allow for use of a palette-based framework for color composition for visual applications via a color wheel user interface to visualize and manipulate colors, providing improved scalability and efficiency for color manipulations. Further incorporating the object extraction features of Kim allows for extracting an object via use of saliency maps and alpha matte functions to distinguish foreground and background elements for more accurate detection, applicable to representative hue determination and image harmonization systems such as those taught in Dorner and Tan.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: See the Notice of References Cited (PTO-892)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TERRELL M ROBINSON whose telephone number is (571)270-3526. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KENT CHANG can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TERRELL M ROBINSON/Primary Examiner, Art Unit 2614
Prosecution Insights