DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
1. The information disclosure statements (IDS) submitted on 1/2/2025 and 6/12/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements have been considered by the examiner.
Double Patenting
2. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
3. Claims 1-13 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. US-12430803-B2 in view of Becker et al. (US-2015/0324394-A1, hereinafter "Becker"). Although the claims at issue are not identical, they are not patentably distinct from each other because they both claim the same subject matter/limitations except for negligible wording/phrasing differences.
4. Regarding claim 1, claim 1 of the present application relates to claim 1 of the US-12430803-B2 patent as follows (underlined sections show the same subject matter/limitations except for negligible wording/phrasing differences):
Current Application limitation:
Claim 1
US-12430803-B2 patent limitation:
Claim 1
1. A method of obtaining a color subpalette from a first moodboard containing one or more digital color images, said method being implemented on a computer or a mobile device with a screen and comprising the steps of:
(1) generating a first color palette with k1 representative colors from the first moodboard,
(2) providing a second color palette with k2 colors,
(3) ranking the colors in the first color palette according to a scoring algorithm, which assigns a score to each of the colors in the first color palette based on its color difference with each of the colors in the second color palette, and
(4) selecting a subset of ksub colors from the first color palette, which have the scores corresponding to the least color difference with the colors in the second color palette, said subset of colors being the color subpalette and displaying said subpalette on the screen, wherein the first color palette is obtained from the first moodboard by a method comprising the steps of:
1. A computer implemented method for determining a predetermined number, k, of representative colours from at least one digital colour image, comprising the steps of:
Current Application limitation:
Claim 1 (cont.)
US-12430803-B2 patent limitation:
Claim 1 (cont.)
(a) obtaining at least one digital color image having a number, p, of pixels each pixel having a color value within an n-dimensional color space;
a) obtaining at least one digital colour image having a number, p, of pixels each pixel having a colour value within an n-dimensional colour space;
(b) defining, irrespective of the obtained image, a predetermined number of K cluster centers, which are distributed along a plurality of lines in cubic RGB color space intersecting at an intersection point located centrally in the cubic RGB color space, wherein K > k1;
b) defining, irrespective of the obtained image, a predetermined number, K, of cluster centres which are distributed along a plurality of lines in cubic RGB colour space intersecting at an intersection point located centrally in the cubic RGB colour space, wherein K>k;
(c) forming clusters by associating for each pixel its color value with the nearest of the cluster centers; and
c) forming clusters by, for each pixel, associating its colour value with the nearest of the cluster centres; and
c1) after forming the clusters, for each cluster, re-determining a cluster center;
after forming the clusters, c1) for each cluster, re-determining a cluster centre;
(d) reducing the number of clusters to k1 by deleting cluster centers and/or merging clusters, wherein deleting comprises:
d) reducing the number of clusters to k by deleting cluster centres and merging clusters, wherein the deleting includes:
- for each cluster, determining the number of pixels associated with that cluster, and
for each cluster, determining the number of pixels associated with that cluster, and
- in case the number of pixels associated with a cluster center is below a predefined pruning threshold, deleting the cluster center; and
wherein merging comprises:
deleting the cluster centres when the number of associated pixels is below or equal to a predefined pruning threshold, and
wherein the merging includes:
- determining a distance between two clusters and
determining a distance between at least two clusters and
- merging those clusters which have a distance that is smaller than a predefined merging threshold;
merging those clusters which have a distance that is smaller than a predefined merging threshold;
wherein the steps c), c1) and d) are iterated; and
wherein the steps c), c1), and d) are iterated; and
Current Application limitation:
Claim 1 (cont.)
US-12430803-B2 patent limitation:
Claim 1 (cont.)
after step d) performing the steps:
d1) determining the number of clusters, and if the determined number of clusters is not equal to k,
d2) adjusting the merging threshold and/or the pruning threshold based on the determined number of clusters;
wherein step d2) includes:
if the number of clusters is higher than k, increasing the merging threshold and/or the pruning threshold; and
if the number of clusters is lower than k, decreasing the merging threshold and/or the pruning threshold; and
(e) defining a respective representative color from each of the resulting k1 clusters,
e) defining a respective representative colour from each of the resulting k clusters.
(f) forming a first color palette from k1 representative colors.
5. As shown above, claim 1 of the US-12430803-B2 patent discloses all of the limitations of claim 1 of the current application with the exception of:
"A method of obtaining a color subpalette from a first moodboard containing one or more digital color images, said method being implemented on a computer or a mobile device with a screen and comprising the steps of:
(1) generating a first color palette with k1 representative colors from the first moodboard,
(2) providing a second color palette with k2 colors,
(3) ranking the colors in the first color palette according to a scoring algorithm, which assigns a score to each of the colors in the first color palette based on its color difference with each of the colors in the second color palette, and
(4) selecting a subset of ksub colors from the first color palette, which have the scores corresponding to the least color difference with the colors in the second color palette, said subset of colors being the color subpalette and displaying said subpalette on the screen, wherein the first color palette is obtained from the first moodboard by a method comprising the steps of:
…
(f) forming a first color palette from k1 representative colors."
Becker, on the other hand, discloses a color palette suggestion system in which a user specifies a keyword and/or a color palette based image search to generate and display suggested color palettes of images satisfying the search criteria. The system scores and ranks colors by how close in volume each of the qualifying image's colors matches the volume of each color in the selected image. Ultimately, a subset of color palettes is generated where the color distance between colors is above a threshold. See Becker, ¶ [0037], [0038], [0044], [0051], [0055], [0059], [0068], [0069], [0070], [0075]. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of the US-12430803-B2 patent to include the disclosure of generating color palettes from a series of images and then generating a suggested color palette based on color rankings of how well colors match, of Becker. The motivation for this modification could have been to help a user find a color palette that matches colors from one or more images. By doing so, this could help a user to create a color theme, such as an ocean theme, that harmonizes specific colors with images. This could ultimately be used to determine what paint colors a user might paint their walls of their home with, for example.
Regarding the “after step d)” limitations unique to US-12430803-B2 that describe steps d1 and d2, it would be obvious to one of ordinary skill in the art to include adjusting a pruning threshold of the color clusters in order to output a determined number of clusters. This could be used as a means to appropriately filter the color clusters to only output the desired number of total colors.
Claim Rejections - 35 USC § 101
6. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
7. Claims 11-13 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Claim 11 recites the following: “A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method of claim 1.” The claim does not recite any other structural limitations. The scope of a computer program product comprising a computer program includes software per se. Software per se constitutes subject matter not eligible for patent, and a claim that includes in its scope embodiments not eligible for patent is rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
9. Claims 1-7 and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Becker et al. (US-2015/0324394-A1, hereinafter "Becker") in view of Zhang et al. (NPL: “Palette-Based Image Recoloring Using Color Decomposition Optimization,” IEEE Transactions on Image Processing, vol. 26, no. 4, pp. 1952–1964, Apr. 2017.), and further in view of Dorner et al. (US-2015/0379743-A1, hereinafter “Dorner”).
10. As per claim 1, Becker discloses: A method of obtaining a color subpalette from a first moodboard containing one or more digital color images, said method being implemented on a computer or a mobile device with a screen and comprising the steps of: (Becker, [0031], “Aspects of the present invention provide a color palette suggestion system for conducting a keyword, and a color palette based image search in response to receiving, on a user interface of a computing device, a user presented keyword and a color palette selection criteria the system comprising: at least one processor and at least one non-volatile computer-readable storage medium encoded with executable instructions that, when executed by the at least one processor, causes the at least one processor to provide a display of searched images, the system comprising: ... (3) a color palette generation unit configured to generate a set of suggested color palettes by iteratively extracting a suggested color palette from each image in the set of pertinent images …”)
(1) generating a first color palette with k1 representative colors from the first moodboard, (Becker, [0031], “Aspects of the present invention provide a color palette suggestion system ... the system comprising: ... (2) an image display unit configured to display the set of pertinent images and the set of colors associated with the set of pertinent images on the user interface; (3) a color palette generation unit configured to generate a set of suggested color palettes by iteratively extracting a suggested color palette from each image in the set of pertinent images;” and [0032], “The first set of operations may comprise: (1) retrieving the top four colors from the first image and the retrieving the color volume scores for the top four colors in the first image ... Note that the “top” colors may be any desired number of colors without departing from the invention. For example, the top five colors may be retrieved.”
(2) providing a second color palette with k2 colors, (Becker, [0044], “For example, if the first palette shown in the results contains a light yellow, light orange, and light brown color. The search results may also include a second palette of light yellow, light orange, and a slightly darker brown colors, or third palette of light yellow, a darker orange, and a darker brown colors.” and [0037], “Further, the method may comprise displaying the set of pertinent images and the set of colors associated with the set of pertinent images on the user interface, generating a set of suggested color palettes by iteratively extracting a suggested color palette from each image in the set of pertinent images, and displaying on the user interface ...”) and [0051], “FIG. 6 depicts an interface 600 displaying the user specified search term “harvest” in the input area 610, and images 620 having color palettes 640. The images 620 are images of harvests, wherein the palette 640 of a given image indicates the colors that are present in the given image. Notably, the images contained in the image search result set displayed on the interface 600 are not classified based on the color of the images in the result set because no color criterion has been specified by the user.”)
(3) ranking the colors in the first color palette according to a scoring algorithm, which assigns a score to each of the colors in the first color palette based on its color difference with each of the colors in the second color palette, and (Becker, [0038], “If the user selects a single color option, the search results may be updated to show only images that match the keyword and the single color. The color is matched against the weighted colors retrieved from each image and secondarily on the volume of the color in each image. In particular, the disclosed method may perform the following operations if a single color is selected via a single color selection option: (1) retrieving the set of pertinent images from the first database satisfying the keyword criterion; (2) comparing the single color to the combination of the color weight and the color volume score in each image that contains the selected color; and (3) displaying the result set comprising the images that match the keyword criterion and the single color on the user interface.” and [0055], “Notably, in one embodiment of the disclosed invention, any number of colors may be selected while configuring the color palette selection criteria. The add color button 540 may be displayed until the user has selected the predetermined number of colors to specify the color palette selection criteria colors.” and [0032], “The first set of operations may comprise: (1) retrieving the top four colors from the first image and the retrieving the color volume scores for the top four colors in the first image, (2) searching the second database to locate the set of pertinent images satisfying the keyword and a threshold color volume score criteria, and (3) generating for each color, a ranked display of the set of pertinent images based on the color volume scores of each image in the set of pertinent images.” and [0075], “At event 1930, the disclosed method may use the color volume scores generated in the event 1920 to weigh the colors in the query proportionally to the selected image. Accordingly, the 4 colors may be entered into the search query along with the color volume scores as boost parameters. The method may return a ranked result set containing images that match with all of the colors in the selected palette and the search term, wherein the result set is ranked based on how close in volume each color in each image matches the volume of each color in the selected image.” and [0070], “The steps of common color identification and index generation is illustrated in an equation format in FIG. 14, where common colors are indicated by the reference numeral 1410 and the generated index is indicated by the reference numeral 1420. The disclosed method may then iteratively process each color in the generated index i.e. red, light red, dark red, darker red and yellow colors. In particular, the method may, using the Euclidean distance of the RGB color values, determine the color distance between a given color and the colors in the index other than the given color. The method may delete a specific palette from the index if the distance between two of the colors is less than a predetermined threshold value.”; Examiner’s note: Although the example in Becker, [0038], illustrates scoring a single color against selected colors from other images, [0055] discloses that “any number of colors may be selected while configuring the color palette selection criteria” by a user.)
(4) selecting a subset of ksub colors from the first color palette, which have the scores corresponding to the least color difference with the colors in the second color palette, said subset of colors being the color subpalette and displaying said subpalette on the screen, wherein the first color palette is obtained from the first moodboard by a method comprising the steps of: (Becker, [0038], “In particular, the disclosed method may perform the following operations … (2) comparing the single color to the combination of the color weight and the color volume score in each image that contains the selected color; and (3) displaying the result set comprising the images that match the keyword criterion and the single color on the user interface.” and [0032], “The first set of operations may comprise: … (3) generating for each color, a ranked display of the set of pertinent images based on the color volume scores of each image in the set of pertinent images.” and [0075], “The method may return a ranked result set containing images that match with all of the colors in the selected palette and the search term, wherein the result set is ranked based on how close in volume each color in each image matches the volume of each color in the selected image.” and [0059], “A color palette 420 may also be displayed on the interface 400 showing colors displayed in the image 440. In a preferred embodiment of the disclosed invention, the order of the colors displayed in the color palette is of significance, in particular, the color having highest weight is displayed in the leftmost position in the color palette while the color having lowest weight is displayed in the rightmost position in the color palette.”)
(a) obtaining at least one digital color image having a number, p, of pixels each pixel having a color value within an n-dimensional color space; (Becker, [0031], “Aspects of the present invention provide a color palette suggestion system ... the system comprising: ... (2) an image display unit configured to display the set of pertinent images and the set of colors associated with the set of pertinent images on the user interface;” and [0045], “The term “image search” referred to in this application implies searching any type of digital image file format for which RGB color data can be extracted.” and [0046], “... in contrast to a jpg image file, which may become blurry or pixelated when enlarged.”)
[[(b) defining, irrespective of the obtained image, a predetermined number of K cluster centers, which are distributed along a plurality of lines in cubic RGB color space intersecting at an intersection point located centrally in the cubic RGB color space, wherein K > k1;]]
[[(c) forming clusters by associating for each pixel its color value with the nearest of the cluster centers; and]]
[[c1) after forming the clusters, for each cluster, re-determining a cluster center;]]
[[(d) reducing the number of clusters to k1 by deleting cluster centers and/or merging clusters, wherein deleting comprises:]]
[[- for each cluster, determining the number of pixels associated with that cluster, and]]
[[- in case the number of pixels associated with a cluster center is below a predefined pruning threshold, deleting the cluster center; and]]
[[wherein merging comprises:]]
[[- determining a distance between two clusters and]]
[[- merging those clusters which have a distance that is smaller than a predefined merging threshold;]]
[[wherein the steps c), c1) and d) are iterated; and]]
[[(e) defining a respective representative color from each of the resulting k1 clusters,]]
(f) forming a first color palette from k1 representative colors. (Becker, [0068], “In one embodiment of the disclosed invention if each palette in the search result set has N colors, the method may create an index of all combinations of N−1 colors.” and [0069], “Notably, each of the palettes 1-5 shown in table 1300 has 3 colors, so the method may determine the value of N to be three. Accordingly, the method according to the preferred embodiment of the disclosed invention may create an index of palettes that have (N−1) common colors, since the value of N is three in the example shown the FIG. 13, the disclosed method may (1) identify the common colors contained in the palettes 1-5 and (2) create an index of palettes that have two common colors.”)
11. Becker doesn't explicitly disclose but Zhang discloses: (b) defining, irrespective of the obtained image, a predetermined number of K cluster centers, which are distributed along a plurality of lines in cubic RGB color space intersecting at an intersection point located centrally in the cubic RGB color space, wherein K > k1; (Zhang, Fig. 2; p. 1955, ¶ 1, “In particular, a desirable palette is expected to have the following three properties. It should consist of a few most representative colors that abstract the main color categories in the source image.” and p. 1955, ¶ 6, “To extract k palette colors free of the above mentioned bias, we begin by selecting 8 (the upper bound of the palette size) deterministic initial centers, and then perform k-means clustering with the 8 initial centers.” and p. 1955, ¶ 3, “We first perform color quantization in RGB color space by assigning all colors to bins in a 16 × 16 × 16 3D histogram, which helps to reduce the number of colors to 163 = 4096.” and p. 1957, ¶ 5, “With this accelerated implementation, we can ... reduce the time consumption for decomposing this 1024×1024 color image from 8 minutes to 2 seconds.”; Examiner’s note: Fig. 2 shows the palette extraction into RGB space. In addition, the initial centers of clusters in 3D RGB color space may be distributed in 16 x 16 x 16 grid points. There will be lines of centers intercepting the center point of the 16 x 16 x 16 grid cube.)
(c) forming clusters by associating for each pixel its color value with the nearest of the cluster centers; and (Zhang, p. 1955, ¶ 3, “We first perform color quantization in RGB color space by assigning all colors to bins in a 16 × 16 × 16 3D histogram, which helps to reduce the number of colors to 163 = 4096. ... Each quantized color ci now actually represents ni pixels that assigned to that bin.”; Examiner’s note: Each bin my represent a color cluster.)
c1) after forming the clusters, for each cluster, re-determining a cluster center; (Zhang, p. 1955, ¶ 4, “We repeatedly perform above center selection until k centers have been selected. Then we use the k-means approach to obtain k color clusters …”)
(d) reducing the number of clusters to k1 by deleting cluster centers and/or [[merging clusters,]] wherein deleting comprises: (See Zhang, p. 1955, ¶ 3 below.)
- for each cluster, determining the number of pixels associated with that cluster, and (See Zhang, p. 1955, ¶ 3 below.)
- in case the number of pixels associated with a cluster center is below a predefined pruning threshold, deleting the cluster center; and (Zhang, p. 1955, ¶ 3, “We first perform color quantization in RGB color space by assigning all colors to bins in a 16 × 16 × 16 3D histogram, which helps to reduce the number of colors to 163 = 4096. ... Each quantized color ci now actually represents ni pixels that assigned to that bin. We further reduce the number of colors by repeatedly removing the least frequently occurring color (namely color corresponds to the smallest ni) ...”)
[[wherein merging comprises:]]
[[- determining a distance between two clusters and]]
[[- merging those clusters which have a distance that is smaller than a predefined merging threshold;]]
wherein the steps c), c1) and d) are iterated; and (See Zhang, p. 1955, ¶ 4 below.)
(e) defining a respective representative color from each of the resulting k1 clusters, (Zhang, p. 1955, ¶ 4, “We repeatedly perform above center selection until k centers have been selected. Then we use the k-means approach to obtain k color clusters …”)
12. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Becker to include the disclosure of defining a predetermined number of K cluster centers, which are distributed along a plurality of lines in cubic RGB color space, forming clusters by associating for each pixel its color value with the nearest of the cluster centers, deleting clusters that are below a pruning threshold, and defining a respective representative color from each of the clusters, of Zhang. The motivation for this modification could have been to help consolidate clustered representative colors from a palette so that the final clusters best represent the overall colors of an image or moodboard. By doing so, this can help a user focus on the most relevant colors related to a representative color palette.
13. Becker in view of Zhang doesn't explicitly disclose but Dorner discloses: [[(d) reducing the number of clusters to k1 by deleting cluster centers and/or]] merging clusters, [[wherein deleting comprises:]] (See Dorner, Fig. 3, [0032] and [0047] below.)
wherein merging comprises: (See Dorner, Fig. 3, [0032] and [0047] below.)
- determining a distance between two clusters and (See Dorner, Fig. 3, [0032] and [0047] below.)
- merging those clusters which have a distance that is smaller than a predefined merging threshold; (Dorner, Fig. 3; [0032], “For example, the palette generation criteria can identify a color distance formula or can include one or more thresholds of color distance for merging similar colors when representative colors are identified from a color image.” and [0047], “At block 406, similar color values can optionally be merged or binned in various ways. In some embodiments, one or more thresholds can be specified or automatically determined so that similar color values are binned. One or more color similarity measures (e.g., calculated based on a color distance formula, such as Euclidean distance in RGB based color space) can be used to quantify color similarity for the purpose of binning color values.”; Examiner’s note: As disclosed by Dorner, when the Euclidean distance in RGB color space is smaller than a threshold or the color similarity is high, the colors are merged.)
14. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Becker in view of Zhang to include the disclosure of merging color clusters when this cluster distance is smaller than a threshold, of Dorner. The motivation for this modification could have been to help consolidate clustered representative colors from a palette so that the final clusters best represent the overall colors of an image or moodboard. By doing so, this can help a user focus on the most relevant colors related to a representative color palette.
15. As per claim 2, Becker in view of Zhang, and further in view of Dorner discloses: The method of claim 1, wherein the second color palette is generated from a second moodboard containing one or more digital color images. (Becker, [0037], “Further, the method may comprise displaying the set of pertinent images and the set of colors associated with the set of pertinent images on the user interface, generating a set of suggested color palettes by iteratively extracting a suggested color palette from each image in the set of pertinent images, and displaying on the user interface ...”) and [0051], “FIG. 6 depicts an interface 600 displaying the user specified search term “harvest” in the input area 610, and images 620 having color palettes 640. The images 620 are images of harvests, wherein the palette 640 of a given image indicates the colors that are present in the given image. Notably, the images contained in the image search result set displayed on the interface 600 are not classified based on the color of the images in the result set because no color criterion has been specified by the user.”)
16. As per claim 3, Becker in view of Zhang, and further in view of Dorner discloses: The method of claim 2, wherein the digital color images in the second color palette represent a design style, word or phrase. (Becker, [0037], “Further, the method may comprise displaying the set of pertinent images and the set of colors associated with the set of pertinent images on the user interface, generating a set of suggested color palettes by iteratively extracting a suggested color palette from each image in the set of pertinent images, and displaying on the user interface ...”) and [0051], “FIG. 6 depicts an interface 600 displaying the user specified search term “harvest” in the input area 610, and images 620 having color palettes 640. The images 620 are images of harvests, wherein the palette 640 of a given image indicates the colors that are present in the given image. Notably, the images contained in the image search result set displayed on the interface 600 are not classified based on the color of the images in the result set because no color criterion has been specified by the user.”)
17. As per claim 4, Becker in view of Zhang, and further in view of Dorner discloses: The method of claim 1, wherein the colors in the second color palette are selected by a designer. (Becker, [0004]-[0005], “2. Brief Description of Related Art Searching pertinent images over the Internet can be a particularly challenging task for artists and designers. ... For example, when an artist is searching for the keyword “ocean,” the Internet search engine may return a large number of images associated with the keyword ocean in various colors and shades. … An artist may be required to conduct a complex search query to perform an image search based on a keyword and a color. In that case, artists may have to create and discover color palettes to meet their business needs or aesthetic tastes. For example, when an artist is searching for the keyword “ocean,” the Internet search engine may return a large number of images associated with the keyword ocean in various colors and shades.” and [0007], “Accordingly a need exists whereby a user can specify the desired color pallet indicating not only specific colors and shades but a combination of the desired colors and shades in conjunction with specifying a keyword search criteria for precisely locating a desired image.” and [0051], “FIG. 6 depicts an interface 600 displaying the user specified search term “harvest” in the input area 610, and images 620 having color palettes 640. The images 620 are images of harvests, wherein the palette 640 of a given image indicates the colors that are present in the given image. Notably, the images contained in the image search result set displayed on the interface 600 are not classified based on the color of the images in the result set because no color criterion has been specified by the user.” and [0060]-[0061], “If the user selects the browse color palettes option 730, then the interface 200 shown in FIG. 200 is displayed whereby a set of images of various colors are displayed to the user, and the user can select one of the images to view additional images having the same color palette as the color palette of the selected image. When the browse color palettes option 730 is selected the disclosed method may update the image search results on the user interface to only show palettes that match the selected colors. In other words, if the user clicks on a palette suggestion the result set may be updated to only show images that match the given color combination. Additionally, the palette suggestions on the user interface may be updated to show multicolor palettes that contain the selected colors.”; Examiner’s note: Becker discloses in the “Brief Description of Related Art,” that artists and designers may face challenges when trying to search for color themes when performing an image search. Becker further states in [0007] that a need exists for a proposed solution for “users” (which at least would include artists and designers). Disclosed in ¶ [0051], a “user” specifies the term “harvest” which generates a number of “first” color palettes associated with images and in ¶ [0060]-[0061], allows a “user” (designer) to browse and select a “second” palette from the generated palettes where “the result set may be updated to only show images that match the given color combination.”)
18. As per claim 5, Becker in view of Zhang, and further in view of Dorner discloses: The method of claim 1, wherein the scoring algorithm comprises the steps of, for each color of the first color palette, calculating a color difference with each of the colors in the second color palette, determining the least color difference from all the obtained color differences and assigning a score to the color from the first color palette based on this least color difference, resulting in assigning scores to all of the colors of the first color palette. (Becker, [0038], “If the user selects a single color option, the search results may be updated to show only images that match the keyword and the single color. The color is matched against the weighted colors retrieved from each image and secondarily on the volume of the color in each image. In particular, the disclosed method may perform the following operations if a single color is selected via a single color selection option: (1) retrieving the set of pertinent images from the first database satisfying the keyword criterion; (2) comparing the single color to the combination of the color weight and the color volume score in each image that contains the selected color; and (3) displaying the result set comprising the images that match the keyword criterion and the single color on the user interface.” and [0055], “Notably, in one embodiment of the disclosed invention, any number of colors may be selected while configuring the color palette selection criteria. The add color button 540 may be displayed until the user has selected the predetermined number of colors to specify the color palette selection criteria colors.” and [0032], “The first set of operations may comprise: (1) retrieving the top four colors from the first image and the retrieving the color volume scores for the top four colors in the first image, (2) searching the second database to locate the set of pertinent images satisfying the keyword and a threshold color volume score criteria, and (3) generating for each color, a ranked display of the set of pertinent images based on the color volume scores of each image in the set of pertinent images.” and [0075], “At event 1930, the disclosed method may use the color volume scores generated in the event 1920 to weigh the colors in the query proportionally to the selected image. Accordingly, the 4 colors may be entered into the search query along with the color volume scores as boost parameters. The method may return a ranked result set containing images that match with all of the colors in the selected palette and the search term, wherein the result set is ranked based on how close in volume each color in each image matches the volume of each color in the selected image.” and [0070], “The steps of common color identification and index generation is illustrated in an equation format in FIG. 14, where common colors are indicated by the reference numeral 1410 and the generated index is indicated by the reference numeral 1420. The disclosed method may then iteratively process each color in the generated index i.e. red, light red, dark red, darker red and yellow colors. In particular, the method may, using the Euclidean distance of the RGB color values, determine the color distance between a given color and the colors in the index other than the given color. The method may delete a specific palette from the index if the distance between two of the colors is less than a predetermined threshold value.”; Examiner’s note: Although the example in Becker, [0038], illustrates scoring a single color against selected colors from other images, [0055] discloses that “any number of colors may be selected while configuring the color palette selection criteria” by a user.)
19. As per claim 6, Becker in view of Zhang, and further in view of Dorner discloses: The method of claim 1, wherein k1 is at least 12. (Becker, [0032], “The first set of operations may comprise: (1) retrieving the top four colors from the first image and the retrieving the color volume scores for the top four colors in the first image … Note that the “top” colors may be any desired number of colors without departing from the invention.” and [0055], “Notably, in one embodiment of the disclosed invention, any number of colors may be selected while configuring the color palette selection criteria.”)
20. As per claim 7, Becker in view of Zhang, and further in view of Dorner discloses: The method of claim 1, wherein ksub is 5 or less. (Becker, [0068], “In one embodiment of the disclosed invention if each palette in the search result set has N colors, the method may create an index of all combinations of N−1 colors.” and [0069], “Notably, each of the palettes 1-5 shown in table 1300 has 3 colors, so the method may determine the value of N to be three. Accordingly, the method according to the preferred embodiment of the disclosed invention may create an index of palettes that have (N−1) common colors, since the value of N is three in the example shown the FIG. 13, the disclosed method may (1) identify the common colors contained in the palettes 1-5 and (2) create an index of palettes that have two common colors.”)
21. As per claim 10, Becker in view of Zhang, and further in view of Dorner discloses: A data processing device comprising means for carrying out the steps of the method of claim 1. (Becker, [0031], “Aspects of the present invention provide a color palette suggestion system for conducting a keyword, and a color palette based image search in response to receiving, on a user interface of a computing device, a user presented keyword and a color palette selection criteria the system comprising: at least one processor and at least one non-volatile computer-readable storage medium encoded with executable instructions that, when executed by the at least one processor, causes the at least one processor to provide a display of searched images, the system comprising: ... (3) a color palette generation unit configured to generate a set of suggested color palettes by iteratively extracting a suggested color palette from each image in the set of pertinent images …”)
22. As per claim 11, Becker in view of Zhang, and further in view of Dorner discloses: A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method of claim 1. (Becker, [0031], “Aspects of the present invention provide a color palette suggestion system for conducting a keyword, and a color palette based image search in response to receiving, on a user interface of a computing device, a user presented keyword and a color palette selection criteria the system comprising: at least one processor and at least one non-volatile computer-readable storage medium encoded with executable instructions that, when executed by the at least one processor, causes the at least one processor to provide a display of searched images, the system comprising: ... (3) a color palette generation unit configured to generate a set of suggested color palettes by iteratively extracting a suggested color palette from each image in the set of pertinent images …”)
23. Claims 8-9 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Becker et al. (US-2015/0324394-A1, hereinafter "Becker") in view of Zhang et al. (NPL: “Palette-Based Image Recoloring Using Color Decomposition Optimization,” IEEE Transactions on Image Processing, vol. 26, no. 4, pp. 1952–1964, Apr. 2017.), further in view of Dorner et al. (US-2015/0379743-A1, hereinafter “Dorner”), and further in view of Moonen et al. (WO-2009/027451-A1, hereinafter “Moonen”).
24. As per claim 8, Becker in view of Zhang, and further in view of Dorner discloses: A method of [[selecting paint colors]] comprising the method of claim 1 and [[further comprising selecting a paint color corresponding to each of the colors of the color subpalette.]] (See rejection for claim 1.)
25. Becker in view of Zhang, and further in view of Dorner doesn't explicitly disclose but Moonen discloses: [[A method of]] selecting paint colors [[comprising the method of claim 1]] and further comprising selecting a paint color corresponding to each of the colors of the color subpalette. (Moonen, page 12, lines 8-14, “Having arrived at a satisfactory colour scheme, the user has created a set of colour data that identifies elements of a design task and associates colours with at least some of those elements. Through the user interface that user may then initiate an instruction to the paint production means to generate the first volume of a paint corresponding to one or more colours of said colour data set or optionally corresponding to one or more commercial products matching the colour data set.”)
26. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Becker in view of Zhang, and further in view of Dorner to include the disclosure of selecting a paint color corresponding to each of the colors of the color subpalette, of Moonen. The motivation for this modification could have been to have a system where a representative color palette can help select paint colors. By selecting paint colors, a user could match a particular palette color to that of an image or moodboard. This then could allow the user to paint a room that matches the color theme of an image or moodboard.
27. As per claim 9, Becker in view of Zhang, further in view of Dorner, and further in view of Moonen discloses: A method according to claim 8, wherein each color of the color subpalette is replaced with the most similar paint color available from a commercially available range of paint colors. (Moonen, page 12, lines 8-14, “Having arrived at a satisfactory colour scheme, the user has created a set of colour data that identifies elements of a design task and associates colours with at least some of those elements. Through the user interface that user may then initiate an instruction to the paint production means to generate the first volume of a paint corresponding to one or more colours of said colour data set or optionally corresponding to one or more commercial products matching the colour data set.”)
The motivation for this modification is the same as claim 8.
28. As per claim 12, Becker in view of Zhang, further in view of Dorner, and further in view of Moonen discloses: A computer program product according to claim 11, which will cause the computer to output an identifier for each color of the color subpalette, which identifier is for a color of paint. (Moonen, Claim 5, “A system according to any one of claims 1 to 4, wherein said information related to each colour of said primary palette includes an identification of those commercial products available in said colour.” and page 12, lines 8-10, “Having arrived at a satisfactory colour scheme, the user has created a set of colour data that identifies elements of a design task and associates colours with at least some of those elements.”)
29. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 11 of Becker in view of Zhang, and further in view of Dorner to include the disclosure of outputting an identifier for each color of the color subpalette, which identifier is for a color of paint, of Moonen. The motivation for this modification could have been to assist a user in quickly identifying a paint color that matches colors in the subpalette. By having a specific identification, a user could then easily talk to paint suppliers about acquiring that specific paint color.
30. Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Becker et al. (US-2015/0324394-A1, hereinafter "Becker") in view of Zhang et al. (NPL: “Palette-Based Image Recoloring Using Color Decomposition Optimization,” IEEE Transactions on Image Processing, vol. 26, no. 4, pp. 1952–1964, Apr. 2017.), further in view of Dorner et al. (US-2015/0379743-A1, hereinafter “Dorner”), further in view of Moonen et al. (WO-2009/027451-A1, hereinafter “Moonen”), and further in view of Caruso et al. (US-2012/0170840-A1, hereinafter “Caruso”).
31. As per claim 13, Becker in view of Zhang, further in view of Dorner, and further in view of Moonen discloses: A computer program according to claim 11, which computer program will cause the computer to [[output a link to an ordering platform, from which paint of at least one color of the color subpalette may be ordered.]] (See rejection for claim 11.)
32. Becker in view of Zhang, further in view of Dorner, and further in view of Moonen doesn't explicitly disclose but Caruso discloses: [[A computer program according to claim 11, which computer program will cause the computer to]] output a link to an ordering platform, from which paint of at least one color of the color subpalette may be ordered. (Caruso, [0043], “The method can further include entering the selected color on a purchase order; displaying a link to a color formula retrieval tool; and/or displaying a link to a distributor or supplier where paint and/or ancillary products may be purchased.” and [0041], “Utilizing several search criteria, the user may narrow the identification and selection to a small group of digital color images. Selecting a digital color image can produce additional information about that color as well as compatible color information. Digital color images can be arranged chromatically for easier selection. Final selection can link the selected color image to a color formula retrieval tool. Alternatively, final selection can link to the nearest distributor/supplier to purchase the paint and ancillary products. Final selection may also link to additional technical information.”)
33. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 11 of Becker in view of Zhang, further in view of Dorner, and further in view of Moonen to include the disclosure of outputting a link to an ordering platform, from which paint of at least one color of the color subpalette may be ordered, of Caruso. The motivation for this modification could have been to provide an easy way for a user to quickly access and order a particular color of paint. An order link would provide a quick and convenient way for a user to find the desired paint product.
Conclusion
34. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW CLOTHIER/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights