DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the "right to exclude" granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Langi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 18978642 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because they can read on to each other, see the following mapping table.
Current Application
21
22
23
24
25
26
27
28
29
Patent application
1
2
3
4
5
6
7
8
9
Current Application
30
31
32
33
34
35
36
Patent application
1
2
3
4
5
6
7
Current Application
37
38
39
40
Patent application
17
18
19
20
Also, shown below is a mapping between the limitations of independent claims of current application U.S. Patent Application 12182913 B2 and independent claims of U.S. Patent Application 18978642 B2.
Claims
Current Application
Claims
Patent Application
21
A computer-implemented method comprising: receiving, from a client device, a user query corresponding to a digital image to be edited;
1
A computer-implemented method comprising: generating an editing preset map between a plurality of edited digital images and a plurality of editing states that correspond to the plurality of edited digital images, the plurality of editing states including one or more editing operations and one or more editing values corresponding to the one or more editing operations that indicate modifications made to a plurality of initial digital images to generate the plurality of edited digital images; receiving, from a client device, a user query corresponding to a first digital image to be edited;
extracting, from the user query, an editing intent for editing the digital image; determining one or more editing presets that corresponds to the editing intent, wherein each editing preset of the one or more editing presets comprises one or more editing operations and one or more corresponding editing values; providing a visual element for each editing preset of the one or more editing presets in a graphical user interface; receiving a selection of a visual element of an editing preset via the graphical user interface; and in response to receiving the selection of the visual element, generating a modified digital image by applying the one or more editing operations utilizing the one or more corresponding editing values of the editing preset to modify pixels of the digital image in accordance with the editing preset.
extracting, from the user query, an editing intent for editing the first digital image based on at least one edited digital image from the plurality of edited digital images; determining, using the editing preset map, an editing preset that corresponds to the editing intent based on an editing state of an edited digital image associated with the editing preset, wherein the edited digital image comprises a second digital image that differs from the first digital image to be edited, and wherein the editing state includes at least one editing operation and at least one editing value corresponding to the at least one editing operation that indicate modifications made to an initial digital image to generate the edited digital image; and
generating, for provision to the client device, a recommendation for the editing preset that comprises pre-saved editing settings associated with the at least one editing operation and the at least one editing value.
30
A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
10
A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
receiving, from a client device, a user query corresponding to a digital image to be edited; extracting, from the user query, an editing intent for editing the digital image; determining one or more editing presets that corresponds to the editing intent, wherein each editing preset of the one or more editing presets comprises one or more editing operations and one or more corresponding editing values; providing a visual element for each editing preset of the one or more editing presets in a graphical user interface; receiving a selection of a visual element of an editing preset via the graphical user interface; and in response to receiving the selection of the visual element, applying the one or more editing operations utilizing the one or more corresponding editing values of the editing preset to modify pixels of the digital image in accordance with the editing preset.
generating an editing preset map between a plurality of edited digital images and a plurality of editing states that correspond to the plurality of edited digital images, the plurality of editing states including one or more editing operations and one or more editing values corresponding to the one or more editing operations that indicate modifications made to a plurality of initial digital images to generate the plurality of edited digital images; receiving, from a client device, a user query corresponding to a first digital image to be edited; extracting, from the user query, an editing intent for editing the first digital image based on at least one edited digital image from the plurality of edited digital images; determining, using the editing preset map, an editing preset that corresponds to the editing intent based on an editing state of an edited digital image associated with the editing preset, wherein the edited digital image comprises a second digital image that differs from the first digital image to be edited, and wherein the editing state includes at least one editing operation and at least one editing value corresponding to the at least one editing operation that indicate modifications made to an initial digital image to generate the edited digital image; and
generating, for provision to the client device, a recommendation for the editing preset that comprises pre-saved editing settings associated with the at least one editing operation and the at least one editing value.
37
A system comprising: one or more memory devices; and one or more processors coupled to the one or more memory devices that cause the system to perform operations comprising:
17
A system comprising: one or more memory devices; and one or more processors coupled to the one or more memory devices that cause the system to perform operations comprising:
generating an editing preset map that associates a plurality of editing presets with a plurality of editing intents;extracting, from a user query, an editing intent for editing a digital image;determining an editing preset associated the editing intent for editing the digital image utilizing the editing preset map;providing a visual element for the editing preset in a graphical user interface;receiving a selection of the visual element via the graphical user interface; andin response to receiving the selection of the visual element, generating a modified digital image by applying one or more editing operations utilizing one or more corresponding editing values to modify pixels of the digital image, wherein the one or more editing operations and the one or more corresponding editing values are associated with the editing preset.
generating an editing preset map between a plurality of edited digital images and a plurality of editing states that correspond to the plurality of edited digital images, the plurality of editing states including one or more editing operations and one or more editing values corresponding to the one or more editing operations that indicate modifications made to a plurality of initial digital images to generate the plurality of edited digital images; extracting, from a user query, an editing intent for editing a first digital image that differs from the plurality of edited digital images based on an edited digital image from the plurality of edited digital images; determining, in response to the user query, at least one edited digital image from the plurality of edited digital images that corresponds to the editing intent using the editing preset map; and
generating a recommendation for an editing preset that comprises pre-saved editing settings associated with the at least one edited digital image.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 21, 22, 27 – 40 are rejected under 35 U.S.C. 103 as being unpatentable over Kadia et al. (Publication: US 2021/0103632 A1) in view of Ding et al. (Publication: US 2021/0027497 A1).
Regarding claim 21, see the rejection on claim 30.
Regarding claim 22, Kadia in view of Ding disclose all the limitation of claim 21.
Kadia discloses receiving a desired edit to apply to the digital image in a search bar of the graphical user interface ([0029] - A drop-down menu 202 is provided with a list of fonts. A user can scroll through the list of fonts in the drop-down menu 202 and can select a font to see what the font will look like on the electronic document 201. In addition, the user can select a filter option 204 on the user interface 200 to cause a filter window 206 to be displayed. The available fonts of the design application can be filtered based on different classifications or types, including “decorative,” “serif,” “sans serif,” “slab serif,” “script,” “cursive,” “handwritten,” among others. As shown in FIG. 2, the “slab serif” (described as “thick bold-like serifs”) is selected, which cases all available fonts to be filtered to only include the fonts that have been classified as “slab serif” fonts. ing, “a search bar of the graphical user interface”.
[0070] The “oil paint” image tag is also mapped to a “playful” font tag based on a beginner user selecting a font having the “playful” font tag assigned to it for an electronic document that has been classified with the “oil paint” class. The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class, “digital image”.) .
Regarding claim 27, Kadia in view of Ding disclose all the limitation of claim 1.
Kadia discloses wherein extracting the editing intent from the user query comprises generating an editing intent vector from the user query utilizing a multi-class classification neural network, the editing vector indicating editing operations and corresponding editing values to apply in editing the digital image ([0039] - The machine learning model can classify one or more objects in the image. In one example, given an image of the electronic document 101 shown in FIG. 1, the machine learning model can classify the bats, the castle, the clouds, and the sky present in the image. [0041] - the machine learning model (e.g., a neural network, such as a CNN) can include a perspective classification model that classifies objects into classes belonging to different categories or perspectives. In one illustrative example, the image 303 of the electronic document 301 can be classified along a 4-orthogonal axis, with each axis corresponding to a category or perspective. The categories or perspectives corresponding to the 4 axes and classes within each of the categories or perspectives can include the following (denoted as “category={class 1, . . . class m}”, where m is a positive integer value): [0042] user group={older, middle aged, child} [0043] genre={comic, funky, retro, sci-fi} [0044] media={oil-painting, pencil-sketched, water color} [0045] mood={happy, horror, peaceful}
[0066] - The mappings in the user map 311 allow the font recommendation system 300 to adaptively learn as the system improves its recommendations based on the calligraphic choices made by various users, user query. using one or more neural network models to determine one or more image tags and determining one or more font tags based on one or more tag scores computed between the one or more font tags and the one or more image tags from the electronic document using a plurality of image tag to font tag maps.
[0065] - See fig. 5 and Fig. 6, map. Each mapping in a designer map can be assigned a path cost of 1 (other path costs can also be assigned). The path cost is used to weight the mapping
[0123], Fig. 13 - Once a destination tag is reached for the source tag, the final path score is taken as the tag score of the destination tag (which is the associated font tag for the source tag). The associated font tag and the tag score for the associated font tag are added to the final result map (M.sub.result). As noted above, M.sub.result includes the font tags 307 and the associated font tags determined using the associated tag determination process. An updated font tag data structure (e.g., an n-tuple with n being the number of font tags accessible by the software application, a vector representation, or the like) can be generated for the electronic document 301 based on M.sub.result. The updated font tag data structure includes tag score values for all available font tags, and the tag scores are updated based on M.sub.result.).
Regarding claim 28, Kadia in view of Ding, Pena disclose all the limitation of claim 21.
Kadia discloses generating a vector representation of the editing state of the edited digital image ([0123], Fig. 13 - Once a destination tag is reached for the source tag, the final path score is taken as the tag score of the destination tag (which is the associated font tag for the source tag). The associated font tag and the tag score for the associated font tag are added to the final result map (M.sub.result). As noted above, M.sub.result includes the font tags 307 and the associated font tags determined using the associated tag determination process. An updated font tag data structure (e.g., an n-tuple with n being the number of font tags accessible by the software application, a vector representation, or the like) can be generated for the electronic document 301 based on M.sub.result. The updated font tag data structure includes tag score values for all available font tags, and the tag scores are updated based on M.sub.result.);
determining that the editing preset associated with the edited digital image corresponds to the editing intent by determining that the editing intent vector is a subset of the vector representation of the editing state or a match with the vector representation of the editing state ([0129], Fig. 13 - The font search engine 310 can determine the Euclidean distance (or other distance or similarity measure) from each of the fonts F1-F7 to the font space tuple 1330 (or n-tuple) representing an electronic document (e.g., the electronic document 301). For example, as shown in FIG. 13, the font space tuple 1330 (or n-tuple) representing the electronic document is set to (0.9, 0.67, 0.75), including a tag score value of 0.9 for the first font tag, a tag score value of 0.67 for the second font tag, and a tag score value of 0.75 for the third font tag. In some examples, as noted above, a tag score value of an entry in the font space tuple can be 0. The Euclidean distance (or other distance or similarity measure) can be measured from the fonts F1-F7 to the font space tuple 1330 by comparing the values in the font space tuple 1330 to the values of the font space tuples for the various fonts F1-F7. The font search engine 310 can output the fonts F1-F7 in a decreasing order of their distance from the font space tuple 1330, and can select the fonts that are a closest match to the font space tuple 1330 representing the electronic document as recommended fonts 317. For example, the font space tuple (1,1,1) of the font F7 is a closest match to the font space tuple 1330, the font space tuple (1,1,0) of the font F3 is a second closest match to the font space tuple 1330, and the font space tuple (0,1,1) of the font F6 is a third closest match to the font space tuple 1330. The font search engine 310 can output the fonts F3, F6, and F7 as recommended fonts 317.
[0123], Fig. 13 - Once a destination tag is reached for the source tag, the final path score is taken as the tag score of the destination tag (which is the associated font tag for the source tag). The associated font tag and the tag score for the associated font tag are added to the final result map (M.sub.result). As noted above, M.sub.result includes the font tags 307 and the associated font tags determined using the associated tag determination process. An updated font tag data structure (e.g., an n-tuple with n being the number of font tags accessible by the software application, a vector representation, or the like) can be generated for the electronic document 301 based on M.sub.result. The updated font tag data structure includes tag score values for all available font tags, and the tag scores are updated based on M.sub.result.).
Regarding claim 29, Kadia in view of Ding, Pena disclose all the limitation of claim 21.
Kadia discloses determining a plurality of editing presets that correspond to the editing intent ([0029] - A drop-down menu 202 is provided with a list of fonts. A user can scroll through the list of fonts in the drop-down menu 202 and can select a font to see what the font will look like on the electronic document 201. In addition, the user can select a filter option 204 on the user interface 200 to cause a filter window 206 to be displayed. The available fonts of the design application can be filtered based on different classifications or types, including “decorative,” “serif,” “sans serif,” “slab serif,” “script,” “cursive,” “handwritten,” among others. As shown in FIG. 2, the “slab serif” (described as “thick bold-like serifs”) is selected, which cases all available fonts to be filtered to only include the fonts that have been classified as “slab serif” fonts. ing.); and
selecting the editing preset from the plurality of editing presets based on determining that an initial tone of the edited digital image associated with the editing preset corresponds to a current tone of the digital image to be edited ([0070], Fig. 6 – User Map is generated.
The “oil paint” image tag is also mapped to a “playful” font tag based on a beginner user selecting a font having the “playful” font tag assigned to it for an electronic document that has been classified with the “oil paint” class. The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class.
[0029] – a list of fonts are shown.) .
Ding discloses
an initial tone of the edited digital image ([0157] - the color matching manager 1018 builds one or more color similarity regions for each color in a set of colors (e.g., colors in a color deck). In this manner, when classifying the color of an object, the color matching manager 1018 can compare pixel mappings from the object to the previously generated color similarity regions 1030 to determine a color match.);
current tone of the digital image ([0157] - when classifying the color of an object, the color matching manager 1018 can compare pixel mappings from the object to the previously generated color similarity regions 1030 to determine a color match.);
determining … after removing ([0035] - the color classification system can further filter out a detected object having a color-matching score that is notably lower than color-matching scores of other detected objects.
[0050] The term “color-matching threshold” refers to a condition or value where a pixel is considered a match to a color (e.g., a target color). For instance, the color-matching threshold indicates when the color value for a pixel is similar enough to a color in multidimensional color space to match the color. The minimum color-matching threshold refers to the lowest color value a pixel can have with a color to be considered a match.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Kadia in view of Ding, Pena with an initial tone of the edited digital image; current tone of the digital image as taught by Ding. The motivation for doing is to improve efficiency as taught by Ding.
Regarding claim 30, Kadia discloses A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising ([0034], Fig. 3 - the font recommendation system 300 may also include, or can be part of a computing device that includes, one or more memory devices (e.g., one or more random access memory (RAM) components, read-only memory (ROM) components, cache memory components, buffer components, database components, and/or other memory devices), one or more processing devices (e.g., one or more CPUs, GPUs, and/or other processing devices) in communication with and/or electrically connected to the one or more memory devices with instructions, processed by the CPU to perform:):
a user query corresponding to a digital image to be edited ([0066] - The mappings in the user map 311 allow the font recommendation system 300 to adaptively learn as the system improves its recommendations based on the calligraphic choices made by users, “user query”. using one or more neural network models to determine one or more image tags and determining one or more font tags based on one or more tag scores computed between the one or more font tags and the one or more image tags from the electronic document using a plurality of image tag to font tag maps.
[0070], Fig. 6 - The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class. The “comic” image tag is mapped the “playful” font tag in response to an intermediate user selecting a font having the “playful” font tag assigned to it for an electronic document classified with a “comic” class.
[0066] - each time one or more users make a typographic decision (e.g., chooses a particular font for a document and saves the document), the global user map can be updated based on the choice to update the document, image “to be edited”.);
extracting, from the user query, an editing intent for editing the digital image ([0039] – bats, castle, cloud, and sky images are classified by dividing into segments [0060] for classification with different category [0062] .
The “peaceful” image tag is mapped to a “smooth” font tag, indicating that a peaceful image or a peaceful object in the image is associated (by the designer) with a font that has a smooth characteristic. The “comic” image tag is mapped to a “funky” font tag. The “watercolor” image tag is mapped to the “smooth” font tag and to the “funky” font tag.
[0070], Fig. 6 – User Map is generated.
The “oil paint” image tag is also mapped to a “playful” font tag based on a beginner user selecting a font having the “playful” font tag assigned to it for an electronic document that has been classified with the “oil paint” class. The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class.
[0029] – a list of fonts are shown.);
determining one or more editing presets that corresponds to the editing intent, wherein each editing preset of the one or more editing presets comprises one or more editing operations and one or more corresponding editing values ([0070], Fig. 6 - The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class, “editing intent”. The “comic” image tag is mapped the “playful” font tag in response to an intermediate user selecting a font having the “playful” font tag assigned to it for an electronic document classified with a “comic” class.
[0083] - The path cost is used to weight the mapping, “values”.);
providing a visual element for each editing preset of the one or more editing presets in a graphical user interface ([0029] - A drop-down menu 202 is provided with a list of fonts. A user can scroll through the list of fonts in the drop-down menu 202 and can select a font to see what the font will look like on the electronic document 201. In addition, the user can select a filter option 204 on the user interface 200 to cause a filter window 206 to be displayed. The available fonts of the design application can be filtered based on different classifications or types, including “decorative,” “serif,” “sans serif,” “slab serif,” “script,” “cursive,” “handwritten,” among others. As shown in FIG. 2, the “slab serif” (described as “thick bold-like serifs”) is selected, which cases all available fonts to be filtered to only include the fonts that have been classified as “slab serif” fonts. ing.) ;
receiving a selection of a visual element of an editing preset via the graphical user interface (
[0029], [0070] The “oil paint” image tag is also mapped to a “playful” font tag based on a beginner user selecting a font from the graphical user interface having the “playful” font tag assigned to it for an electronic document that has been classified with the “oil paint” class. The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class.); and
in response to receiving the selection of the visual element, applying the one or more editing operations utilizing the one or more corresponding editing values of the editing preset to [[modify pixels of the digital image]] in accordance with the editing preset (
[0029] - A drop-down menu 202 is provided with a list of fonts. A user can scroll through the list of fonts in the drop-down menu 202 and can select a font to see what the font will look like on the electronic document 201. In addition, the user can select a filter option 204 on the user interface 200 to cause a filter window 206 to be displayed, “visual element, editing preset”.
[0029], [0070] The “oil paint” image tag is also mapped to a “playful” font tag based on a beginner user selecting a font from the graphical user interface having the “playful” font tag assigned to it for an electronic document that has been classified with the “oil paint” class. The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto, “in response to receiving the selection of the visual element”.
[0065], [0066] - The mappings in the user map 311 allow the font recommendation system 300 to adaptively learn as the system improves its recommendations based on the calligraphic choices made by various users, user query. using one or more neural network models to determine one or more image tags and determining one or more font tags based on one or more tag scores computed between the one or more font tags and the one or more image tags from the electronic document using a plurality of image tag to font tag maps. See fig. 5 and Fig. 6, map. Each mapping in a designer map can be assigned a path cost of 1 (other path costs can also be assigned). The path cost is used to weight the mapping. each time one or more users make a typographic decision (e.g., chooses a particular font for a document and saves the document), the map can be updated based on the choice to update the document, image “to modify edited. accordance with the editing preset. editing values”).
Kadia does not disclose; however Ding discloses
receiving, from a client device ([0062] - the color classifier server system 112 can include a web hosting application that allows the client device 102 to interact with content and services hosted on the server device 110. To illustrate, in one or more embodiments, the client device 102 accesses a web page supported by the server device 110 that automatically detects object colors in images based on user input from the client device 102. As another example, the client device 102 provides an image editing application that provides the image and a selection query to the color classifier server system 112 on the server device 110 that includes a query object and a query color, which then detects the query object in the query color and provides an object mask of the detected query object back to the client device 102. Then, utilizing the object mask, the image editing application on the client device 102 selects the detected query object.);
applying operations to modify pixels of the digital image ([0143] As shown in FIG. 9C, the color classification system 106 can generate or otherwise obtain an object mask 910 for each of the detected query object instances. For example, the color classification system 106 provides the detected object to an object mask neural network, which generates an object mask (e.g., selection mask) for the object. In particular, the color classification system 106 provides bounding boxes of the one or more query object instances to the object mask neural network. In some embodiments, the color classification system 106 can downsample pixels in the bounding boxes, modify.
[0045] - In addition to hue, color can include attributes such as brightness, contrast, saturation, tone, shade, tinge, tint, pigment, chroma, luminosity, chromaticity, undertone, iridescence, intensity, polychromasia, and/or colorimetric quality. each pixel in an image is shown as one color. In addition, the color of a pixel is represented by a color value associated with a color model. Color models are associated with different multidimensional color spaces (e.g., 3 or more dimensions) that correspond to the values and attributes defined by the corresponding color model.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Kadia in view of Ding with receiving, from the client device; applying operations to modify pixels of the digital image as taught by Ding. The motivation for doing is to improve efficiency as taught by Ding.
Regarding claim 31, see the rejection on claim 22.
Regarding claim 32, see the rejection on claim 23.
Regarding claim 33, see the rejection on claim 27.
Regarding claim 34, Kadia in view of Ding disclose all the limitation of claim 30.
Kadia discloses wherein applying the one or more editing operations utilizing the one or more corresponding editing values of the editing preset to modify the pixels of the digital image in accordance with the editing preset comprises applying one or more of operations ([0005] - the information obtained by the font recommendation system can include one or images of an electronic document and/or text from the electronic document. For instance, to generate one or more recommended fonts for an electronic document, the font recommendation system and techniques can obtain one or images of the electronic document and/or text from the electronic document as input.
[0028] - To select an appropriate font for the text portions of the electronic document 101, a user must scroll through the list of fonts in the drop-down menu 102 and select a font to see what the font will look like on the electronic document 101.).
Ding discloses modify the pixel … applying one or more of an exposure operation, a contrast operation, a highlight operation, a sharpening operation, or a dehaze operation to the digital image ([0143] As shown in FIG. 9C, the color classification system 106 can generate or otherwise obtain an object mask 910 for each of the detected query object instances. For example, the color classification system 106 provides the detected object to an object mask neural network, which generates an object mask (e.g., selection mask) for the object. In particular, the color classification system 106 provides bounding boxes of the one or more query object instances to the object mask neural network. In some embodiments, the color classification system 106 can downsample pixels in the bounding boxes, modify.
[0045] - In addition to hue, color can include attributes such as brightness, contrast, saturation, tone, shade, tinge, tint, pigment, chroma, luminosity, chromaticity, undertone, iridescence, intensity, polychromasia, and/or colorimetric quality. each pixel in an image is shown as one color. In addition, the color of a pixel is represented by a color value associated with a color model. Color models are associated with different multidimensional color spaces (e.g., 3 or more dimensions) that correspond to the values and attributes defined by the corresponding color model.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Kadia in view of Ding with wherein applying the one or more editing operations utilizing the one or more corresponding editing values of the editing preset to modify the pixels of the digital image in accordance with the editing preset comprises applying one or more of operations as taught by Ding. The motivation for doing is to improve efficiency as taught by Ding. .
Regarding claim 35, see the rejection on claim 24.
Regarding claim 36, see the rejection on claim 25.
Regarding claim 37, see the rejection on claim 30.
Regarding claim 38, Kadia in view of Ding disclose all the limitation of claim 37.
Kadia discloses receiving the user query via a help search bar of the graphical user interface ([0029] - A drop-down menu 202 is provided with a list of fonts. A user can scroll through the list of fonts in the drop-down menu 202 and can select a font to see what the font will look like on the electronic document 201. In addition, the user can select a filter option 204 on the user interface 200 to cause a filter window 206 to be displayed. The available fonts of the design application can be filtered based on different classifications or types, including “decorative,” “serif,” “sans serif,” “slab serif,” “script,” “cursive,” “handwritten,” among others. As shown in FIG. 2, the “slab serif” (described as “thick bold-like serifs”) is selected, which cases all available fonts to be filtered to only include the fonts that have been classified as “slab serif” fonts. ing, “a search bar of the graphical user interface”.
[0070] The “oil paint” image tag is also mapped to a “playful” font tag based on a beginner user selecting a font having the “playful” font tag assigned to it for an electronic document that has been classified with the “oil paint” class. The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class.).
Regarding claim 39, see the rejection on claim 23.
Regarding claim 40, see the rejection on claim 27.
Claims 23 – 26 are rejected under 35 U.S.C. 103 as being unpatentable over Kadia et al. (Publication: US 2021/0103632 A1) in view of Ding et al. (Publication: US 2021/0027497 A1) and Durante et al. (Publication: US 2019/0130623 A1).
Regarding claim 23, Kadia in view of Ding disclose all the limitation of claim 1.
Kadia in view of Ding do not disclose; however, Durante discloses
generating a preview thumbnail for each editing preset, a given preview thumbnail showing a preview of how the digital image would appear if an associated editing preset were selected ([0031] FIG. 3B is a simplified diagram of a user interface for previewing a background image. As shown in FIG. 3B, the user interface is suitable for use on a small form factor display, such as those commonly found on a smart phone, tablet, and/or the like. As shown, the user interface is configured for use with the display in a portrait orientation, however, one of skill in the art would understand that other configurations, landscape orientation, and/or the like are possible. The user interface of FIG. 3B also includes the occasion selection region 320 and menu region 330 from FIG. 3A. In some examples, when a thumbnail is selected from thumbnail region 310, a preview of the selected thumbnail is displayed in a preview region 340 that replaces thumbnail region 310. In some examples, preview region 340 may be scrolled left, right, up, or down using swipes, scroll gadgets (not shown), and/or the like in order to view a preview of other background images.); and
displaying one or more preview thumbnails for the one or more editing presets in the graphical user interface ([0031] FIG. 3B is a simplified diagram of a user interface for previewing a background image. As shown in FIG. 3B, the user interface is suitable for use on a small form factor display, such as those commonly found on a smart phone, tablet, and/or the like. As shown, the user interface is configured for use with the display in a portrait orientation, however, one of skill in the art would understand that other configurations, landscape orientation, and/or the like are possible. The user interface of FIG. 3B also includes the occasion selection region 320 and menu region 330 from FIG. 3A. In some examples, when a thumbnail is selected from thumbnail region 310, a preview of the selected thumbnail is displayed in a preview region 340 that replaces thumbnail region 310. In some examples, preview region 340 may be scrolled left, right, up, or down using swipes, scroll gadgets (not shown), and/or the like in order to view a preview of other background images.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Kadia in view of Ding with generating a preview thumbnail for each editing preset, a given preview thumbnail showing a preview of how the digital image would appear if an associated editing preset were selected; and displaying one or more preview thumbnails for the one or more editing presets in the graphical user interface as taught by Ding. The motivation for doing is to have more options available in the interface to improve warmth and sentimentality as taught by Durante.
Regarding claim 24, Kadia in view of Ding, Durante disclose all the limitation of claim 23.
Kadia discloses further comprising generating a global editing preset map that associates a plurality of edited digital images with map keys indicating global editing states ( [0039] – bats, castle, cloud, and sky images are classified by dividing into segments [0060] for classification with different category [0062] .
The “peaceful” image tag is mapped to a “smooth” font tag, indicating that a peaceful image or a peaceful object in the image is associated (by the designer) with a font that has a smooth characteristic. The “comic” image tag is mapped to a “funky” font tag. The “watercolor” image tag is mapped to the “smooth” font tag and to the “funky” font tag.),
wherein determining the editing preset that corresponds to the editing intent based on the editing state of the edited digital image associated with the editing preset comprises determining the editing preset that corresponds to the editing intent using the global editing preset map (
[0065], Fig. 5 – Designer map is generated.
[0039] – bats, castle, cloud, and sky images are classified by dividing into segments [0060] for classification with different category [0062] “intent”.
The “peaceful” image tag is mapped to a “smooth” font tag, indicating that a peaceful image or a peaceful object in the image is associated (by the designer) with a font that has a smooth characteristic. The “comic” image tag is mapped to a “funky” font tag. The “watercolor” image tag is mapped to the “smooth” font tag and to the “funky” font tag.
[0085] - An image-to-font tag mapping 854C of watercolor-to-smooth is obtained from the designer map 313. The watercolor-to-smooth mapping has a path cost of 2, due to the “watercolor” image tag being a contextual synonym. For example, as noted above, the cost factor is increased by a factor of 2 when a contextual synonym is obtained for an image tag, which is then multiplied by the path cost of the contextual synonym during the path cost determination process. The tag score 858C for the “smooth” font tag is determined to be 0.5 (the inverse of the path cost (½)). ).
Regarding claim 25, Kadia in view of Ding, Durante disclose all the limitation of claim 24.
Kadia discloses generating a map key for the local editing preset map, the map key representing editing operations and corresponding editing values applied to an object ([0070], Fig. 6 - The “3D graphics” image tag is mapped to a “fantasy” font tag based on an advanced user selecting a font having the “fantasy” font tag assigned thereto for an electronic document classified with a “3D graphics” class. The “comic” image tag is mapped the “playful” font tag in response to an intermediate user selecting a font having the “playful” font tag assigned to it for an electronic document classified with a “comic” class.
[0083] - The path cost is used to weight the mapping, “values”. );
determining a local editing state for at least one edited digital image, the local editing state indicating a set of editing operations and a corresponding set of editing values applied to at least one object portrayed in the at least one edited digital image([0086] - An image-to-font tag mapping 856C of oil-painting-to-playful is obtained from the user map 311 based on input from a beginner user.); and
associating the at least one edited digital image with the map key in response to determining that the local editing state corresponds to the map key ([0086] - The oil-painting-to-playful mapping 856C has a path cost of 6 based on the mapping being from a beginner user (path cost of 3) and based on “oil-painting” being a contextual synonym of “painting.” The tag score 860C for the “playful” font tag is determined to be 0.17 (the inverse of the path cost (⅙)). An additional image-to-font tag mapping of the “oil-painting” image tag is also determined, and includes an oil-painting-to-handwritten mapping 857C obtained from the user map 311 based on input from an intermediate user. The oil-painting-to-handwritten mapping 857C has a path cost of 4 based on the mapping being from an intermediate user (path cost of 2) and based on “oil-painting” being a contextual synonym of “painting.” The tag score 861C for the “handwritten” font tag is determined to be 0.25 (the inverse of the path cost (¼)).).
Regarding claim 26, Kadia in view of Ding, Durante disclose all the limitation of claim 25.
Kadia discloses determining a score for the at least one object portrayed in the at least one edited digital image( [0041] - the machine learning model (e.g., a neural network, such as a CNN) can include a perspective classification model that classifies objects into classes belonging to different categories or perspectives. In one illustrative example, the image 303 of the electronic document 301 can be classified along a 4-orthogonal axis, with each axis corresponding to a category or perspective. The categories or perspectives corresponding to the 4 axes and classes within each of the categories or perspectives can include the following (denoted as “category={class 1, . . . class m}”, where m is a positive integer value): [0042] user group={older, middle aged, child} [0043] genre={comic, funky, retro, sci-fi} [0044] media={oil-painting, pencil-sketched, water color} [0045] mood={happy, horror, peaceful}); and
associating an editing state of [[the mask]] with the at least one edited digital image as the local editing state based on the score for the at least one object ([0080] - An image-to-font tag mapping 856A of 3D graphic-to-fantasy is obtained from the user map 311 based on input from an advanced user. The graphic-to-fantasy mapping 856A has a path cost of 1, based on the mapping being from an advanced user. The tag score 860A for the “fantasy” font tag is determined as the inverse of the path cost (1/1), and thus is also determined to be 1.
[0077] - if the mapping CT1.fwdarw.FT1 exists in the user map 311 (beginner user: path cost 3), and the mapping CT24 FT1 also exists in the user map (intermediate user: path cost 2), the minimum path cost to FT1 is 2, and hence the tag score for the font tag FT1 is determined to be ½ or 0.5. If a font tag is not mapped to any image tag in the mapping procedure, the minimum path cost to the font tag is considered to be infinite. If the minimum path cost to a font tag is infinite, the tag score is 0.).
Ding discloses
determining a mask associated with the at least one edited digital image ([0143] As shown in FIG. 9C, the color classification system 106 can generate or otherwise obtain an object mask 910 for each of the detected query object instances. For example, the color classification system 106 provides the detected object to an object mask neural network, which generates an object mask (e.g., selection mask) for the object. In particular, the color classification system 106 provides bounding boxes of the one or more query object instances to the object mask neural network. In some embodiments, the color classification system 106 can downsample pixels in the bounding boxes);
determining using a bounding box for the at least one object and the mask; the mask ([0143] As shown in FIG. 9C, the color classification system 106 can generate or otherwise obtain an object mask 910 for each of the detected query object instances. For example, the color classification system 106 provides the detected object to an object mask neural network, which generates an object mask (e.g., selection mask) for the object. In particular, the color classification system 106 provides bounding boxes of the one or more query object instances to the object mask neural network. In some embodiments, the color classification system 106 can downsample pixels in the bounding boxes).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Kadia in view of Ding with determining a mask associated with the at least one edited digital image; determining using a bounding box for the at least one object and the mask; the mask as taught by Ding. The motivation for doing is to improve efficiency as taught by Ding.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MING WU whose telephone number is (571)270-0724. The examiner can normally be reached on Monday - Thursday and alternate Fridays: 9:30am - 6:00pm EST .
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached on 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MING WU/
Primary Examiner, Art Unit 2618
Prosecution Insights