Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, 15, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Phogat et al. (U.S. Patent Application Publication No. 2022/0262003), referred herein as Phogat, in view of Cohen et al. (U.S. Patent Application Publication No. 2015/0170005), referred herein as Cohen.
Regarding claim 1, Phogat teaches a computer-implemented method comprising: accessing a vector graphic comprising a set of shapes (paragraph 40, lines 1-3 and 8-12; paragraph 64, lines 1-3; a vector graphic comprising shapes is accessed);
in response to user input indicating a portion of the vector graphic, generating, utilizing a segmentation model, a mask of a region based on the portion indicated by the user input (paragraph 41, the last 9 lines; paragraph 43, lines 1-8; paragraph 46, lines 1-7; paragraph 68; in response to user input identifying a portion of the graphic, a mask region based on the portion is generated using a segmentation model);
selecting, utilizing a polyline model, a set of vector paths in the vector graphic corresponding to a location of the mask, and providing the set of vector paths as a selection of the region (paragraph 47, lines 1-14; paragraph 48, lines 1-4; paragraph 59, lines 9-19; paragraph 70; paragraph 106, lines 1-11; a polyline model is used to select a set of vector paths in the graphic corresponding to the mask, and the paths are provided as a selection of the region).
Phogat does not explicitly teach a semantic segmentation model, semantic mask, and semantic region.
However, in a similar field of endeavor, Cohen teaches a method comprising accessing an image comprising a set of shapes, and in response to user input indicating a portion of the graphic, generating, utilizing a semantic segmentation model, a semantic mask of a semantic region based on the user input (fig 2; paragraph 27, lines 1-9; paragraphs 33 and 34; paragraphs 64 and 65).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the semantic mask segmentation of Cohen with the mask segmentation of Phogat because this provides an easier and more intuitive way for users to interact with the image segmentation interface, and reduce the amount of preprocessed training, and can segment object classes that have large appearance variations, thereby providing both ease of use and accuracy of segmentation (see, for example, Cohen, paragraph 156, the last 21 lines).
Regarding claim 7, Phogat in view of Cohen teaches the computer-implemented method of claim 1, wherein generating the semantic mask comprises: generating a rasterized version of the vector graphic; and generating, utilizing the semantic segmentation model, the semantic mask from the user input indicating the portion of the vector graphic and the rasterized version of the vector graphic (Phogat, paragraph 41, the last 9 lines; paragraph 43, lines 1-8; paragraph 44, lines 1-6; paragraph 46, lines 1-7; paragraph 68; Cohen, paragraph 27, lines 1-9; paragraphs 33 and 34; paragraphs 64 and 65; the motivation to combine is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 15, Phogat teaches a non-transitory computer-readable medium storing executable instructions which, when executed by at least one processing device, cause the at least one processing device to perform operations comprising (fig 11; paragraphs 159 and 160; paragraph 163):
in response to user input indicating a point of a vector graphic comprising one or more groups of vector objects, generating, utilizing a segmentation model, a mask that defines a region in the vector graphic (paragraph 40, lines 1-3 and 8-12; paragraph 41, the last 9 lines; paragraph 43, lines 1-8; paragraph 46, lines 1-7; paragraph 68; in response to user input identifying a point of a vector graphic comprising a group of vector graphics, a mask defining a region in the graphic is generated using a segmentation model);
generating, utilizing a polyline model, a set of mask polylines from the mask and a set of visible vector path polylines from visible vector paths of the vector graphic (paragraph 47; paragraph 48, lines 1-4; paragraph 59, lines 9-19; paragraph 60; paragraph 70; paragraph 106; a polyline model is used to generate mask polylines from the mask and a set of visible vector path polylines);
based on the set of mask polylines and the set of visible vector path polylines, selecting visible vector paths associated with the set of visible vector path polylines that satisfy a visible vector path overlap threshold, and providing the visible vector paths as a selection of the region in the vector graphic (paragraph 47; paragraph 48, lines 1-4; paragraph 59, lines 9-19; paragraph 60; paragraph 70; paragraph 106; paragraph 151, the last 8 lines; based on the mask polylines and visible vector path polylines, visible vector paths are selected that are associated with the polylines and have an amount of path overlap, and are provided as a selection of the region).
Phogat does not explicitly teach a semantic segmentation model, semantic mask, and semantic region.
However, in a similar field of endeavor, Cohen teaches a medium performing operations comprising accessing an image comprising a set of shapes, and in response to user input indicating a portion of the graphic, generating, utilizing a semantic segmentation model, a semantic mask of a semantic region based on the user input (fig 2; paragraph 27, lines 1-9; paragraphs 33 and 34; paragraphs 64 and 65).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the semantic mask segmentation of Cohen with the mask segmentation of Phogat because this provides an easier and more intuitive way for users to interact with the image segmentation interface, and reduce the amount of preprocessed training, and can segment object classes that have large appearance variations, thereby providing both ease of use and accuracy of segmentation (see, for example, Cohen, paragraph 156, the last 21 lines).
Regarding claim 19, Phogat in view of Cohen teaches the non-transitory computer-readable medium of claim 15, wherein generating the set of mask polylines comprises: accessing Bezier curve approximations of the semantic mask; extracting mask contours of the semantic mask; and generating the set of mask polylines from the Bezier curve approximations of the semantic mask and the mask contours of the semantic mask (Phogat, paragraph 47; paragraph 48, lines 1-4; paragraph 59, lines 1-19; paragraph 60; paragraph 70; paragraph 106; paragraph 151, the last 8 lines; Cohen, paragraph 27, lines 1-9; paragraphs 33 and 34; paragraphs 64 and 65; the motivation to combine is substantially similar to that discussed above in the rejection of claim 15).
Regarding claim 20, Phogat in view of Cohen teaches the non-transitory computer-readable medium of claim 15, wherein the operations further comprise: receiving a refinement selection of the semantic mask as user input, wherein the refinement selection comprises a reduced semantic region relative to the semantic region; generating an updated semantic mask based on the refinement selection; and from the updated semantic mask, providing additional visible vector paths as the selection of the reduced semantic region (Phogat, paragraph 47, the last 12 lines; paragraphs 60 and 61; paragraphs 101 and 103; paragraphs 106 and 111; Cohen, paragraph 27, lines 1-9; paragraphs 33 and 34; paragraphs 64 and 65; the motivation to combine is substantially similar to that discussed above in the rejection of claim 15).
Claims 2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Phogat, in view of Cohen, and further in view of Beri et al. (U.S. Patent Application Publication No. 2021/0333960), referred herein as Beri.
Regarding claim 2, Phogat in view of Cohen teaches the computer-implemented method of claim 1, wherein utilizing the polyline model comprises: generating, utilizing the polyline model, a set of mask polylines from the semantic mask; and generating, utilizing the polyline model, polylines from vector paths in the vector graphic (Phogat, paragraph 47, lines 1-14; paragraph 48, lines 1-4; paragraph 59, lines 9-19; paragraph 70; paragraph 106, lines 1-11; Cohen, paragraph 27, lines 1-9; paragraphs 33 and 34; paragraphs 64 and 65; the motivation to combine is substantially similar to that discussed above in the rejection of claim 1).
Phogat in view of Cohen does not explicitly teach a set of group polylines from a grouping of vector paths.
However, in a similar field of endeavor, Beri teaches a method comprising accessing vector graphics comprising shapes, and in response to user input indicating a portion of the shapes, generating a mask based on the portion using a polyline model, and selecting a set of vector paths in the graphic (figs 2, 3, and 5; paragraph 38; paragraphs 54 and 55; paragraph 61), wherein the generating and selecting comprises generating a set of group polylines from a grouping of vector paths in the vector graphic (paragraph 21, the last 17 lines; paragraph 31; paragraph 39, lines 1-10; paragraph 61).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the polyline and vector path grouping of Beri with the polyline and vector path processing of Phogat in view of Cohen because this helps increase the efficiency and ease of the user input to manipulate the vector graphics, while maintaining high quality vector graphics output (see, for example, Beri, paragraph 28).
Regarding claim 10, Phogat teaches a system comprising: one or more memory devices; and one or more processors coupled to the one or more memory devices, configured to cause the system to (fig 11; paragraphs 159 and 160; paragraph 163):
in response to user input indicating a point of a vector graphic, generate, utilizing a segmentation model, a mask that defines a region in the vector graphic (paragraph 40, lines 1-3 and 8-12; paragraph 41, the last 9 lines; paragraph 43, lines 1-8; paragraph 46, lines 1-7; paragraph 68; in response to user input identifying a point of a vector graphic, a mask defining a region in the graphic is generated using a segmentation model);
generate, utilizing a polyline model, a set of mask polylines from the mask and polylines from vector objects (paragraph 47; paragraph 48, lines 1-4; paragraph 59, lines 9-19; paragraph 60; paragraph 70; paragraph 106; a polyline model is used to generate mask polylines from the mask and polylines from vector objects);
based on the set of mask polylines and the object polylines, select a set of vector paths in the vector graphic that satisfies an overlap threshold with the mask that defines the region in the vector graphic, and provide the set of vector paths as a selection of the region (paragraph 47; paragraph 48, lines 1-4; paragraph 59, lines 9-19; paragraph 60; paragraph 70; paragraph 106; paragraph 151, the last 8 lines; based on the mask polylines and object polylines, vector paths are selected that have an amount of path overlap, and are provided as a selection of the region).
Phogat does not explicitly teach a semantic segmentation model, semantic mask, and semantic region.
However, in a similar field of endeavor, Cohen teaches a system performing operations comprising accessing an image comprising a set of shapes, and in response to user input indicating a portion of the graphic, generating, utilizing a semantic segmentation model, a semantic mask of a semantic region based on the user input (fig 2; paragraph 27, lines 1-9; paragraphs 33 and 34; paragraphs 64 and 65).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the semantic mask segmentation of Cohen with the mask segmentation of Phogat because this provides an easier and more intuitive way for users to interact with the image segmentation interface, and reduce the amount of preprocessed training, and can segment object classes that have large appearance variations, thereby providing both ease of use and accuracy of segmentation (see, for example, Cohen, paragraph 156, the last 21 lines).
Phogat in view of Cohen does not explicitly teach a set of group polylines from a grouping of vector paths.
However, in a similar field of endeavor, Beri teaches a system performing operations comprising accessing vector graphics comprising shapes, and in response to user input indicating a portion of the shapes, generating a mask based on the portion using a polyline model, and selecting a set of vector paths in the graphic (figs 2, 3, and 5; paragraph 38; paragraphs 54 and 55; paragraph 61), wherein the generating and selecting comprises generating a set of group polylines from a grouping of vector paths in the vector graphic (paragraph 21, the last 17 lines; paragraph 31; paragraph 39, lines 1-10; paragraph 61).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the polyline and vector path grouping of Beri with the polyline and vector path processing of Phogat in view of Cohen because this helps increase the efficiency and ease of the user input to manipulate the vector graphics, while maintaining high quality vector graphics output (see, for example, Beri, paragraph 28).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Phogat, in view of Cohen, and further in view of Price et al. (U.S. Patent Application Publication No. 2022/0198671), referred herein as Price.
Regarding claim 9, Phogat in view of Cohen teaches the computer-implemented method of claim 1, wherein accessing the vector graphic comprises: accessing the set of shapes arranged in different regions of a digital design document, wherein the set of shapes are vector objects (Phogat, paragraph 40, lines 1-3 and 8-12; paragraph 43, lines 1-8; paragraph 64, lines 1-3); and
accessing one or more groupings in the vector graphic, wherein the one or more groupings comprises one or more vector objects categorized as a single unit (Phogat, paragraph 30, the last 20 lines; paragraph 80, lines 1-17; Beri, paragraph 21, the last 17 lines; paragraph 31; paragraph 39, lines 1-10; paragraph 61).
Phogat in view of Cohen does not explicitly teach shapes in different layers.
However, in a similar field of endeavor, Price teaches a method comprising accessing vector graphics comprising a plurality of objects comprising shapes, and in response to user input indicating portions of the objects, utilizing a segmentation model to generate a mask of a region (paragraphs 22 and 23; paragraph 24, lines 1-12; paragraphs 98 and 99), wherein the shapes are arranged in different layers of a digital design document (paragraph 38; paragraphs 82 and 84).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the shape layering of Price with the shapes of Phogat in view of Cohen because this separates intended segmentation targets, so as to improve the accuracy and sensitivity of the segmentation (see, for example, Price, paragraph 17, the last 11 lines; paragraphs 22 and 23).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Phogat, in view of Cohen, further in view of Beri, and further in view of Price et al. (U.S. Patent Application Publication No. 2022/0198671), referred herein as Price.
Regarding claim 14, Phogat in view of Cohen, further in view of Beri teaches the system of claim 10, wherein the one or more processors are configured to cause the system to access the vector graphic by: accessing a set of shapes arranged in different regions of a digital design document, wherein the set of shapes are vector objects of the grouping of vector objects (Phogat, paragraph 40, lines 1-3 and 8-12; paragraph 43, lines 1-8; paragraph 64, lines 1-3; Beri, paragraph 21, the last 17 lines; paragraph 31; paragraph 39, lines 1-10; paragraph 61); and
accessing one or more groupings in the vector graphic, wherein the one or more groupings comprises one or more vector objects categorized as a single unit and the one or more groupings further comprises one or more nested groupings (Phogat, paragraph 30, the last 20 lines; paragraph 80, lines 1-17; Beri, paragraph 21, the last 17 lines; paragraph 31; paragraph 39, lines 1-10; paragraph 61).
Phogat in view of Cohen, further in view of Beri does not explicitly teach shapes in different layers.
However, in a similar field of endeavor, Price teaches a system performing operations comprising accessing vector graphics comprising a plurality of objects comprising shapes, and in response to user input indicating portions of the objects, utilizing a segmentation model to generate a mask of a region (paragraphs 22 and 23; paragraph 24, lines 1-12; paragraphs 98 and 99), wherein the shapes are arranged in different layers of a digital design document (paragraph 38; paragraphs 82 and 84).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the shape layering of Price with the shapes of Phogat in view of Cohen, further in view of Beri, because this separates intended segmentation targets, so as to improve the accuracy and sensitivity of the segmentation (see, for example, Price, paragraph 17, the last 11 lines; paragraphs 22 and 23).
Allowable Subject Matter
Claims 3-6, 8, 11-13, and 16-18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following are statements of reasons for the indication of allowable subject matter:
Regarding claim 3, the prior art teaches the computer-implemented method of claim 2. In the context of claims 1, 2, and 3 as a whole, however, the prior art does not appear to teach the computer-implemented method of claim 2, wherein selecting the set of vector paths in the vector graphic comprises: determining an intersection over union between the set of mask polylines and the set of group polylines; and based on the intersection over union satisfying an overlap threshold, selecting the grouping of vector paths as the set of vector paths to return as the selection of the semantic region. The subject matter of this claim is therefore allowable.
Regarding claim 4, the prior art teaches the computer-implemented method of claim 2. In the context of claims 1, 2, and 4 as a whole, however, the prior art does not appear to teach the computer-implemented method of claim 2, wherein selecting the set of vector paths in the vector graphic comprises: based on an intersection over union between the set of mask polylines and the set of group polylines failing to satisfy an overlap threshold, generating a set of nested polylines for a nested grouping of vector paths within the grouping of vector paths; determining an additional intersection over union between the set of mask polylines and the set of nested polylines; and based on the additional intersection over union satisfying the overlap threshold, selecting the nested grouping of vector paths as the set of vector paths to return as the selection of the semantic region. The subject matter of this claim is therefore allowable.
Regarding claim 5, the prior art teaches the computer-implemented method of claim 2. In the context of claims 1, 2, and 5 as a whole, however, the prior art does not appear to teach the computer-implemented method of claim 2, wherein selecting the set of vector paths in the vector graphic comprises: based on an intersection over union between the set of mask polylines and the set of group polylines failing to satisfy an overlap threshold, identifying visible vector paths in the vector graphic by using a pre-order traversal of the vector graphic that defines a painting order of the vector graphic; and generating a set of visible vector path polylines from the visible vector paths in the vector graphic. The subject matter of this claim is therefore allowable.
Regarding claim 6, this claim comprises allowable subject matter insomuch as it depends from claim 5, which comprises allowable subject matter.
Regarding claim 8, the prior art teaches the computer-implemented method of claim 1. In the context of claims 1 and 8 as a whole, however, the prior art does not appear to teach the computer-implemented method of claim 1, further comprising: generating a prediction score for the semantic mask; generating a first additional semantic mask, a first additional prediction score, a second additional semantic mask, and a second additional prediction score based on the user input indicating the portion in the vector graphic; and based on the prediction score being greater than the first additional prediction score and the second additional prediction score, selecting the semantic mask. The subject matter of this claim is therefore allowable.
Regarding claim 11, the prior art teaches the system of claim 10. In the context of claims 10 and 11 as a whole, however, the prior art does not appear to teach the system of claim 10, wherein the one or more processors are configured to cause the system to select the set of vector paths in the vector graphic by: determining an intersection over union between the set of mask polylines and the set of group polylines; based on the intersection over union failing to satisfy an overlap threshold, generating a set of nested polylines for a nested grouping of vector paths within the grouping of vector objects; and determining a first additional intersection over union between the set of mask polylines and the set of nested polylines for the nested grouping of vector paths. The subject matter of this claim is therefore allowable.
Regarding claims 12 and 13, these claims comprise allowable subject matter insomuch as they depend from claims that comprise allowable subject matter.
Regarding claim 16, the prior art teaches the non-transitory computer-readable medium of claim 15. In the context of claims 15 and 16 as a whole, however, the prior art does not appear to teach the non-transitory computer-readable medium of claim 15, wherein generating the set of visible vector path polylines comprises: constructing a list of vector paths in the vector graphic by using a pre-order traversal of the vector graphic that defines a painting order of the vector graphic, wherein vector paths earlier in the painting order appear on top of vector paths that come later in the painting order; and generating, utilizing the polyline model, the set of visible vector path polylines from the list of vector paths in the vector graphic. The subject matter of this claim is therefore allowable.
Regarding claims 17 and 18, these claims comprise allowable subject matter insomuch as they depend from claims that comprise allowable subject matter.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Antony (U.S. Patent No. 7,426,455); Optimal boolean set operation generation among polygon-represented regions.
Mori (U.S. Patent Application Publication No. 2013/0249911); Display processing method and apparatus.
Bouhnik (U.S. Patent No. 10,540,757); Method and system for generating combined images utilizing image processing of multiple images.
Phogat (U.S. Patent Application Publication No. 2021/0042965); Flow-based color transfer from source graphic to target graphic.
Phogat (U.S. Patent Application Publication No. 2021/0142479); Segmenting objects in vector graphics images.
Fisher (U.S. Patent Application Publication No. 2022/0254076); Automated digital tool identification from a rasterized image.
Ling (U.S. Patent Application Publication No. 2022/0383570); High-precision semantic image editing using neural networks for synthetic data generation systems and applications.
Smith (U.S. Patent Application Publication No. 2024/0265692); Generating semantic scene graphs utilizing template graphs for digital image modification.
Pandey (U.S. Patent No. 12,417,570); Vectorizing by piecewise deconstruction of object strokes.
Nagi (U.S. Patent Application Publication No. 2026/0279025); Optimizing recognition accuracy by thin line segmentation models using intersection over length.
Chung (U.S. Patent Application Publication No. 2026/0162449); Systems and methods for automated semantic segmentation and label generation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached on Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights