DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Amendment filed on 4/22/2026.
Claims 1-20 are pending. Claims 9, 17 have been amended.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tremblay et al. (US 20190251397 A1, hereinafter Tremblay) in view of Gupta et al. (US 20190158112 A1, hereinafter Gupta).
Regarding Claim 1, Tremblay teaches a system comprising: a memory component; and a processing device coupled to the memory component, the processing device to perform operations including: (Tremblay, Paragraph [0021], "The labeled training data generation system 200 includes a graphics processing unit (GPU) 110, the task-specific training data computation unit 115, and an input image generator 220."; Fig. 3, Element 304 (Memory); Paragraph [0020], "the generated training data may be used to train neural networks.") receiving a labeled vector image that specifies use of a digital tool; (Tremblay, Paragraph [0020], "Training deep neural networks requires a large amount of labeled training data"; Paragraph [0030], "one or more of the 3D synthetic objects are selected by the labeled training data generation system 100"; Paragraph [0034], "At step 135, the GPU 110 renders a 3D object of interest to produce a rendered image of the object of interest. The 3D object of interest is rendered according to the rendering parameters <read on digital tool>.") identifying one or more parameter configurations for the digital tool identified in the labeled vector image; (Tremblay, Paragraph [0022], "The rendering parameters may specify a position and/or orientation of the object of interest in a 3D scene, a position and/or orientation of a virtual camera, one or more texture maps, one or more lights including color, type, intensity, position and/or orientation, and the like.") generating a trained segmentation network and a trained classification network using the labeled [[vector]] image and the one or more parameter configurations; (Tremblay, Paragraph [0004], "The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects"; Paragraph [0024], "In an embodiment, the task is segmentation and the training data computation unit 115 determines an object identifier ... and computes the task-specific training data as a segmentation map corresponding to the input image ... where each pixel ... is colored according to the object identifier"; Paragraph [0127], "a deep learning or neural learning system needs to be trained in object recognition and classification for it get smarter and more efficient at identifying basic objects.") outputting the trained segmentation network and the trained classification network, the trained segmentation network and the trained classification network configured to automatically identify the digital tool used to achieve a visual appearance in an input image. (Tremblay, Paragraph [0020], "A domain randomization technique to generate training data that is automatically labeled is described. The generated training data may be used to train neural networks for object detection and segmentation tasks"; Paragraph [0032], "During supervised training of a neural network model, the task-specific training data is ground truth labels ... compared with an output generated by the neural network model when the input image is processed by the neural network model.")
But Tremblay does not explicitly disclose receiving a labeled vector image that specifies use of a digital tool, generating a trained classification network, outputting the trained segmentation network and the trained classification network configured to automatically identify the digital tool used to achieve a visual appearance in an input image.
However, Gupta teaches a memory component; and a processing device coupled to the memory component (Gupta, Paragraph [0091], "includes a processor 802 communicatively coupled to one or more memory devices 804."; Paragraph [0092], "The memory device 804 includes any suitable non-transitory computer-readable medium for storing data, program code, or both.") receiving a labeled vector image that specifies use of a digital tool (Gupta, Paragraph [0051], "At block 312, the process 300 involves causing a computing device (e.g., a user device 227) to display or otherwise provide an output vector graphic 226 <read on labeled vector image>"; Paragraph [0029], "uses a customization-identification network 208 to convert various types of raster graphics ... into output vector graphics 226"; Paragraph [0037], "allows a graphic manipulation application 206 to use the visual characteristics <read on digital tool> of an input raster graphic 204 to select customization operations"; Paragraph [0084], "the 'sketch' label in the tag 702 indicates a set of customization operations <read on specifies use of a digital tool> performed by the vectorization algorithm 400.") generating a trained classification network using the labeled vector image and the one or more parameter configurations (Gupta, Paragraph [0020], "a deep-learning network or other neural network model classifies an input image or other raster graphic into a set of classes ... In various embodiments, using a trained neural network to select multiple vectorization operations specific to the characteristics of the input graphic can provide improved raster-to-vector conversions without requiring manual input to modify the vectorization process"; Paragraph [0035], "executes the customization training module 228 to generate, train, or otherwise develop a customization-identification network 208 <read on classification network>") and outputting the trained classification network (Gupta, Paragraph [0037], "outputs the customization-identification network 208 <read on outputting the trained classification network>") configured to automatically identify the digital tool used to achieve a visual appearance in an input image (Gupta, Paragraph [0037], "allows a graphic manipulation application 206 to use the visual characteristics <read on digital tool> of an input raster graphic 204"; Paragraph [0011], "FIG. 3 depicts an example of a process for using a customization-identification network to automatically select and apply one or more custom vectorization operations <read on automatically identify the digital tool used to achieve a visual appearance in an input image>.")
Gupta and Tremblay are analogous since both are dealing with generating labeled training data and training neural network models for image-related processing workflows. Tremblay provided a way of generating task-specific labeled training data including segmentation maps paired with input images for training a neural network model. Gupta provided a way of training a classification network (customization-identification network) to identify the digital tool used to achieve a visual appearance in an input image, and outputting the trained network for downstream use. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the classification network training and output capability taught by Gupta into the modified invention of Tremblay such that the combined system trains and outputs a classification network configured to automatically identify the digital tool used to achieve a visual appearance in an input image. The motivation is to enable automated identification of the specific digital tool responsible for a visual appearance.
Regarding Claim 2, the combination of Tremblay and Gupta teaches the
invention in Claim 1.
The combination further teaches wherein the trained segmentation network and
the trained classification network are further configured to automatically identify a
parameter configuration for the digital tool used to achieve the visual appearance
(Tremblay, Paragraph [0020], "A domain randomization technique to generate
training data that is automatically labeled is described. The generated training
data may be used to train neural networks for object detection and segmentation
tasks" [0032], "During supervised training of a neural network model, the task specific training data is ground truth labels ... compared with an output generated
by the neural network model when the input image is processed by the neural
network model"). Tremblay does not explicitly disclose but Gupta teaches configured to automatically identify a parameter configuration for the digital tool used to achieve the visual appearance (Gupta, Paragraph [0037], "allows a graphic manipulation application 206 to use the visual characteristics < read on digital tool> of an input raster graphic 204"; [0011], "FIG. 3 depicts an example of a process for using a customization-identification network to automatically select
and apply one or more custom vectorization operations").
As explained in rejection of Claim 1, the obviousness for combining of outputting the trained network for downstream use of Gupta into Tremblay is provided above.
Regarding Claim 3, the combination of Tremblay and Gupta teaches the
invention of generating the trained segmentation network and the trained classification network in Claim 1.
The combination further teaches generating a rasterized image (Tremblay,
Paragraph [0034], "At step 135, the GPU 110 renders a 30 object of interest to
produce a rendered image of the object of interest; [0114], “the graphics processing pipeline 600 comprises a pipeline architecture that includes a number of stages. The stages include…a rasterization stage 660; [0112], “After the processed vertex data is rasterized <read on rasterized image>"). and generating a training sample mask for the rasterized image (Tremblay, Paragraph [0035], "for segmentation, the task-specific training data is the rendered objects of interest having each pixel within a rendered object of interest replaced with an object identifier <read on training sample mask>").
However, Tremblay does not explicitly disclose: [[from the labeled vector image]].
But Gupta teaches generating a rasterized image from the labeled vector image (Gupta, Paragraph [0051], "At block 312, the process 300 involves causing a computing device (e.g., a user device 227) to display or otherwise provide an output vector graphic 226 <read on labeled vector image>").
As explained in rejection of Claim 1, the obviousness for combining Gupta's
output vector graphic into Tremblay's rasterization/training pipeline is provided above.
Regarding Claim 4, The combination of Tremblay and Gupta teaches the
invention in Claim 3.
The combination further teaches wherein the processing device is configured to
generate the rasterized image (Tremblay, Paragraph [0034], "At step 135, the GPU
110 renders a 3D object of interest to produce a rendered image of the object of
interest <read on rasterized image>").
But Tremblay does not explicitly disclose: generating an augmented vector
image from the labeled vector image and generate the rasterized image from the
augmented vector image.
However, Gupta teaches generating an augmented vector image from the
labeled vector image (Gupta, Paragraph [0047], "the vectorization algorithm applies
a set of customization operations <read on augmented vector image>").
and wherein the processing device is configured to generate the rasterized image from
the augmented vector image (Gupta, Paragraph [0047], "Executing the graphic
manipulation application 206 causes the processing device to perform a
vectorization algorithm <read on processing device configured to generate
rasterized image from augmented vector image>"; Paragraph [0051], "At block 312, the process 300 involves causing a computing device (e.g., a user device 227) to display or otherwise provide an output vector graphic 226 <read on rasterized image>").
Gupta and Tremblay are analogous since both are directed to image-processing
pipelines that transform source representations into rasterized images for downstream
processing and analysis. Tremblay provides a way of generating rasterized images for
training neural network models by rendering source data using configurable parameters. Gupta provides a way of modifying and outputting vector graphics via customization operations executed by a processing device. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the vector-image customization operations taught by Gupta into the modified invention of Tremblay, such that an augmented vector image is first generated and then rasterized for use in training or processing, as opposed to rasterizing an unmodified source. The motivation for such a combination is to increase variation and diversity in generated training data by modifying vector representations prior to rasterization, as discussed by Gupta in Paragraph [0047], which explains applying customization operations during vectorization, and consistent with Tremblay's objective of generating diverse rendered images for training purposes.
Regarding Claim 5, the combination of Tremblay and Gupta teaches the
invention in Claim 4 with processing device is configured to generate the augmented vector image.
The combination further teaches varying a position of a stylized geometry
element in the labeled [[ vector ]] image (Tremblay, Paragraph [0042], “one or more of the 3D geometric shape are selected by the labeled training data generation system 100” [0043], "positioning the rendered object(s) of interest and the rendered geometric shapes at various positions within the input image <read on varying a position>"). varying a size of the stylized geometry element in the labeled [[ vector ]] image (Tremblay, Paragraph [0043], "Each the rendered geometric shape may be scaled in size and/or rotated <read on varying a size>"). varying a color of the stylized geometry element in the labeled [[ vector ]] image (Tremblay, Paragraph [0042], "A color of and/or the orientation of each 3D geometric shape may vary <read on varying a color>"). altering a parameter configuration of the digital tool applied to the stylized geometry element in the labeled [[ vector ]] image (Tremblay, Paragraph [0042], "Note that at least a portion of the rendering parameters vary for each one of the rendered images. <read on altering a parameter configuration> "). adjusting a hierarchical placement of the stylized geometry element in the labeled [[ vector ]] image (Tremblay, Paragraph [0044], "A particular rendered object of interest may be occluded by one or more other rendered objects of interest and/or rendered geometric shapes <read on adjusting a hierarchical
placement>"). inserting a stylized geometry element from a different labeled [[ vector ]] image into the labeled vector image (Tremblay, Paragraph [0037], "a random number of rendered 30 geometric shapes may be inserted into the input image <read on inserting a stylized geometry element>"; [0042], “the object of interest may be rendered according to different rendering parameters to produce additional rendered images of the object of interest.”).
But, Tremblay does not explicitly disclose that [[ the above variations are performed in the labeled ]] vector [[ image and/or that the inserted stylized geometry element is from a different labeled ]] vector [[ image]].
However, Gupta teaches varying/altering aspects of a vector image (Gupta,
Paragraph [0047], "The graphic manipulation application 206 transforms
the accessed raster graphic into a vector graphic…the vectorization algorithm applies a set of customization operations <read on varying position/size/color and inserting elements in a vector image>"). altering a parameter configuration of the digital tool (Gupta, Paragraph [0081], "sets the edge-sensitivity value to the user-specified slide value V for that pixel in the edge-sensitivity map <read on parameter configuration>").
Gupta and Tremblay are analogous since both are directed to systematic
variation of visual characteristics in images using parameterized operations executed by
a processing device. Tremblay provides a way of varying position, size, color, rendering
parameters, relative placement, and insertion of visual elements during generation of
rendered images for training purposes. Gupta provides a way of applying customization
operations and parameter configurations directly to vector graphics during vectorization.
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention was made to incorporate vector-image
customization operations and parameter configurations taught by Gupta into the
modified invention of Tremblay, such that the variations of stylized geometry elements
(including position, size, color, parameter configuration, hierarchical placement, and
insertion) are performed at the vector-image level prior to rasterization, rather than
being limited to variations applied only during rendering. The motivation is to enable controlled, repeatable, and parameter-driven variation of stylized geometry elements using vector-level representations, as discussed by Gupta in Paragraphs [004 7] and [0081], which improves flexibility and efficiency in generating diverse training data consistent with Tremblay's image-generation objectives.
Regarding Claim 6, the combination of Tremblay and Gupta teaches the invention in Claim 1.
The combination further teaches generating, using the trained segmentation network and the trained classification network, an interactive image based on the input image (Tremblay, Paragraph [0049], "the training data computation unit 115 compares the output generated by the neural network model 260 when the input image is processed to the ground truth labels. <read on using trained
network to process an input image and generate an output>").
But Tremblay does not explicitly disclose interactive image configured to display an image tool description]; the image tool description that provides an indication of the digital tool; and a parameter configuration for the digital tool.
However, Gupta teaches an interactive image ... configured to display an image tool description (Gupta, Paragraph [0084], "the interface displays a tag 702 that provides feedback data 240. <read on image tool description>"). the image tool description that provides an indication of the digital tool (Gupta, Paragraph [0084], "the "sketch" label in the tag 702 indicates a set of customization operations performed by the vectorization algorithm 400. <read on indication of the digital tool>"). and a parameter configuration for the digital tool used to achieve the visual appearance (Gupta, Paragraph (0084], "The interface 700 also depicts an edge-sensitivity box 708 and a corresponding slide tool 710. <read on parameter configuration>").
Gupta and Tremblay are analogous since both of them are dealing with image-processing workflows that use trained-model outputs and present results to a user via a user interface. Tremblay provided a way of generating labeled training data (including segmentation maps) for training a neural network model that processes an input image. Gupta provided a way of displaying, in a user interface, an indication of identified operations and an associated adjustable parameter value (e.g., tag 702 and edge sensitivity 708). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to inco rporate user-interface presentation of identified operations and parameter information taught by Gupta into modified invention of Tremblay such that the trained segmentation network and trained classification network outputs are used to generate an interactive image configured to display an image tool description providing an indication of a digital tool and a parameter configuration for the digital tool used to achieve the visual appearance. The motivation is to provide user-visible feedback and controls corresponding to automatically determined operations/parameters discussed by Gupta in Paragraph [0084];
Regarding Claim 7, the combination of Tremblay and Gupta teaches the invention in Claim 6.
The combination further teaches wherein the processing device is configured to apply the parameter configuration for the digital tool to an additional input image displayed by the processing device responsive to receiving input at the image tool description. (Gupta, Paragraph [0049] “At block 308, the process 300 involves selecting the first customization operation as the customization specific to the input raster graphic” [0045] “At block 302, the process 300 involves accessing an input raster graphic 204, such as a scan of a drawing, a scan of an oil painting, an image, etc <read on apply the parameter configuration>”; Paragraph [0084], "The interface 700 also provides an option ( i.e., a "default" button) to revert the classification decision <read on responsive to receiving input at the image tool description>").
Gupta and Tremblay are analogous since both of them are dealing with image-processing workflows in which identified operations/parameters can be applied and adjusted by a user. Tremblay provided a way of producing trained-model outputs from an input image. Gupta provided a way of selecting operations to be performed and providing user interface controls (e .g., a default button) to apply or revert the selected operations. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate user-driven application/reversion of selected operations and associated parameter configurations taught by Gupta into modified invention of Tremblay such that, responsive to receiving input at the image tool description, the processing device applies the parameter configuration for the digital tool to an additional input image displayed by the processing device. The motivation is to allow a user to apply,
adjust, or revert identified settings via interface interaction discussed by Gupta in Paragraphs [0049] and [0084];
Regarding Claim 8, the combination of Tremblay and Gupta teaches the invention in Claim 1.
The combination further teaches wherein the t rained segmentation network is configured to output a binary mask that indicates whether each pixel of the input image was generated using the digital tool (Tremblay, Paragraph [0035], "for segmentation, the task-specific training data is the rendered objects of interest having each pixel within a rendered object of interest replaced with an object identifier"; Paragraph [0024], " the task is segmentation and the training data computation unit 115 determines an object identifier for each rendered image of an object of interest and computes the task-specific training data as a segmentation map corresponding to the input image"; it is noted since rendered is apply to each pixel and it's apply to identifier vs no identifier (which is two states), so it read on a binary mask output).
Claim(s) 9-12, 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tremblay et al. (US 20190251397 A1, hereinafter Tremblay) in view of Gupta et al. (US 20190158112 A1, hereinafter Gupta), further in view of Yang et al. (US 20190026550 A1, hereinafter Yang).
Regarding Claim 9, Tremblay teaches receiving, by a processing device, a labeled vector image [[ having metadata that specifies use of a digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image ]] (Tremblay, Paragraph [0020], "Training deep neural networks requires a large amount of labeled training data. A domain randomization technique to generate training data that is automatically labeled is described"; Paragraph [0034], "At step 135, the GPU 110 renders a 3D object of interest to produce a rendered image of the object of interest, where an input image comprises the rendered image of the object of interest and a background image. The 3D object of interest is rendered according to the rendering parameters"),
generating, by the processing device, a training sample mask that denotes a region at which the digital tool was applied to the labeled vector image [[based on the metadata]] (Tremblay, Paragraph [0035], "for segmentation, the task-specific training data is the rendered objects of interest having each pixel within a rendered object of interest replaced with an object identifier associated with the rendered object of interest"; Paragraph [0024], "the task is segmentation and the training data computation unit 115 determines an object identifier for each rendered image of an object of interest and computes the task-specific training data as a segmentation map corresponding to the input image … where each pixel that is covered by a rendered image is colored according to the object identifier determined for the instance"), training, by the processing device, a segmentation network using the labeled vector image and the training sample mask, the segmentation network configured to automatically identify the digital tool used to achieve the visual appearance in the input image (Tremblay, Paragraph [0032], "During supervised training of a neural network model, the task-specific training data is ground truth labels … compared with an output generated by the neural network model when the input image is processed by the neural network model"; Paragraph [0004, "The generated training data may be used to train neural networks for object detection and segmentation labelling tasks"), outputting, by the processing device, the segmentation network (Tremblay, Paragraph [0052], "the loss function unit 270 generates updated weights to reduce differences between the task-specific training data and the output data. When the differences are reduced to a predetermined value, training is complete").
But Tremblay does not explicitly disclose having metadata that specifies use of a digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image, and based on the metadata.
However, Gupta teaches receiving, by a processing device, a labeled vector image (Gupta, Paragraph [0029], "Gupta's graphic manipulation application uses a customization-identification network 208 to convert various types of raster graphics … into output vector graphics 226" <read on: labeled vector image>), the segmentation network configured to automatically identify the digital tool used to achieve the visual appearance in the input image (Gupta, Paragraph [0037], "allows a graphic manipulation application 206 to use the visual characteristics <read on: digital tool] of an input raster graphic 204"; Paragraph [0035], "executes the customization training module 228 to generate, train, or otherwise develop a customization-identification network 208" <read on: segmentation network configured to automatically identify the digital tool]; Paragraph [0011], "FIG. 3 depicts an example of a process for using a customization-identification network to automatically select and apply one or more custom vectorization operations" <read on: automatically identify the digital tool used to achieve the visual appearance in the input image>).
Gupta and Tremblay are analogous since both are dealing with generating labeled training data and training neural network models that automatically identify visual characteristics and digital tool operations applied to image content. Tremblay provided a way of generating segmentation maps paired with input images and training a segmentation neural network model. Gupta provided a way of training a network to automatically identify the digital tool used to achieve a visual appearance in an input image. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the digital-tool identification capability taught by Gupta into the modified invention of Tremblay such that the segmentation network is configured to automatically identify the digital tool used to achieve the visual appearance in the input image. The motivation is to enable automatic identification of the specific digital tool responsible for a visual appearance, as discussed by Gupta in Paragraph [0037].
But the combination of Tremblay and Gupta does not explicitly disclose having metadata that specifies use of a digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image, and based on the metadata.
However, Yang teaches having metadata that specifies use of a digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image (Yang, Paragraph [0037], "Digital tools, as described herein, include tools such as the content manipulation application 110 that are used to perform a function or a workflow electronically … a creation tool, content editing tool, content publishing tool, content tracking tool … or any other tool that is used for creating, editing, managing, generating, tracking, consuming or performing any other function or workflow related to content" <read on: digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image]; Paragraph [0048], "An application program 108 is an example of a digital tool" <read on: computer-implemented mechanism>; Paragraph [0042], "In some embodiments, each asset includes metadata" <read on: labeled vector image having metadata>; Paragraph [0020], "Appearance-based information includes information about visual characteristics in an image of the page. Some examples of these visual characteristics include different text sizes, fonts, column-layouts, etc." <read on: metadata that specifies use of a digital tool executable for altering a visual appearance of an input image>), based on the metadata (Yang, Paragraph [0083], "If the electronic document is a vector graphics document, the post-processing algorithm identifies the rendering commands used to generate different objects in the document. For instance, a set of rendering commands could indicate that a portion of the document belongs to a common object" <read on: based on the metadata/rendering commands embedded in the labeled vector image>; Paragraph [0089], "the training process uses rendering commands from a vector graphic document … rendering commands might indicate that two different document regions … belong to the same object … a common bounding box is applied to both document regions" <read on: generating a training sample mask that denotes the region at which the digital tool was applied, based on the metadata>).
Yang and Tremblay as modified by Gupta are analogous since all three are dealing with generating training data for neural network models by leveraging structured information embedded in document or image representations to identify the regions and operations associated with specific digital tools. Tremblay as modified by Gupta provided a way of generating segmentation training data from input images and training a segmentation network to automatically identify the digital tool used to achieve the visual appearance. Yang provided a way of using metadata embedded in a labeled vector graphics document — specifically rendering commands that specify the digital tool applied and the regions to which it was applied — to generate training masks and supply them to a neural network training workflow. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the metadata-bearing labeled vector image and the metadata-based training mask generation taught by Yang into the modified invention of Tremblay as further modified by Gupta, such that the processing device receives a labeled vector image having metadata that specifies use of a digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image, and generates the training sample mask that denotes the region at which the digital tool was applied based on the metadata. The motivation is to leverage metadata already embedded in vector graphics documents to automatically generate accurate training masks without manual annotation, as discussed by Yang in Paragraph [0089].
Regarding Claim 10, the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 9.
The combination further teaches receiving a vector image (Gupta, Paragraph [0051], "At block 312, the process 300 involves causing a computing device (e.g., a user device 227) to display or otherwise provide an output vector graphic 226."); and wherein the labeled vector image is generated automatically and without user intervention (Gupta, Paragraph [0020], "using a trained neural network to select multiple vectorization operations specific to the characteristics of the input graphic can provide improved raster-to-vector conversions without requiring manual input <read on without user intervention> to modify the vectorization process <read on generated automatically>"). based on metadata associated with the digital tool and the vector image (Gupta, Paragraph [0037], "allows a graphic manipulation application 206 to use the visual characteristics <read on metadata> of an input raster graphic 204 to select customization operations <read on digital tool>"; Paragraph [0051], "At block 312, the process 300 involves causing a computing device (e.g., a user device 227) to display or otherwise provide an output vector graphic 226.").
As explained in rejection of Claim 9, the obviousness for combining Gupta
with Tremblay is provided above.
Regarding Claim 11, the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 10.
The combination further teaches generate a rasterized image (Tremblay,
Paragraph [0034], "At step 135, the GPU 110 renders a 3D object of interest
to produce a rendered image of the object of interest. <read on rasterized
image>"). and generate the training sample mask from the rasterized image (Tremblay, Paragraph [0035], "for segmentation, the task-specific training data is the rendered objects of interest having each pixel within a rendered object of
interest replaced with an object identifier. <read on generating the training
sample mask from the rasterized image>").
But Tremblay does not explicitly disclose: generating an augmented
vector image from the labeled vector image and generating the rasterized
image from the augmented vector image.
However, Gupta teaches generating an augmented vector image from the
labeled vector image {Gupta, Paragraph [0047], "the vectorization algorithm
applies a set of customization operations <read on augmented vector
image>,") wherein the processing device is configured to generate the rasterized
image from the augmented vector image (Gupta, Paragraph [0047], "Executing
the graphic manipulation application 206 causes the processing device to
perform a vectorization algorithm <read on generating the rasterized image
from the augmented vector image>" Paragraph [0051], "At block 312, the
process 300 involves causing a computing device (e.g., a user device 227)
to display or otherwise provide an output vector graphic 226. <read on
labeled vector image used as source for rasterization>").
Gupta and Tremblay are analogous since both of them are dealing with
generating image data for downstream processing/training by applying
configurable operations to source representations. Tremblay provided a way of
generating rasterized images (rendered images) and corresponding
segmentation training data. Gupta provided a way of generating an augmented
vector image by applying customization operations and executing the application
to perform vectorization. Therefore, it would have been obvious to one of
ordinary skill in the art before the effective filing date of the claimed invention was
made to incorporate vector-image customization operations taught by Gupta into
modified invention of Tremblay such that the system generates augmented
vector images prior to rasterization for use in generating rasterized images and
training sample masks. The motivation is to improve diversity and controllability
of generated training data discussed by Gupta in Paragraph [0047];
Regarding Claim 12, the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 11 .
The combination further teaches altering a parameter configuration of the
digital tool applied to a stylized geometry element in the labeled [[vector]] image.
(Tremblay, Paragraph [0042], "Note that at least a portion of the rendering
parameters vary for each one of the rendered images <read on altering a
parameter configuration>”); adjusting a hierarchical placement of the stylized
geometry element in the labeled [[vector]] image. (Tremblay, Paragraph [0044],
"A particular rendered object of interest may be occluded by one or more
other rendered objects of interest and/or rendered geometric shapes <read
on adjusting a hierarchical placement>") or inserting a stylized geometry
element from a different labeled [[vector]] image into the labeled vector image.
(Tremblay, Paragraph [0037], "a random number of rendered 3D geometric
shapes may be inserted into the input image <read on inserting a stylized
geometry element>")
But Tremblay does not explicitly disclose: variations are performed in the labeled vector image and/or that the inserted stylized geometry element is from a different labeled vector image.
However, Gupta teaches the above variations are performed in the
labeled vector image and/or that the inserted stylized geometry element is from a
different labeled vector image (Gupta, Paragraph [0047], "the vectorization
algorithm applies a set of customization operations <read on performing
the above variations in a vector image and inserting elements from other
vector images>")
Gupta and Tremblay are analogous since both are directed to systematic
variation of visual characteristics in images using parameterized operations
executed by a processing device. Tremblay provides a way of varying position,
size, color, rendering parameters, relative placement, and insertion of visual
elements during generation of rendered images for training purposes. Gupta
provides a way of applying customization operations and parameter
configurations directly to vector graphics during vectorization. Therefore, it would
have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention was made to incorporate vector-image customization
operations and parameter configurations taught by Gupta into the modified
invention of Tremblay, such that the variations of stylized geometry elements
(including position, size, color, parameter configuration, hierarchical placement,
and insertion) are performed at the vector-image level prior to rasterization,
rather than being limited to variations applied only during rendering. The
motivation is to enable controlled, repeatable, and parameter-driven variation of
stylized geometry elements using vector-level representations, as discussed by
Gupta in Paragraphs [004 7] and [0081], which improves flexibility and efficiency
in generating diverse training data consistent with Tremblay's image-generation
objectives.
Regarding Claim 15, the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 9.
The combination further teaches receiving, by the processing device, a classification network (Tremblay, Paragraph [0032], "During supervised
training of a neural network model <read on classification network>, the
task-specific training data is ground truth labels ... compared with an
output generated by the neural network model when the input image is
processed by the neural network model."); and generating an interactive image based on the input image ... using the segmentation network and the classification network (Tremblay, Paragraph [0049], "the training data computation unit 115 compares the output generated by the neural network model 260 when the input image is processed to the ground truth labels. <read on generating an output image based on processing the input image using the network>"); using the segmentation network (Tremblay, Paragraph [0024], "the task specific training data computation unit computes the task-specific training data as a segmentation map corresponding to the input image. <read on segmentation network>").
But, Tremblay does not explicitly disclose generating an interactive
image ... configured to display an image tool description that provides an
indication of the digital tool and a parameter configuration for the digital tool
used to achieve the visual appearance.
However, Gupta teaches generating an interactive image ... configured to
display an image tool description that provides an indication of the digital tool
(Gupta, Paragraph (0084], "the interface displays a tag 702 that provides
feedback data 240 ... the "sketch" label in the tag 702 indicates a set of
customization operations <read on digital tool> performed by the vectorization algorithm 400. <read on image tool description indicating the digital tool>"). and a parameter configuration for the digital tool used to achieve the visual
appearance (Gupta, Paragraph [0084], "The interface 700 also depicts an
edge-sensitivity box 708 and a corresponding slide tool 710 <read on
parameter configuration>.").
Gupta and Tremblay are analogous since both of them are dealing with
image-processing workflows that apply (and/or infer) operations/parameters and
present results to a user. Tremblay provided a way of processing an input image
using a neural network model (including segmentation-map related processing).
Gupta provided a way of displaying feedback data in a user interface, including
an indication of an operation (tag 702) and an adjustable parameter (edgesensitivity
708). Therefore, it would have been obvious to one of ordinary skill in
the art before the effective filing date of the claimed invention was made to
incorporate user-interface display of tool indication and parameter controls taught
by Gupta into modified invention of Tremblay such that the interactive image
generated from the segmentation/classification networks is configured to display
an image tool description indicating the digital tool and a parameter configuration.
The motivation is to provide feedback data and adjustable controls discussed by
Gupta in Paragraph [0084];
Regarding Claim 16, the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 9.
The combination further teaches wherein the training the segmentation
network includes predicting whether the digital tool was applied to each pixel of
the training sample mask using the labeled vector image as ground truth data.
(Tremblay, Paragraph [0032], "During supervised training of a neural
network model, the task-specific training data is ground truth labels ...
compared with an output generated by the neural network model when the
input image is processed by the neural network model <read on predicting
whether applied per pixel using ground truth>")
using the labeled vector image as ground truth data (Tremblay, Paragraph [0035], "each pixel within a rendered object of interest replaced with an object identifier <read on ground truth labels per pixel>").
Regarding Claim 17, Tremblay teaches a non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising (Tremblay, Paragraph [0005], “A method, computer readable medium, and system are disclosed for generating synthetic images for training a neural network model” [0061], “a program executed by the host processor encodes a command stream in a buffer that provides workloads to the PPU 300 for processing. A workload may comprise several instructions and data to be processed by those instruction”), receiving a labeled vector image [[having metadata that specifies use of a digital tool, the digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image]] (Tremblay, Paragraph [0020], "Training deep neural networks requires a large amount of labeled training data. A domain andomization technique to generate training data that is automatically labeled is described"; Paragraph [0034], "At step 135, the GPU 110 renders a 3D object of interest to produce a rendered image of the object of interest. The 3D object of interest is rendered according to the rendering parameters"), identifying one or more parameter configurations for the digital tool identified in the labeled vector image [[based on the metadata]] (Tremblay, Paragraph [0022], "The rendering parameters may specify a position and/or orientation of the object of interest in a 3D scene, a position and/or orientation of a virtual camera, one or more texture maps, one or more lights including color, type, intensity, position and/or orientation, and the like" <read on: one or more parameter configurations>), training a classification network based on the labeled vector image and the one or more parameter configurations, the classification network trained to ascertain a parameter configuration for the digital tool used to achieve the visual appearance in the input image (Tremblay, Paragraph [0004], "The generated training data may be used to train neural networks for object detection and segmentation labelling tasks"; Paragraph [0032], "During supervised training of a neural network model, the task-specific training data is ground truth labels … compared with an output generated by the neural network model when the input image is processed by the neural network model"), outputting the classification network (Tremblay, Paragraph [0052], "the loss function unit 270 generates updated weights to reduce differences between the task-specific training data and the output data. When the differences are reduced to a predetermined value, training is complete").
But Tremblay does not explicitly disclose having metadata that specifies use of a digital tool, the digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image, based on the metadata, and the classification network trained to ascertain a parameter configuration for the digital tool used to achieve the visual appearance in the input image.
However, Gupta teaches training a classification network based on the labeled vector image and the one or more parameter configurations, the classification network trained to ascertain a parameter configuration for the digital tool used to achieve the visual appearance in the input image (Gupta, Paragraph [0035], "executes the customization training module 228 to generate, train, or otherwise develop a customization-identification network 208" <read on: training a classification network]; Paragraph [0020, "a deep-learning network or other neural network model classifies an input image or other raster graphic into a set of classes … using a trained neural network to select multiple vectorization operations specific to the characteristics of the input graphic" <read on: classification network trained to ascertain a parameter configuration for the digital tool used to achieve the visual appearance in the input image]), outputting the classification network (Gupta, Paragraph [0037], "outputs the customization-identification network 208" <read on: outputting the classification network]).
Gupta and Tremblay are analogous since both are dealing with generating labeled training data and training neural network models to identify and ascertain digital tool operations and their parameter configurations applied to image content. Tremblay provided a way of training a network on a computer-readable medium using labeled training data with associated parameter configurations and outputting the trained network. Gupta provided a way of training a classification network to ascertain parameter configurations for a digital tool used to achieve a visual appearance in an input image, and outputting that trained classification network. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the classification network training and output capability taught by Gupta into the modified invention of Tremblay such that the computer-readable medium stores instructions to train and output a classification network that ascertains a parameter configuration for the digital tool used to achieve the visual appearance in the input image. The motivation is to enable ascertainment of specific digital tool parameter configurations from an input image, as discussed by Gupta in Paragraph [0035].
But the combination of Tremblay and Gupta does not explicitly disclose having metadata that specifies use of a digital tool, the digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image, and based on the metadata.
However, Yang teaches having metadata that specifies use of a digital tool, the digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image (Yang, Paragraph [0037], "Digital tools, as described herein, include tools such as the content manipulation application 110 that are used to perform a function or a workflow electronically … a creation tool, content editing tool, content publishing tool, content tracking tool … or any other tool that is used for creating, editing, managing, generating, tracking, consuming or performing any other function or workflow related to content" <read on: digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image]; Paragraph [0048, "An application program 108 is an example of a digital tool" <read on: computer-implemented mechanism]; Paragraph [0042, "In some embodiments, each asset includes metadata" <read on: labeled vector image having metadata]), based on the metadata (Yang, Paragraph [0083], "If the electronic document is a vector graphics document, the post-processing algorithm identifies the rendering commands used to generate different objects in the document. For instance, a set of rendering commands could indicate that a portion of the document belongs to a common object" <read on: identifying parameter configurations for the digital tool based on the metadata in the labeled vector image]; Paragraph [0089], "the training process uses rendering commands from a vector graphic document … rendering commands might indicate that two different document regions … belong to the same object … a common bounding box is applied to both document regions" <read on: identifying one or more parameter configurations for the digital tool identified in the labeled vector image based on the metadata]).
Yang and Tremblay as modified by Gupta are analogous since all three are dealing with leveraging structured information embedded in vector graphics documents or image data to identify digital tools and their parameter configurations for use in training a classification network. Tremblay as modified by Gupta provided a way of training and outputting a classification network that ascertains digital tool parameter configurations from labeled training data on a computer-readable medium. Yang provided a way of using metadata embedded in a labeled vector graphics document — specifically rendering commands identifying the digital tool and its parameter onfigurations — to supply those configurations to a neural network training workflow. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the metadata-bearing labeled vector image and the metadata-based identification of digital tool parameter configurations taught by Yang into the modified invention of Tremblay as further modified by Gupta, such that the computer-readable medium stores instructions to receive a labeled vector image having metadata specifying use of a digital tool configured as a computer-implemented mechanism executable for altering a visual appearance of an input image, identify one or more parameter configurations for the digital tool based on the metadata, train a classification network to ascertain a parameter configuration for the digital tool used to achieve the visual appearance in the input image, and output the trained classification network. The motivation is to leverage metadata already embedded in vector graphics documents to automatically identify digital tool parameter configurations for classification network training without manual annotation, as discussed by Yang in Paragraph [0089].
Regarding Claim 18, the combination of Tremblay and Gupta and Yang teaches the invention of generating the trained segmentation network and the trained classification network in Claim 17.
The combination further teaches generating a rasterized image (Tremblay,
Paragraph [0034], "At step 135, the GPU 110 renders a 30 object of interest to
produce a rendered image of the object of interest; [0114], “the graphics processing pipeline 600 comprises a pipeline architecture that includes a number of stages. The stages include…a rasterization stage 660; [0112], “After the processed vertex data is rasterized <read on rasterized image>"). and generating a training sample mask for the rasterized image (Tremblay, Paragraph [0035], "for segmentation, the task-specific training data is the rendered objects of interest having each pixel within a rendered object of interest replaced with an object identifier <read on training sample mask>").
However, Tremblay does not explicitly disclose: [[from the labeled vector image]].
But Gupta teaches generating a rasterized image from the labeled vector image (Gupta, Paragraph [0051], "At block 312, the process 300 involves causing a computing device (e.g., a user device 227) to display or otherwise provide an output vector graphic 226 <read on labeled vector image>").
As explained in rejection of Claim 17, the obviousness for combining Gupta's
output vector graphic into Tremblay's rasterization/training pipeline is provided above.
Regarding Claim 19, the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 17.
The combination further teaches wherein the labeled vector image further
specifies a parameter configuration for the digital tool used to stylize a geometry
element in the labeled vector image (Gupta, Paragraph [0084], "The interface
700 also depicts an edge-sensitivity box 708 and a corresponding slide tool
710 <read on parameter configuration>";Paragraph [0051], "provide an
output vector graphic 226 <read on labeled vector image>.").
Gupta and Tremblay are analogous since both of them are dealing with
digital graphics tools/operations that are parameterized and whose parameter
values may be associated with a graphics representation for downstream use.
Tremblay provided a way of training and outputting network(s) for use with
images. Gupta provided a way of presenting and selecting a parameter value
(edge-sensitivity) associated with a customization operation and providing an
output vector graphic. Therefore, it would have been obvious to one of ordinary
skill in the art before the effective filing date of the claimed invention was made to
incorporate associating a parameter configuration with a vector graphic taught by
Gupta into modified invention of Tremblay such that the labeled vector image
specifies the parameter configuration used to stylize a geometry element. The
motivation is to enable subsequent reuse and consistent reproduction of the
selected parameter setting discussed by Gupta in Paragraph [0084];
Regarding Claim 20, the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 17.
The combination further teaches receiving a segmentation network (Tremblay, Paragraph [0024], "the task-specific training data computation unit computes the task-specific training data as a segmentation map corresponding to the input image <read on segmentation network>."). and generating an interactive image based on the input image (Tremblay, Paragraph [0049], "the training data computation unit 115 compares the output generated by the neural network model 260 when the input image is processed to the ground truth labels <read on generating an output based on the input image>.").
But, Tremblay does not explicitly disclose: configured to display an image
tool description that provides an indication of the digital tool and a parameter
configuration for the digital tool used to achieve the visual appearance using the
segmentation network and the classification network.
However, Gupta teaches configured to display an image tool description
that provides an indication of the digital tool and a parameter configuration for the
digital tool used to achieve the visual appearance (Gupta, Paragraph [0084],
"the interface displays a tag 702 that provides feedback data 240 ... the
"sketch" label in the tag 702 indicates a set of customization operations
<read on digital tool> ... The interface 700 also depicts an edge-sensitivity
box 708 and a corresponding slide tool 710 <read on parameter
configuration>.").
Gupta and Tremblay are analogous since both of them are dealing with
using trained models in image-processing applications and presenting results to
a user via a user interface. Tremblay provided a way of processing an input
image using a neural network model and producing outputs (including
segmentation-map related outputs). Gupta provided a way of displaying feedback
data in a user interface, including an indication of an operation (tag 702) and an
adjustable parameter (edge-sensitivity 708). Therefore, it would have been
obvious to one of ordinary skill in the art before the effective filing date of the
claimed invention was made to incorporate user-interface feedback of identified
operations/parameters taught by Gupta into modified invention of Tremblay such
that an interactive image is generated using the segmentation network and the
classification network and is configured to display an image tool description
indicating the digital tool and parameter configuration. The motivation is to
provide feedback data to a user about identified operations and adjustable
parameters discussed by Gupta in Paragraph [0084];
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tremblay et al. (US 20190251397 A1, hereinafter Tremblay) in view of Gupta et al. (US 20190158112 A1, hereinafter Gupta), further in view of Yang et al. (US 20190026550 A1, hereinafter Yang). as applied to Claim 9 above and further in view of Carbonneau et al. (US 20130328879 A1, hereinafter Carbonneau).
Regarding Claim 13. the combination of Tremblay, Gupta and Yang teaches the
invention in Claim 9.
The combination further teaches generating a [[ cropped ]] labeled [[ vector ]] image (Tremblay, Paragraph [0004], "The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks.
wherein the segmentation network is trained using the [[ cropped ]] labeled [[ vector ]] image and the [[ cropped ]] rasterized image (Tremblay, Paragraph [0020], "A domain randomization technique to generate training data that is automatically labeled is described. The generated training data may be used to train neural networks for object detection and segmentation tasks"; [0112], “After the processed vertex data is rasterized…to produce fragment data).
But, Tremblay does not explicitly disclose the [[ labelled ]] vector [[ image ]].
However, Gupta teaches generating a labeled vector image and a rasterized image (Gupta, Paragraph [0051], "At block 312, the process 300 involves causing a computing device (e.g., a user device 227) to display or otherwise provide an output vector graphic 226 <read on labeled vector image>"; [0005], “The content-creation computing system provides the input raster graphic to a customization-identification network having a multi-label classifier”).
As explained in rejection of Claim 9, the obviousness for combining Gupta's
output vector graphic into Tremblay's rasterization/training pipeline is provided above.
But the combination does not explicitly disclose generating a cropped labeled vector image and a cropped rasterized image.
However, Carbonneau teaches generating a cropped labeled vector image (Carbonneau, Paragraph [0050], "Graphical images in a tile <read on cropped> are built up from individual pixels ... reduces the amount of data for a tile that could not be displayed separately on the scale of that tile"; [0051], "receiving at 310 a set of road vectors for a set of map tiles") and a cropped rasterized image (Carbonneau, Paragraph [0051], "The process 300 plots the vectors on the map tiles at a particular zoom level and rasterizes (at 320) the vectors. The rasterization identifies a set of pixels in the map tiles that the road vectors pass through") that depict a stylized geometry element of the labeled vector image
(Carbonneau, Paragraph [0050], "detail smaller than 100 meters wide ... multiple roads in the same direction"), and wherein the segmentation network is trained using the cropped labeled vector image and the cropped rasterized image (Carbonneau, Paragraph [0005], "generates an equivalent of the road data by 1 rasterizing the vectors representing road segments lying within a tile").
Carbonneau and Tremblay (as modified by Gupta) are analogous since
both of them are dealing with processing vector data and rasterizing it for
downstream use. Tremblay provided a way of rendering 30 vector objects into
2D raster images for training neural networks. Carbonneau provided a way of
handling large vector datasets (map data) by dividing them into tiles (crops) and
rasterizing vectors within those tiles to manage data volume and complexity.
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention was made to incorporate the tiling
and cropping approach for vector data taught by Carbonneau into modified
invention of Tremblay such that the training data generation process can handle
large, complex vector scenes more efficiently by processing them in smaller,
manageable cropped sections (tiles). The motivation is to reduce the
computational load and memory usage when processing large vector images by
breaking them down into smaller tiles, which allows for more efficient
rasterization and mask generation reduces the amount of data for a tile").
Regarding Claim 14. the combination of Tremblay, Gupta, Yang and Carbonneau teaches the invention in Claim 13.
The combination further teaches wherein the generating the training sample mask includes generating a mask including the stylized geometry element (Tremblay, Paragraph [0024], "computes the task-specific training data as a segmentation map ... where each pixel that is covered by a rendered image is colored according to the object identifier determined for the instance"), [[dilating]] the stylized geometry element (Tremblay, Paragraph [0024], "segmentation map comprises the input image, where each pixel that is covered by a rendered image is colored"), and rasterizing the [[ dilated ]] stylized geometry element against a [[ black backdrop ]] (Tremblay, Paragraph [0022], "rendered image of the 3D object").
But Tremblay does not explicitly disclose dilating the stylized geometry element.
However, Gupta teaches wherein the generating the training sample mask includes generating a mask including the stylized geometry element (Gupta, Paragraph [0005], " a content-creation computing system transforms an input raster graphic into a output vector graphic by applying a customization specific to visual characteristics of the input raster graphic"), [[ dilating ]] the stylized geometry element (Gupta, Paragraph [0003], “designer may want to convert raster graphics depicting hand-drawn sketches, hazy images, or oil paintings into vector graphics in order to apply more robust editing techniques available to vector graphics”), and rasterizing the [[dilated]] stylized geometry element against a [[ black backdrop ]] (Gupta, Paragraph [0022] , "The graphic manipulation application outputs a vector graphic generated by the vectorization pipelines").
Gupta and Tremblay are analogous since both of them are dealing with generating synthetic training data for neural networks. Tremblay provided a way of generating segmentation maps for training. Gupta provided a way of generating synthetic images with diverse visual properties. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the synthetic data generation techniques taught by Gupta into modified invention of Tremblay such that the system produces high-quality training data. The motivation is to enhance the robustness of the trained models.
But the combination does not explicitly disclose dilating the stylized geometry element.
However, Carbonneau teaches wherein the generating the training sample mask includes generating a mask including the stylized geometry element (Carbonneau, Paragraph [0005], "generating a connectivity mask for the tile(s )."), dilating the stylized geometry element (Carbonneau, Paragraph [0005], "The connectivity mask keeps track of which pixels are connected to which other pixels along the vectors"; [0050], “Similar problems arise for other relatively small features… Sending such redundant information would use up potentially expensive bandwidth…process that reduces the amount of data for a tile” [0052], “By converting the vectors to pixels in a connectivity mask, the process 300 weeds out redundant vectors and vector segments”), and rasterizing the dilated stylized geometry element against a black backdrop (Carbonneau, Paragraph [0051], “The rasterization identifies a set of pixels in the map tiles that the road vectors pass through” [0058], “Because in the tile, each pixel is the smallest measure of detail, each pixel is either on or off When no vectors (roads) pass through the pixel, the pixel is off”).
Carbonneau and Tremblay (as modified by Gupta) are analogous since both of them are dealing with converting vector data into rasterized formats for processing. Tremblay provided a way of creating segmentation masks for rendered objects. Carbonneau provided a way of creating connectivity masks from rasterized vectors to ensure data continuity and handle close vector segments. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the mask generation and pixel connectivity processing (which effectively dilates the element to ensure vector coverage) taught by Carbonneau into modified invention of Tremblay such that the generated training masks robustly capture the footprint of the vector elements, especially thin or complex ones, ensuring accurate training data. The motivation is to ensure that the rasterized representation accurately reflects the vector geometry and maintains connectivity between pixels representing the same element.
Response to Arguments
Applicant’s arguments with respect to claim 1, filed on 4/22/2026, with respect to rejection under 35 USC § 103 that neither Tremblay nor Gupta, alone or in combination, teaches or suggests receiving a labeled vector image that specifies use of a digital tool, on the grounds that: (a) Tremblay's 3D synthetic objects are not "vector images" as defined in the application; (b) Tremblay's rendering parameters are not "digital tools"; and (c) Gupta does not cure these deficiencies because Gupta's input is a raster graphic, not a labeled vector image. These arguments have been considered but are not found persuasive for the following reasons.
Applicant asserts that Tremblay's 3D synthetic objects are three-dimensional models rendered into two-dimensional images and are fundamentally different from vector images defined by mathematical paths and curves. The Examiner acknowledges that Tremblay does not explicitly disclose a labeled vector image as the training input. This gap is supplied by Gupta, not Tremblay, as expressly stated in the rejection above. Applicant's argument is attacking Tremblay alone on the "labeled vector image", however, the combination of prior art fully anticipate the limitations, Therefore, applicant remark cannot be considered persuasive.
Applicant further argues that Gupta's system receives raster graphics as input and converts them to vector graphics, and therefore Gupta does not disclose receiving a labeled vector image that specifies use of a digital tool.
In response to the argument, the rejection does not rely on Gupta's input raster graphic as the claimed labeled vector image. Rather, the rejection relies on Gupta's output vector graphic i.e. a vector image that encodes the visual characteristics and applied customization operations corresponding to the digital tool used which reading on the claimed "labeled vector image that specifies use of a digital tool." (Gupta, Paragraph [0051]; Paragraph [0084]). The combination rationale is that one of ordinary skill in the art would have used Gupta's output vector graphic 226, which encodes the applied digital operation as a labeled visual characteristic, as the labeled training input for Tremblay's network-training pipeline. Applicant's argument regarding Gupta's raster input does not address this aspect of the combination and is therefore not persuasive.
Applicant further asserts that Tremblay's rendering parameters are 3D scene configuration parameters and not digital tools such as gradients, brushes, or glows. Applicant further asserts that Gupta's customization operations (cleaning, blending, boosting, smoothing) are vectorization operations and do not specify the digital tool as recited. These arguments are not persuasive. As to Tremblay, the Examiner has revised the rejection such that the "digital tool" limitation is now primarily mapped to Gupta, not Tremblay. As to Gupta, Applicant's own specification defines "digital tool" broadly as "a computer-implemented mechanism for altering a visual appearance of image data, such as functionality supported by vector graphics editors, raster graphics editors, and other digital graphics editing software." (Application, Paragraph [0030].) Gupta's customization operations are computer-implemented mechanisms — executed by a graphic manipulation application (Gupta, Paragraphs [0046]–[0047]) — that alter the visual appearance of image data. They therefore fall squarely within the scope of "digital tool" as broadly defined in the subject application. The fact that Applicant's specification lists preferred examples such as gradients, drop shadow, glow, and brushes does not limit the claim to those specific examples, absent a clear and unmistakable disclaimer in the intrinsic record. Furthermore, Gupta's output vector graphic 226 encodes a label identifying the applied customization operation (e.g., the "sketch" tag (Gupta, Paragraph [0084])) along with associated parameter configurations (e.g., edge-sensitivity values (Gupta, Paragraph [0081])). This structure — a vector image bearing embedded identification of the applied operation and its parameters — directly reads on "a labeled vector image that specifies use of a digital tool." The distinction Applicant attempts to draw based on the specific type of operation (vectorization vs. graphic editing) is a distinction in application domain, not a structural or functional distinction that would preclude the prior art from reading on the claim under a proper claim construction. Therefore, applicant remark cannot be considered persuasive.
In regard to Claims 2-8, they directly/indirectly depends on independent Claim 1,. Applicant does not argue anything other than the independent claim 1. The limitations in those claims in conjunction with combination previously established as explained.
Applicant’s arguments with respect to claim 9, 17, filed on 4/22/2026, with respect to rejection under 35 USC § 103 have been considered but are moot in view of the new ground(s) of rejection. it is now been taught by the combination of prior arts Tremblay, Gupta and Yang.
In regard to Claims 10-16 and 18-20, they directly/indirectly depends on independent Claim 9, 17 respectively,. Applicant does not argue anything other than the independent claim 9, 17. The limitations in those claims in conjunction with combination previously established as explained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230206670 A1 SEMANTIC REPRESENTATION OF TEXT IN DOCUMENT
US 10810491 B1 Real-time visualization of machine learning models
US 20200019769 A1 MULTI-MODAL ELECTRONIC DOCUMENT CLASSIFICATION
US 20180322339 A1 PAGE SEGMENTATION OF VECTOR GRAPHICS DOCUMENTS
US 10002450 B2 Analyzing a document that includes a text-based visual representation
US 7702673 B2 System and methods for creation and use of a mixed media environment.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights