Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Response to Arguments
Applicant’s arguments, see pg. 15, filed 02/25/2026, with respect to the rejection(s) of claim(s) 1 under 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Ricci.
Allowable Subject Matter
Claims 7-8, and 16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Pub No. US 8385634 B1) in view of Ricci (Pub No. US 10748650 B1) and further in view of Blackburne (Pub No. US 20230360282 A1).
As per claim 1, Wang teaches the claimed:
A computer-implemented method comprising: determining, by at least one processor and based on mask inclusivity attributes of a raster mask or a vector mask, (Wang: col. 1 lines 23-58: “Digital images may include raster graphics, vector graphics, or a combination thereof. Vector graphics data may be stored and manipulated as one or more geometric objects built with geometric primitives. The geometric primitives (e.g., points, lines, polygons, Bezier curves, and text characters) may be based upon mathematical equations to represent parts of digital images. Raster graphics data (also referred to herein as bitmaps) may be stored and manipulated as a grid of individual picture elements called pixels. A bitmap may be characterized by its width and height in pixels and also by the number of bits per pixel. Commonly, a color bitmap defined in the RGB (red, green blue) color space may comprise between one and eight bits per pixel for each of the red, green, and blue channels. An alpha channel may be used to store additional data such as per-pixel transparency values. Many digital image editing operations may be applied selectively to a portion of the digital image. In selecting a portion of the digital image, a mask may be used to define a portion of a digital image on which an operation is sought to be performed. A hard mask may represent a binary, "all or nothing" inclusion or exclusion of pixels. A soft mask may comprise a mask having intermediate values which lie between the minimum and maximum values for membership in the mask. For example, a soft mask may potentially comprise integer values between 0 and 255 or floating-point values between 0 and 1. Soft masks may be used for gradual blending of selected pixels into surrounding regions of the digital image. Suitable operations may be applied to modify a mask. For example, various filters (e.g., Gaussian blur, median filter, add noise, reduce noise, fragment, unsharp mask), image adjustments (e.g., levels, curves, brightness/contrast, shadow/highlight), and other operations (e.g., resizing, cropping, thresholding, rotation, perspective distortion) may be applied to masks.” This is done on a processor. Wang col. 10 lines 24-40: “FIG. 12 is a block diagram illustrating constituent elements of a computer system 900 that is configured to implement embodiments of the systems and methods described herein. The computer system 900 may include one or more processors 910 implemented using any desired architecture or chip set, such as the SPARC.TM. architecture, an x86-compatible architecture from Intel Corporation or Advanced Micro Devices, or an other architecture or chipset capable of processing data. Any desired operating system(s) may be run on the computer system 900, such as various versions of Unix, Linux, Windows.RTM. from Microsoft Corporation, MacOS.RTM. from Apple Inc., or any other operating system that enables the operation of software on a hardware platform. The processor(s) 910 may be coupled to one or more of the other illustrated components, such as a memory 920, by at least one communications bus.”).
Wang alone does not explicitly teach the remaining claim limitations.
However, Wang in combination with Ricci and Blackburne teaches the claimed:
one or more area bounding boxes corresponding to one or more hidden areas or one or more visible areas of a layer of a digital image, the one or more area bounding boxes being different from mask boundaries of the according to a raster mask or a vector mask applied to the layer; (Ricci teaches masks separate from bounding boxes, as different options. Ricci col. 18 lines 30-40: “(64) Furthermore, instance segmentation process 1000 may be performed with overlapping objects, multiple overlapping objects, different backgrounds, a Region of Interest Align (ROI Align), a class awareness, an instance awareness, anchor boxes, ground truth boxes, object confidence scores and binary masks generated for individual and/or multiple objects. Instance segmentation may be processed by Region proposed networks (RPN), Featured Pyramid Networks (FPN) and Fully Convolutional Networks (FCN). Delineated dental image objects may also be processed in color, gray scale and black and white resolutions.” These are treated as different options. The bounding boxes are the area bounding boxes that are separate from the masks. Ricci col. 19 line 38-col. 20 line 5: “FIG. 12 shows a display diagram (process 1200) illustrating a dental object tracking mechanism 1205. The aggregator server 104 may use a computer vision component 214 and a machine learning mechanism 216 to execute the aggregator service 106 and process the dental image 1208 for e-commerce with a dental object tracking mechanism 1205. Dental image 1208 of an e-commerce consumer 133 may be processed with a dental object tracking mechanism 1205. Object tracking may use pooling layers, fully connected layers, multiple resolutions, multiple grid components, bounding boxes, a classified image score, an object classification, a semantic segmentation, a instance segmentation, anchor boxes, ground truth boxes, dental image landmark probabilities, image class landmark probabilities, object class landmark probabilities, spatial landmark probability relationships, object probability landmarks, object probability relationships, and dental image landmark probability maps. Further, an object tracked image may be a frame, an image, a layer, a slice and/or a section. A first time interval, dental image 1208 of an e-commerce consumer may be processed and compared with a second time interval dental image 1210 of an e-commerce consumer 133. The second dental image 1210 of an e-commerce consumer 133 may be compared with a third time interval dental image 1215 of an e-commerce consumer.” Ricci fig. 11 shows multiple different object identification methods in the same image. Ricci teaches objects being tracked based on different layers and can be used for bounding boxes. This can be combined with the layers for images taught by Blackburne. Blackburne [0034]: “As also used herein, the term “video processing data” refers to data representing properties of a video. In particular, the term “video processing data” can refer to data representing properties or characteristics of one or more objects depicted within a video. For example, video processing data can include face tracking (or face recognition) data that indicates features and/or attributes of one or more faces depicted within a video (e.g., vectors and/or points that represent a structure of a depicted face, bounding box data to localize a depicted face, pixel coordinates of a depicted face). In addition, video processing data can include segmentation data that indicates salient objects, background pixels and/or foreground pixels, and/or mask data that utilize binary (or intensity values) per pixel to represent various layers of video frames (e.g., to distinguish or focus on objects depicted in a frame, such as hair, persons, faces, and/or eyes).”).
determining, by the at least one processor, display attributes for the one or more bounding boxes in response to determining that the one or more area bounding boxes correspond to the one or more hidden areas or the one or more visible areas; (Ricci col.19 line 38-col. 20 line 5: “(67) FIG. 12 shows a display diagram (process 1200) illustrating a dental object tracking mechanism 1205. The aggregator server 104 may use a computer vision component 214 and a machine learning mechanism 216 to execute the aggregator service 106 and process the dental image 1208 for e-commerce with a dental object tracking mechanism 1205. Dental image 1208 of an e-commerce consumer 133 may be processed with a dental object tracking mechanism 1205. Object tracking may use pooling layers, fully connected layers, multiple resolutions, multiple grid components, bounding boxes, a classified image score, an object classification, a semantic segmentation, a instance segmentation, anchor boxes, ground truth boxes, dental image landmark probabilities, image class landmark probabilities, object class landmark probabilities, spatial landmark probability relationships, object probability landmarks, object probability relationships, and dental image landmark probability maps. Further, an object tracked image may be a frame, an image, a layer, a slice and/or a section. A first time interval, dental image 1208 of an e-commerce consumer may be processed and compared with a second time interval dental image 1210 of an e-commerce consumer 133. The second dental image 1210 of an e-commerce consumer 133 may be compared with a third time interval dental image 1215 of an e-commerce consumer.”).
and generating, for display with the layer within a graphical user interface, one or more boundary highlights representing the one or more area bounding boxes with the display attributes. (Blackburne teaches outputting the results of the mask segmentation in an interface. Blackburne [0019]: “To illustrate, in one or more embodiments, the shared AR scene video call system enables a client device to initiate an AR scene (or space) during a video call. Furthermore, upon initiating the AR scene during the video call, the shared AR scene video call system enables client devices participating in the video call to render an AR scene within a video call interface that portrays a three-dimensional (or two-dimensional) graphical scene instead of presenting only captured videos between the client devices. In addition, the shared AR scene video call system enables the client devices to receive video processing data (with video data) from other participant client devices and to utilize the video processing data to render videos of participants as video textures within AR effects in the AR scene. Indeed, the shared AR scene video call system can cause the client device to present a video call as an AR scene in which the participants of the video call are portrayed to be within the AR scene (as the video textures) instead of simply presenting captured videos between the client devices.” The claimed feature is taught when the GUI with boundary highlights from Blackburne is used with Ricci, which teaches area bounding boxes that are separate from masks. The area bounding boxes are the area bounding boxes and are separate from the masks.).
one or more bounding boxes corresponding to one or more hidden areas or one or more visible areas of a layer of a digital image according to a raster mask or the vector mask applied to the layer (Wang teaches that images may include raster masking. Wang col. 1 lines 23-45: “Digital images may include raster graphics, vector graphics, or a combination thereof. Vector graphics data may be stored and manipulated as one or more geometric objects built with geometric primitives…
(7) Many digital image editing operations may be applied selectively to a portion of the digital image. In selecting a portion of the digital image, a mask may be used to define a portion of a digital image on which an operation is sought to be performed.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to the bounding boxes that are separate from masks and can show different images layers as taught by Ricci with the system of Wang in order to allow bounding boxes to be used for different areas that don’t exactly correspond to masks but still allow identification of different segments and different layers of the image.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to the output on a graphical interface of a segmented layered image as taught by Blackburne with the system of Wang in order to show the results of the segmenting and bounding of masks and areas.
As per claim 17 and 20, these claims are similar in scope to limitations recited in claim 1, and thus are rejected under the same rationale.
As per claim 2, Wang teaches the claimed:
2. The computer-implemented method of claim 1, wherein determining the one or more area bounding boxes comprises:
Determining the mask inclusivity attributes indicating an inclusive mode or an exclusive mode for the raster mask or the vector mask; (Wang teaches digital images that have vector or raster graphics or a combination of the two in the same image. They are defined by masks. Wang col 1 line 22-col. 2 line 10: “(6) Digital images may include raster graphics, vector graphics, or a combination thereof. Vector graphics data may be stored and manipulated as one or more geometric objects built with geometric primitives. The geometric primitives (e.g., points, lines, polygons, Bezier curves, and text characters) may be based upon mathematical equations to represent parts of digital images. Raster graphics data (also referred to herein as bitmaps) may be stored and manipulated as a grid of individual picture elements called pixels. A bitmap may be characterized by its width and height in pixels and also by the number of bits per pixel. Commonly, a color bitmap defined in the RGB (red, green blue) color space may comprise between one and eight bits per pixel for each of the red, green, and blue channels. An alpha channel may be used to store additional data such as per-pixel transparency values.
(7) Many digital image editing operations may be applied selectively to a portion of the digital image. In selecting a portion of the digital image, a mask may be used to define a portion of a digital image on which an operation is sought to be performed. A hard mask may represent a binary, "all or nothing" inclusion or exclusion of pixels. A soft mask may comprise a mask having intermediate values which lie between the minimum and maximum values for membership in the mask. For example, a soft mask may potentially comprise integer values between 0 and 255 or floating-point values between 0 and 1. Soft masks may be used for gradual blending of selected pixels into surrounding regions of the digital image. Suitable operations may be applied to modify a mask. …” Wang describes the mask and whether it is inclusive or exclusive and shows values that show the inclusiveness or exclusiveness of the mask. It is represented as a number. This is the inclusivity factor: Wang Col. 4 lines 45-70: “(21) The output mask 135 may represent particular regions or objects in the image 110. The output mask 135 may include various elements (e.g., pixels) that are contiguous or not contiguous. In one embodiment, the output mask 135 may be a soft mask that includes one or more pixels having intermediate values. The range of values in the soft mask may be represented as values in an alpha channel (e.g., alpha values). For example, if full exclusion from the output mask 135 is indicated by a value of zero, and if full inclusion in the output mask 135 is indicated by a value of one, then a range of intermediate values between zero and one (e.g., 0.5) may indicate partial or "soft" inclusion in the output mask 135. Alternatively, integer values in an appropriate range (e.g., 0 and 255) may be used. This partial inclusion of some pixels may be used for transparency effects, feathering effects, blending effects, etc. FIG. 4 illustrates an example of an output mask 135A generated in accordance with one embodiment…”).
and determining the mask boundaries for one or more regions of the layer indicated by the raster mask or the vector mask. (The boundary determination of mask edges can be used for an image separated into layers, like those in Blackburne).
Claims 4, 6, and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ricci and further in view of Blackburne and further in view of Ping (Ping Ding and Yan Song, "Robust object tracking using color and depth images with a depth based occlusion handling and recovery," 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Zhangjiajie, 2015, pp. 930-935, doi: 10.1109/FSKD.2015.7382068).
As per claim 4, Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Ping teaches the claimed:
4. The computer-implemented method of claim 1, wherein determining the display attributes for the one or more area bounding boxes comprises determining one or more color values of the one or more area bounding boxes based on whether the one or more area bounding boxes correspond to the one or more hidden areas or the one or more visible areas of the layer. (Ping Fig 6 shows color coded bounding boxes for objects, and it shows the color changing with an object is fully concealed. The concealed object could be in a different layer of the image than the object occluding it.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes with different colors based on hidden or visible areas as taught by Ping with the system of Wang in order to highlight masks of objects that contain visible or hidden areas.
As per claim 6, Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Blackburne and Ping teaches the claimed:
6. The computer-implemented method of claim 5, wherein generating the one or more boundary highlights comprises:
generating, for display with the layer within the graphical user interface, a first boundary highlight representing the first bounding box with a first size and a first color value corresponding to the visible area of the layer;
and generating, for display with the layer within the graphical user interface, a second boundary highlight representing the second bounding box with a second size and a second color value corresponding to the hidden area of the layer. (Ping fig. 6 shows bounding boxes of different colors and size based on visibility.
This can be combined with images separated into layers like those of Blackburne, as some layers may overlap each other. Blackburne fig. 6 shows the boundaries of a portion of an image being determined to convert it to a mask. This is the boundary highlight. Blackburne [0034] also teaches identifying objects for masking with bounding boxes, as explained above.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the different colors and sized of bounding boxes based on the visibility of a layer as taught by Ping with the system of Wang in order to in order to show which layers contain objects that are hidden.
As per claim 13, Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Ping and Blackburne teaches the claimed:
13. The system of claim 9, wherein determining the one or more area bounding boxes comprises: determining a first bounding box corresponding to a visible area of the layer based on the mask boundaries for the raster mask or the vector mask, the layer inclusivity factor of the layer, and the layer boundaries of the layer;
and determining a second bounding box corresponding to a hidden area of the layer based on the mask boundaries for the raster mask or the vector mask, the layer inclusivity factor of the layer, and the layer boundaries of the layer. (Ping teaches determining bounding boxes for hidden areas of the image in fig. 6. Blackburne
Finally, Wang teaches the inclusivity of the masks, including the combinations of vector and raster masks, as described above. This could be applied to the different masks in different layers of the image of Blackburne).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes indicating hidden and visible areas of an image as taught by Ping with the system of Wang and Blackburne in order to accurately output bounding boxes around the masks of images concealed by images in other layers of Blackburne.
As per claim 14, Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Ping teaches the claimed:
14. The system of claim 13, wherein generating the one or more boundary highlights comprises: generating a first boundary highlight corresponding to the first bounding box and having a first set of display attributes in response to the first bounding box corresponding to the visible area of the layer;
and determining a second boundary highlight corresponding to the second bounding box and having a second set of display attributes in response to the second bounding box corresponding to the hidden area of the layer. (Ping fig. 6 shows different color bounding boxes for visible objects and hidden objects. These different colors are the first and second boundary highlights. These can be combined with the different layers of images for masking as taught in Blackburne above.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the different color boundary highlights as taught by Ping with the system of Wang and Blackburne in order to show which of the layers decomposed by Blackburne cover each other by being in layers nearer the viewer.
As per claim 15, this claim is similar in scope to limitations recited in claim 4, and thus are rejected under the same rationale.
Claims 3, 5, 9-12, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ricci and further in view of Blackburne and further in view of Ping and further in view of Zitnick (Pub No. US 7630541 B2).
As per claim 3, Wang teaches the claimed:
3. The computer-implemented method of claim 1, wherein determining the one or more area bounding boxes comprises:
determining first mask inclusivity attributes and first mask boundaries for the raster mask; determining second mask inclusivity attributes and second mask boundaries for the vector mask; (Wang teaches determining inclusivity attributes for both a vector and raster mask, which would requiring knowing the boundaries of the mask to analyze. Wang col. Lines 22-45: “Digital images may include raster graphics, vector graphics, or a combination thereof. Vector graphics data may be stored and manipulated as one or more geometric objects built with geometric primitives. The geometric primitives (e.g., points, lines, polygons, Bezier curves, and text characters) may be based upon mathematical equations to represent parts of digital images. Raster graphics data (also referred to herein as bitmaps) may be stored and manipulated as a grid of individual picture elements called pixels. A bitmap may be characterized by its width and height in pixels and also by the number of bits per pixel. Commonly, a color bitmap defined in the RGB (red, green blue) color space may comprise between one and eight bits per pixel for each of the red, green, and blue channels. An alpha channel may be used to store additional data such as per-pixel transparency values.
(7) Many digital image editing operations may be applied selectively to a portion of the digital image. In selecting a portion of the digital image, a mask may be used to define a portion of a digital image on which an operation is sought to be performed. A hard mask may represent a binary, "all or nothing" inclusion or exclusion of pixels.”).
Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Blackburne, Ping, and Nitnick teaches the claimed:
and determining the one or more area bounding boxes corresponding to the one or more hidden areas or the one or more visible areas of the layer based on the first mask inclusivity attributes, the first mask boundaries, the second mask inclusivity attributes, and the second mask boundaries. (Ping fig. 6 shows bounding boxes around objects based on their visibility. To do so, the objects can be identified with masks. It would be obvious to highlight the positions of the masks with bounding boxes. Additionally, as describes above, Zitnick teaches using bounding boxes to indicate masks and their characteristics, which could include inclusivity.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes of portions of an image based on visibility as taught by Ping with the system of Wang in order to break down the objects in the layers and mask them to determine which portions of an image are in which layers.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes to indicate characteristics of a masks as taught by Nitnick with the system of Wang in order to show which masks shown for different objects in Wang are inclusive or exclusive.
As per claims 18 and 19, these claims are similar in scope to limitations recited in claim 3, and thus are rejected under the same rationale. Claim 3 describes inclusivity attributes and boundaries for each of the raster or vector images. Claims 18-19 describe both. Claim 3 describes determining bounding boxes based on both visible and hidden areas, while claim 18 and 19 describe visible and hidden areas respectively.
As per claim 5, Wang teaches the claimed:
5. The computer-implemented method of claim 1, wherein determining the one or more bounding boxes comprises:
determining a layer inclusivity factor of the layer based on mask inclusivity attributes of the raster mask or the vector mask; (The inclusivity of the masks it measures as a factor as taught by Wang col. 4: “The output mask 135 may represent particular regions or objects in the image 110. The output mask 135 may include various elements (e.g., pixels) that are contiguous or not contiguous. In one embodiment, the output mask 135 may be a soft mask that includes one or more pixels having intermediate values. The range of values in the soft mask may be represented as values in an alpha channel (e.g., alpha values). For example, if full exclusion from the output mask 135 is indicated by a value of zero, and if full inclusion in the output mask 135 is indicated by a value of one, then a range of intermediate values between zero and one (e.g., 0.5) may indicate partial or "soft" inclusion in the output mask 135. Alternatively, integer values in an appropriate range (e.g., 0 and 255) may be used.” This can be used for either raster or vector masks.).
Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Ping and Zitnick teaches the claimed:
determining a first bounding box corresponding to a visible area of the layer based on the layer inclusivity factor of the layer and the mask inclusivity attributes of the raster mask or the vector mask; (Ping Ding fig. 6 shows bounding boxes around visible or hidden areas. Additionally, Nitnick teaches bounding boxes to indicate attributes of a mask. It would be obvious to a bounding box to indicate inclusivity of a mask.).
and determining a second bounding box corresponding to a hidden area of the layer based on the layer inclusivity factor of the layer and the mask inclusivity attributes of the raster mask or the vector mask. (The bounding boxes are based on visibility or hidden as shown in Ping fig. 6.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes to indicate characteristics of a masks as taught by Nitnick with the system of Wang in order to show which masks shown for different objects in Wang are inclusive or exclusive.
As per claim 9, Wang alone does not explicitly teach all the claimed limitations.
However, Wang in combination with Blackburne teaches the claimed:
9. A system comprising: one or more memory devices; and one or more processors coupled to the one or more memory devices that cause the system to perform operations comprising: (Segmentation of images requires a memory and processor to perform, like the taught by Wang in the rejection to claim 1.).
determining, by at least one processor, mask inclusivity attributes and mask boundaries for a raster mask or a vector mask applied to a layer of a digital image; (Wang teaches inclusivity attributes of either raster or vector masks, as well as combinations of them, as described above in the rejection for claim 2. The mask boundaries are necessary to analyze the mask inclusivity attributes.).
determining, by the at least one processor, a layer inclusivity factor of the layer based on the mask inclusivity attributes of the raster mask or the vector mask; (Blackburne in figs. 6-7 show images with layers being segmented and masked. This would determine which objects in which layers. Because there might be multiple masks in a layer, and because Wang teaches finding the inclusivity for a combination of masks, one could determine the inclusivity of a layer by combining these two.).
Blackburne alone does not explicitly teach the claimed limitations.
However, Blackburne in combination with Ping and Zitnick teaches the claimed:
determining, by the at least one processor, one or more area bounding boxes different from the mask boundaries and corresponding to one or more hidden areas of the layer or one or more visible areas of the layer according to the mask boundaries of the raster mask or the vector mask, the layer inclusivity factor of the layer, and layer boundaries of the layer; (Ping fig. 6 depicts different bounding boxes based on visible or hidden areas of an image. This could include layered images of Blackburne. Blackburne [0034] teaches using bounding boxes to identify portions of the objects to be masked. These are the layer boundaries.
Additionally bounding boxes could be used to indicate the nature of masks. Zitnick teaches using bounding boxes for identifying masks before performing operations on the masks. The bounding boxes are related to characteristics of the masks. Zitnick col. 16 lines 5-15: “(93) The task of performing local image operations is traditionally broken into two parts: Defining the area, called a mask, where the operation is to be performed, followed by performing the operation within the specified area. The decoupling of finding the mask and performing the operation is necessary since the actions required for finding a mask, such as marking mask boundaries, drawing bounding boxes, or marking inside and outside regions, differs from that of the manipulation.”
This could include the inclusivity and exclusivity attributes of the masks. Since there are multiple masks indicated in a layer, the box could indicate the inclusivity of the layer).
and generating, by the at least one processor and for display with the layer within a graphical user interface, one or more boundary highlights representing the one or more area bounding boxes corresponding to the one or more hidden areas of the layer or the one or more visible areas of the layer. (Blackburne teaches identifying the locations of features with bounding boxes and represented by layers. Blackburne [0034]: “As also used herein, the term “video processing data” refers to data representing properties of a video. In particular, the term “video processing data” can refer to data representing properties or characteristics of one or more objects depicted within a video. For example, video processing data can include face tracking (or face recognition) data that indicates features and/or attributes of one or more faces depicted within a video (e.g., vectors and/or points that represent a structure of a depicted face, bounding box data to localize a depicted face, pixel coordinates of a depicted face). In addition, video processing data can include segmentation data that indicates salient objects, background pixels and/or foreground pixels, and/or mask data that utilize binary (or intensity values) per pixel to represent various layers of video frames (e.g., to distinguish or focus on objects depicted in a frame, such as hair, persons, faces, and/or eyes).”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the segmentation of objects into different layers taught by Blackburne with the system of Wang in order to find inclusivity and exclusivity attributes of masks for different layers of an image or portions that might be hidden.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes of different types as taught by Ping with the system of Wang modified by Blackburne in order to indicate different portions of the images in a layer are visible or hidden.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes to indicate characteristics of a masks as taught by Zitnick with the system of Wang in order to show more clearly which masks shown for different objects in Wang are inclusive or exclusive.
As per claim 10, Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Blackburne and Ping teaches the claimed:
10. The system of claim 9, wherein determining the mask inclusivity attributes and the mask boundaries for the raster mask or the vector mask comprises:
determining first mask inclusivity attributes and first mask boundaries for the raster mask corresponding to the layer, the first mask inclusivity attributes indicating that the raster mask is inclusive or exclusive; (The inclusivity attributes are taught by Wang for both vector and raster masks. The inclusivity factor indicates whether the mask is inclusive or exclusive, as taught above by Wang. The bounding boxes taught by Ping in fig. 6 can be combined with the masks and what they include and the layers in which they are present. This would show the boundaries of the masks.).
and determining second mask inclusivity attributes and second mask boundaries for the vector mask corresponding to the layer, the second mask inclusivity attributes indicating that the vector mask is inclusive or exclusive. (The same features apply to the vector mask as do the raster masks.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the different colored bounding boxes for different portions of the image in different layers as taught by Ping with the system of Wang in order to indicate the inclusivity of the masks whose boundaries and layer membership that are indicated by Blackburne.
As per claim 11, Wang teaches the claimed:
11. The system of claim 10, wherein determining the layer inclusivity factor comprises determining the layer inclusivity factor of the layer as exclusive or inclusive based on a combination of the first mask inclusivity attributes of the raster mask and the second mask inclusivity attributes of the vector mask. (Wang teaches that an image can have a combination of raster and vector graphics and that they can be used for masks with inclusivity information. Wang col. 1 lines 22-50 “Digital images may include raster graphics, vector graphics, or a combination thereof. Vector graphics data may be stored and manipulated as one or more geometric objects built with geometric primitives. The geometric primitives (e.g., points, lines, polygons, Bezier curves, and text characters) may be based upon mathematical equations to represent parts of digital images. … Many digital image editing operations may be applied selectively to a portion of the digital image. In selecting a portion of the digital image, a mask may be used to define a portion of a digital image on which an operation is sought to be performed. A hard mask may represent a binary, "all or nothing" inclusion or exclusion of pixels.” Thus, inclusivity information could be obtained for the combined graphic with raster and vector masks.).
As per claim 12, Wang alone does not explicitly teach the claimed limitations.
However, Wang in combination with Ricci and Zitnick teaches the claimed:
12. The system of claim 9, wherein determining the one or more area bounding boxes comprises determining the one or more area bounding boxes further based on the mask inclusivity attributes of the raster mask or the vector mask in connection with the mask boundaries of the raster mask or the vector mask. (Zitnick teaches using bounding boxes for identifying masks before performing operations on the masks. The bounding boxes are related to characteristics of the masks. Zitnick col. 16 lines 5-15: “(93) The task of performing local image operations is traditionally broken into two parts: Defining the area, called a mask, where the operation is to be performed, followed by performing the operation within the specified area. The decoupling of finding the mask and performing the operation is necessary since the actions required for finding a mask, such as marking mask boundaries, drawing bounding boxes, or marking inside and outside regions, differs from that of the manipulation.” This would be combined with the inclusivity and exclusivity attributes of the masks as taught by Wang.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bounding boxes to identify certain masks as taught by Nitnick with the system of Wang in order to indicate visually different masks that have different characteristics as taught by Wang.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS JOHN FOSTER whose telephone number is (571)272-5053. The examiner can normally be reached Mon, Fri 8:30-6. Tues-Thurs 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS JOHN FOSTER/ Examiner, Art Unit 2616
/DANIEL F HAJNIK/ Supervisory Patent Examiner, Art Unit 2616
Prosecution Insights