Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/17/2026 has been entered.
Response to amendments
Submission dated 01/23/2026 amends claims 15, 22 and 25-27, and adds claims 30-32. Claims 1-14 were previously cancelled. Claims 15-32 are pending.
In view of the amendment to claims 22 and 25, the previously set forth objections of claims 22 and 25 are withdrawn.
Response to arguments
Applicant’s arguments with respect to claim(s) 15, 26 and 27 and their dependents have been considered but are moot because the applicant’s arguments are directed to how the previously set forth references applied to the amended claim language of the independent claims while the examiner is instead applying a new reference to the amended claim language.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention
Claim(s) 15, 16, 19-21, 23, 24, 26, 27, and 30-32 is/are rejected under 35 U.S.C. 103 as being unpatentable over Us patent application publication no. 2021/0295525 to Price et al. (hereinafter Price) in view of us patent application publication no. 2003/0202698 to Simard et al. (hereinafter Simard).
For claim 15, Price as applied discloses a method for instance segmentation, comprising the following steps:
receiving an image and an instance mask identifying an instance in the image (see, e.g., pars. 44-45 and FIG. 1, which teach receiving a digital image depicting objects and a mask describing the objects depicted in the image);
extracting a set of image patches from the image based on a boundary of the instance mask (see, e.g., pars. 52-55 and FIGS. 2 and 3A, which teach sampling the image along the curve that is mapped to the contour of the object in the mask, wherein the sampling including extracting strips of pixels that are withing a defined threshold of the points along the curve);
generating, for each image patch of the set of image patches, a respective refined mask patch based on at least a part of the instance mask corresponding to the image patch (see, e.g., pars. 58-60 and FIGS. 2 and 3B, which teach generating the boundary data from the strip image, i.e., the collection of extracted pixel strips, wherein the boundary data includes the boundary coordinates/locations and representation; the examiner interprets the boundary data as the claimed refined mask patches because it represents a collection of respective mask strips for the collection of the pixel strips), and
refining the boundary of the instance mask based on the respective refined mask patch for each of the set of image patches (see, e.g., pars. 60-61 and FIGS. 2 and 3C, which teach generating, from the boundary data, a strip recovery that represents a refined boundary of the mask).
While Price as applied teaches generating a refined mask for each image patch, it does not explicitly teach that each refined mask patch assigns a binary foreground-or-background classification to pixels of the image patch. Simard in the analogous art teaches generating a retouched mask altering a binary foreground/background classification of pixels in the binary mask (see, e.g., pars. 41-42, 46 and 48-49 and FIGS. 7 and 9 of Simard).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Price to generate the refined masks as taught by Simard because doing would reduce the effect of the fake boundary/artifact (see pars. 12, 14 and 38 of Simard).
For claim 26, Price as applied discloses an apparatus for instance segmentation, comprising:
a memory (see, e.g., pars. 102-105 and FIG. 8); and
at least one processor coupled to the memory and configured for instance segmentation (see, e.g., pars. 102-104 and FIG. 8), the at least one processor configured to:
receive an image and an instance mask identifying an instance in the image (44-45and FIGS. 1, which teach receiving a digital image depicting objects and a mask describing the objects depicted in the image);
extract a set of image patches from the image based on a boundary of the instance mask (see, e.g., pars. 52-55and FIGS. 2 and 3A, which teach sampling the image along the curve that is mapped to the contour of the object in the mask, wherein the sampling including extracting strips of pixels that are withing a defined threshold of the points along the curve),
generate, for each image patch of the set of image patches, a respective refined mask patch based on at least a part of the instance mask corresponding to the image patch (see, e.g., pars. 58-60 and FIGS. 2 and 3B, which teach generating the boundary data from the strip image, i.e., the collection of extracted pixel strips, wherein the boundary data includes the boundary coordinates/locations and representation; the examiner interprets the boundary data as the claimed refined mask patches because it represents a collection of respective mask strips for the collection of the pixel strips), and
refine the boundary of the instance mask based on the respective refined mask patch for each of the set of image patches (see, e.g., pars. 60-61 and FIGS. 2 and 3C, which teach generating, from the boundary data, a strip recovery that represents a refined boundary of the mask).
While Price as applied teaches generating a refined mask for each image patch, it does not explicitly teach that each refined mask patch assigns a binary foreground-or-background classification to pixels of the image patch. Simard in the analogous art teaches generating a retouched mask altering a binary foreground/background classification of pixels in the binary mask (see, e.g., pars. 41-42, 46 and 48-49 and FIGS. 7 and 9 of Simard).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Price to generate the refined masks as taught by Simard because doing would reduce the effect of the fake boundary/artifact (see pars. 12, 14 and 38 of Simard).
For claim 27, Price as applied discloses a non-transitory computer readable medium (pars. 100-109 and FIG. 8) on which is stored computer code for instance segmentation, the computer code when executed by a processor (see, e.g., pars. 99-102 and FIG. 8), causing the processor to perform the following steps:
receiving an image and an instance mask identifying an instance in the image (see, e.g., pars. 44-45 and FIG. 1, which teach receiving a digital image depicting objects and a mask describing the objects depicted in the image);
extracting a set of image patches from the image based on a boundary of the instance mask (see, e.g., pars. 52-55 and FIGS. 2 and 3A, which teach sampling the image along the curve that is mapped to the contour of the object in the mask, wherein the sampling including extracting strips of pixels that are withing a defined threshold of the points along the curve);
generating, for each image patch of the set of image patches, a respective refined mask patch based on at least a part of the instance mask corresponding to the image patch (see, e.g., pars. 58-60 and FIGS. 2 and 3B, which teach generating the boundary data from the strip image, i.e., the collection of extracted pixel strips, wherein the boundary data includes the boundary coordinates/locations and representation; the examiner interprets the boundary data as the claimed refined mask patches because it represents a collection of respective mask strips for the collection of the pixel strips), and
refining the boundary of the instance mask based on the respective refined mask patch for each of the set of image patches (see, e.g., pars. 60-61 and FIGS. 2 and 3C, which teach generating, from the boundary data, a strip recovery that represents a refined boundary of the mask).
While Price as applied teaches generating a refined mask for each image patch, it does not explicitly teach that each refined mask patch assigns a binary foreground-or-background classification to pixels of the image patch. Simard in the analogous art teaches generating a retouched mask altering a binary foreground/background classification of pixels in the binary mask (see, e.g., pars. 41-42, 46 and 48-49 and FIGS. 7 and 9 of Simard).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Price to generate the refined masks as taught by Simard because doing would reduce the effect of the fake boundary/artifact (see pars. 12, 14 and 38 of Simard).
For claim 16, Price in view of Simard teaches that a center of an image patch in the set of image covers the boundary of the instance mask (see, e.g., pars. 52-54 and FIGS. 2 and 3A, which teach sampling the 5image along the curve that is mapped to the contour of the object in the mask, wherein the sampling including extracting strips of pixels that are withing a defined threshold of the points along the curve; the examiner interprets the strips to be centered on the points along the curve).
For claim 19, Price in view of Simard teaches:
extracting a set of mask patches from the instance mask based on the boundary of the instance mask (see, e.g., pars. 51-52, which teach receiving curve data describing the upsampled curve; the examiner interprets the received curve data as the claimed set of mask patches), each of the set of mask patches covering a corresponding image patch of the set of image patches (see, e.g., pars. 51-52, which teach receiving curve data describing the upsampled curve; the examiner interprets each point in the received curve data as the claimed mask patch because each point centers a corresponding strip of pixels);
wherein the generating of the respective refined mask patch for each of the set of image patches is based on a corresponding mask patch of the set of mask patches (see, e.g., pars. 58-60 and FIGS. 2 and 3B, which teach generating boundary data for the strip image, i.e., the collection of extracted pixel strips, wherein the boundary data is based on the curve data).
For claim 20, Price in view of Simard teaches that each of the set of mask patches provides context information for a corresponding image patch, the context information indicating location and semantic information of the instance in the corresponding image patch (see, e.g., pars. 28, 30, 34, and 60, which teach that curve data provides the pixels to extracted along the curve, wherein the extracted pixels have corresponding locations and semantic contents of the image).
For claim 21, Price in view of Simard teaches:
performing binary segmentation on each of the set of image patches through a semantic segmentation network (see, e.g., par. 59 and FIGS. 2 and 3B, which teach generating the boundary data, which is in black and white and segmenting the object from the background).
For claim 23, Price in view of Simard teaches that each of the set of image patches is resized to match an input size of the semantic segmentation network (see, e.g., pars. 57-58 and FIGS. 2 and 3B, which teach resizing the sampled pixels to generate a strip image, which is inputted into the segmentation network).
For claim 24, Price in view of Simard teaches that the generating of the refined mask patch for each of the set of image patches is further based on at least a part of a second instance mask identifying a second instance adjacent to the instance in the image (see, e.g., par. 62, which teach predicting boundaries for objects with inner and out contours, such as a donut shaped object).
For claims 30-32, while Price as applied does not explicitly teach, Simard in the analogous art teaches that each refined mask patch has the same spatial dimensions as the corresponding image patch, and assigns a binary foreground-or-background classification to each pixel of the image patch (see, e.g., pars. 41-42, 46 and 48-49 and FIGS. 7 and 9 of Simard, which teach generating retouched masks that are not dimensionally modified from the input binary mask and each pixel of the retouched mask is assigned to either foreground or background).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Price to generate the refined masks as taught by Simard because doing would reduce the effect of the fake boundary/artifact (see pars. 12, 14 and 38 of Simard).
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Price in view of Simard and further in view of us patent application publication no. 2013/0121577 to Wang et al.
For claim 17, while Price in view of Simard teaches obtaining image patches along the contour of the mask, it does not explicitly teach obtaining a plurality of image patches from the image by sliding a window along the boundary of the instance mask; and filtering out the set of image patches from the plurality of images patches based on an overlapping threshold.
Wang in the analogous art teaches obtaining a plurality of image patches from the image by sliding a window along the boundary (see, e.g., pars. 79-86 and FIG. 12 of Wang, which teach obtaining image patches by sliding windows along the contour); and
filtering out the set of image patches from the plurality of images patches based on an overlapping threshold (see, e.g., par. 86 and FIG. 12 of Wang, which teach centering each window at equally spaced sample points along the contour and also setting a number of windows such that each point on the contour is covered by at least two windows).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Price in view of Simard to use sliding windows as taught by Wang because doing would yield a predictable results of having a consistent sampling along the contour (see MPEP 2143(I)(D)).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Price in view of Simard and Wang and further in view of us patent application publication no. 2021/0319327 to Poirier et al. (hereinafter Poirier).
For claim 18, while Price in view of Wang does not explicitly teach, Poirier in the analogous art teaches that the filtering out the set of image patches is based on a non-maximum suppression (NMS) algorithm, and the overlapping threshold is an NMS eliminating threshold (see, e.g., par. 35 of Poirier, which teach filtering out windows by applying NMR algorithm).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Price in view of Simard and Wang to use NMS for filtering out the windows as taught by Poirier because doing would yield a predictable results of filtering out multiple windows with the same image pixels (see par. 35 of Poirier and MPEP 2143(I)(D)).
Claim(s) 28-29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Price in view of Simard and further in view of us patent application publication no. 2017/0287137 to Lin et al. (hereinafter Lin).
For claim 28, while Price in view of Simard teaches refining of the boundary of the instance mask (see, e.g., pars. 60-61 and FIG. 3C, which teach generating the strip recovery), it does not explicitly teach reassembling the respective refined mask patches into the instance mask. Lin in the analogous art teaches reassembling pixels in the segmentation mask by correcting/replacing false pixel identifications of the segmentation mask around an edge of the object over multiple iterations (see, e.g., pars. 61-70 and FIG. 3 of Lin).
It would have been obvious to one of ordinary skill in the art to modify Price in view of Simard to reassemble/reintegrate the refined patches into the instance mask as Lin reassembles/reincorporates the correctly identified pixels to its segmentation mask because doing so would allow instance mask to fit precisely to the edges of the object (see e.g., pars. 61 and 63 of Lin).
For claim 29, while Price in view of Simard does not explicitly teach, Lin the analogous art teaches reassembling the respective refined mask patches into the instance mask by replacing a previous prediction for each pixel in the patches while pixels without refinement remain unchanged (see, e.g., pars. 61-70 and FIG. 3 of Lin teaches reassembling pixels in the segmentation mask by correcting/replacing false pixel identifications of the segmentation mask around an edge of the object over multiple iterations; the examiner interprets the correctly identified pixels as those pixels without refinement and remain unchanged).
It would have been obvious to one of ordinary skill in the art to modify Price in view of Simard to reassemble/reintegrate the refined patches into the instance mask as Lin reassembles/reincorporates the correctly identified pixels to its segmentation mask because doing so would allow instance mask to fit precisely to the edges of the object (see e.g., pars. 61 and 63 of Lin).
Allowable Subject Matter
Claims 22 and 25 are allowed.
In regard to claim 22, when considered as a whole, prior art of record fails to disclose or render obvious, alone or in combination:
“A method for instance segmentation, comprising the following steps:
…
generating a respective refined mask patch for each of the set of image patches based on at least a part of the instance mask corresponding to the each of the set of image patches;
…
the semantic segmentation network has one or more channels for an image patch, one channel for a mask patch, and 2 classes of output.”
In regard to claim 25, when considered as a whole, prior art of record fails to disclose or render obvious, alone or in combination:
“A method for instance segmentation, comprising the following steps:
…
refining the boundary of the instance mask based on the respective refined mask patch for each of the set of image patches;
the refining of the boundary of the instance mask includes:
averaging values of overlapping pixels in the refined mask patches for adjacent image patches in the set of image patches; and
determining whether a corresponding pixel in the instance mask identifies the instance based on a comparison between the averaged values and a threshold.”
Additional Citations
The following table lists several references that are relevant to the subject matter claimed and disclosed in this Application. The references are not relied on by the Examiner, but are provided to assist the Applicant in responding to this Office action.
Citation
Relevance
Paradkar et al. (us pat. pub. 2017/0091948)
Describes a method, a computer readable medium, and a system for cell segmentation. The method including generating a binary mask from an input image of a plurality of cells, wherein the binary mask separates foreground cells from a background; classifying each of the cell regions of the binary mask into single cell regions, small cluster regions, and large cluster regions; performing, on each of the small cluster regions, a segmentation based on a contour shape of the small cluster region; performing, on each of the large cluster regions, a segmentation based on a texture in the large cluster regions; and outputting an image with cell boundaries.
Table 1
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Table 1 and form 892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WOO RHIM whose telephone number is (571)272-6560. The examiner can normally be reached Mon - Fri 9:30 am - 6:00 pm et.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WOO C RHIM/Examiner, Art Unit 2676
Prosecution Insights