Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
INFORMATION DISCLOSURE STATEMENT
The information disclosure statement (IDS) submitted on 01/30/2025 & 12/09/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
ALLOWABLE SUBJECT MATTER
Claims 3, 9-10, 13-16 & 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
CLAIM REJECTIONS - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 6-8 & 11 are rejected under 35 U.S.C. 103 as being unpatentable over Kudelski (U.S. Publication 2022/0366544) in view of Tang (U.S. Publication 2024/0127455)
As to claims 1 & 11, Kudelski discloses a system comprising: one or more memory devices comprising a digital image ([0021, 0023]); and one or more processors coupled to the one or more memory devices that cause the system to perform operations comprising: determining a first bounding box indicating a first connected masked region in a base mask; determining a second bounding box indicating a second connected masked region in the base mask, the first connected masked region and the second connected masked region being separated in the base mask; generating a first region mask from the first bounding box and a second region mask from the second bounding box; ([0044-0045, 0049] & Fig. 5 discloses examples of masks composed of several mask regions. See corresponding disclosure discloses ‘A mask region’ should be understood as a set of pixels forming a continuous shape within the inpainting mask. [0044] discloses the quoted first and second connected region bounding box and region mask limitations. [0045] discloses wherein each mask region is first surrounded with minimal bounding box. As a result, several mask/image crops can be passed further to the machine leaning inpainting model. [0049] discloses the corresponding rectangular crop of the inpainting mask with the same coordinates as the image crop. )
Tang is silent to a base mask generated for the digital image utilizing a mask generation neural network; generating, utilizing a mask refinement neural network, a first refined region mask from the first region mask and a second refined region mask from the second region mask; and combining the first refined region mask and the second refined region mask into a final mask for the digital image.
However, Tang discloses a base mask generated for the digital image utilizing a mask generation neural network ([0024] discloses Mask R-CNN is a prevailing two stage method for instance segmentation [0031] discloses the instance mask 415 may be generated by a Mask R-CNN model commonly used for instance segmentation.); generating, utilizing a mask refinement neural network, a first refined region mask from the first region mask and a second refined region mask from the second region mask ([0045] discloses the extracted image patches 425a, 425b…425n and corresponding mask patches 430a, 430b…430n may be input to the semantic segmentation network 435 sequentially or in parallel. Refined mask patches 440a, 440b…440n are output by the semantic segmentation network 435); and combining the first refined region mask and the second refined region mask into a final mask for the digital image. ([0052] discloses the generating refined mask patches 440a, 440b…440n may be reassembled into the instance mask 415 to generate a refined instance mask 450. )
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kudelski’s disclosure to include the above limitations in order to automate production of the base mask and improve its boundary accuracy without altering Kudelski crop selection process.
As to claim 4, Kudelski in view of Tang discloses everything as disclosed in claim 1. In addition, Kudelski discloses to wherein determining the plurality of bounding boxes comprises: determining a first bounding box corresponding to a first set of connected pixels of a first masked region in the base mask; and determining a second bounding box corresponding to a second set of connected pixels of a second masked region in the base mask. ([0045, 0049])
As to claim 6, Kudelski in view of Tang discloses everything as disclosed in claim 4. In addition, Kudelski discloses wherein generating the plurality of separate region masks comprises: generating, from the base mask, a first region mask based on coordinates of the first bounding box; and generating, from the base mask, a second region mask based on coordinates of the second bounding box. ([0044, 0046, 0049])
As to claim 7, Kudelski in view of Tang discloses everything as disclosed in claim 6. In addition, Tang discloses wherein generating the plurality of refined region masks comprises generating, utilizing the mask refinement neural network, a first refined region mask from the first region mask and a second refined region mask from a second region mask corresponding to the second bounding box. ([0041, 0044-0045])
As to claim 8, Kudelski in view of Tang discloses everything as disclosed in claim 7. In addition, Tang discloses wherein combining the plurality of refined region masks comprises combining the first refined region mask, the second refined region mask, and a portion of the base mask outside boundaries of the first refined region mask and the second refined region mask to generate the final mask for the digital image. ([0052])
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Kudelski (U.S. Publication 2022/0366544) in view of Tang (U.S. Publication 2024/0127455) as applied in claim 1 above further in view of CHEN et al. (U.S. Publication 2025/0252731)
As to claim 2, Kudelski in view of Tang discloses everything as disclosed in claim 1 but is silent to determining, for the digital image, a plurality of base masks generated by a mask generation neural network, the plurality of base masks comprising the base mask; and selecting the base mask from the plurality of base masks in response to determining that a mask quality score of the base mask meets a score threshold.
However, CHEN’s discloses determining, for the digital image, a plurality of base masks generated by a mask generation neural network, the plurality of base masks comprising the base mask; and selecting the base mask from the plurality of base masks in response to determining that a mask quality score of the base mask meets a score threshold. ([0015-0016] discloses overall plural mask ranking and selection. [0018-0029] discloses plural masks, multiple detectors and common detector software with different settings. [0034-0042] discloses mask selector, quality scores and quality threshold. [0045-0066] discloses plural masks for the same image and individual quality scores. [0078-0084] discloses selecting the subset of masks according to quality score. [0099-0112] discloses generating masks assign quality scores, receiving the threshold, and selecting masks meeting the threshold. )
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kudelski in view of Tang’s disclosure to include the above limitations in order to prevent a low quality automatically generated mask from being propagated into connected region extraction and neural network refinement, thereby improving the quality of the final mask and reducing unnecessary processing of defective mask candidates.
Claims 5 & 12 are rejected under 35 U.S.C. 103 as being unpatentable over Kudelski (U.S. Publication 2022/0366544) in view of Tang (U.S. Publication 2024/0127455) as applied in claims 1 & 11 above further in view of Gao (U.S. Publication 2018/0047193)
As to claim 5, Kudelski in view of Tang discloses everything as disclosed in claim 4 but is silent to merging the first bounding box and the second bounding box into a merged bounding box of the plurality of bounding boxes in response to determining that a first area of the first bounding box and a second area of the second bounding box overlap; and generating a region mask from the merged bounding box including the second area of the first bounding box and the second area of the second bounding box.
However, Gao discloses merging the first bounding box and the second bounding box into a merged bounding box of the plurality of bounding boxes in response to determining that a first area of the first bounding box and a second area of the second bounding box overlap; and generating a region mask from the merged bounding box including the second area of the first bounding box and the second area of the second bounding box. ([0103-0109])
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kudelski in view of Tang’s disclosure to include the above limitations in order to consolidate overlapping masked regions into one crop, avoid duplicate crop coverage and prevent redundant neural network refinement of overlapping mask regions.
As to claim 12, Kudelski in view of Tang discloses everything as disclosed in claim 11. In addition, Gao discloses a mask region should be understood as a set of pixels forming a continuous shape within the inpainting mask. See masks containing several mask regions ([0045]). See wherein each mask region is first surrounded with a minimal bounding box ([0049])
Kudelski in view of Tang but is silent to each set of connected pixels being separated from other sets of connected pixels according to mask values.
However, Gao discloses determining sets of connected pixels in the base mask, each set of connected pixels being separated from other sets of connected pixels according to mask values. ([0074, 0082-0089])
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kudelski in view of Tang’s disclosure to include the above limitations in order to identify each continuous masked region deterministically according to the foreground and background values of the binary mask and to generate a separate bounding box for each resulting region.
Claims 17 & 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kudelski (U.S. Publication 2022/0366544) in view of Tang (U.S. Publication 2024/0127455) & Gao (U.S. Publication 2018/0047193)
As to claim 17, Kudelski discloses a non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising ([0023, 0069]): determining a plurality of bounding boxes indicating a plurality of separate connected masked regions corresponding to one or more objects in a base mask of a digital image ([0044-0045, 0049] & Fig. 5 discloses examples of masks composed of several mask regions. See corresponding disclosure discloses ‘A mask region’ should be understood as a set of pixels forming a continuous shape within the inpainting mask. [0044] discloses the quoted first and second connected region bounding box and region mask limitations. [0045] discloses wherein each mask region is first surrounded with minimal bounding box. As a result, several mask/image crops can be passed further to the machine leaning inpainting model. [0049] discloses the corresponding rectangular crop of the inpainting mask with the same coordinates as the image crop. ); generating a plurality of region masks based on a portion of the base mask corresponding to a boundary of the merged bounding box and a portion of the base mask corresponding to an additional bounding box of the plurality of bounding boxes; ([0044, 0049] discloses a mask crop should be understood here as the corresponding rectangular crop of the inpainting mask (with the same coordinate as the image crop). See wherein if possible, a single mask crop is always preferred. If it is impossible to surround mask regions with a single crop, mask regions close to each other are grouped together. As a result, several mask/image crops can be passed further to the machine learning inpainting model.)
Kudelski discloses generating, utilizing a mask refinement neural network, a first refined region mask from the first region mask and a second refined region mask from the second region mask; and combining the first refined region mask and the second refined region mask into a final mask for the digital image.
However, Tang discloses generating, utilizing a mask refinement neural network, a first refined region mask from the first region mask and a second refined region mask from the second region mask ([0045] discloses the extracted image patches 425a, 425b…425n and corresponding mask patches 430a, 430b…430n may be input to the semantic segmentation network 435 sequentially or in parallel. Refined mask patches 440a, 440b…440n are output by the semantic segmentation network 435); and combining the first refined region mask and the second refined region mask into a final mask for the digital image. ([0052] discloses the generating refined mask patches 440a, 440b…440n may be reassembled into the instance mask 415 to generate a refined instance mask 450. )
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kudelski’s disclosure to include the above limitations in order to automate production of the base mask and improve its boundary accuracy without altering Kudelski crop selection process.
Kudelski in view of Tang is silent to determining a merged bounding box by merging a subset of the plurality of bounding boxes based on a proximity of the plurality of bounding boxes and sizes of the plurality of bounding boxes.
However, Gao discloses determining a merged bounding box by merging a subset of the plurality of bounding boxes ([0174] discloses the process 1600 includes determining a candidate merged bounding box for a first bounding box and a second bounding box. The Candidate merged bounding box is the bounding box that would result should the first bounding box and the second bounding box be merged.) based on a proximity of the plurality of bounding boxes and sizes of the plurality of bounding boxes ([0175] discloses the process 1600 includes comparing the size of the candidate merged bounding box against a size threshold. The process 1600 includes determining to merge the first bounding box and the second bounding box based on the size of the candidate merged bounding box being less than the threshold. [0176-0177] discloses the width of the first bounding box, and the width of the second bounding box. [0180-0181] discloses height of the first bounding box, and a height of the second bounding box. As to claimed “based on a proximity of the plurality of bounding boxes”: [0178-0179, 0182] discloses the process 1700 includes determining a vertical distance between the first bounding box and second bounding box.).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kudelski in view of Tang’s disclosure to include the above limitations in order to reduce risk of incorrectly combining nearby but unrelated regions or creating an excessively large merged crop.
As to claim 20, Kudelski in view of Tang & Gao discloses everything as disclosed in claim 17. In addition, Tang discloses generating a plurality of refined region masks comprises: generating a first refined region mask for a first region mask corresponding to a first bounding box of the plurality of bounding boxes; and generating a second refined region mask for a second region mask corresponding to a second bounding box of the plurality of bounding boxes; and combining the plurality of refined region masks comprises combining the first refined region mask and the second refined region mask to generate the final mask. ([0041, 0045, 0052])
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Kudelski (U.S. Publication 2022/0366544) in view of Tang (U.S. Publication 2024/0127455) & Gao (U.S. Publication 2018/0047193) as applied in claim 1 above further in view of CHEN et al. (U.S. Publication 2025/0252731)
As to claim 18, Kudelski in view of Tang & Gao discloses everything as disclosed in claim 17 but is silent to determining, for the digital image, a plurality of base masks generated by a mask generation neural network, the plurality of base masks comprising the base mask; and selecting the base mask from the plurality of base masks in response to determining that a mask quality score of the base mask meets a score threshold.
However, CHEN’s discloses determining, for the digital image, a plurality of base masks generated by a mask generation neural network, the plurality of base masks comprising the base mask; and selecting the base mask from the plurality of base masks in response to determining that a mask quality score of the base mask meets a score threshold. ([0015-0016] discloses overall plural mask ranking and selection. [0018-0029] discloses plural masks, multiple detectors and common detector software with different settings. [0034-0042] discloses mask selector, quality scores and quality threshold. [0045-0066] discloses plural masks for the same image and individual quality scores. [0078-0084] discloses selecting the subset of masks according to quality score. [0099-0112] discloses generating masks assign quality scores, receiving the threshold, and selecting masks meeting the threshold. )
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kudelski in view of Tang & Gao’s disclosure to include the above limitations in order to prevent a low quality automatically generated mask from being propagated into connected region extraction and neural network refinement, thereby improving the quality of the final mask and reducing unnecessary processing of defective mask candidates.
CONCLUSION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stephen P Coleman whose telephone number is (571)270-5931. The examiner can normally be reached Monday-Thursday 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Stephen P. Coleman
Primary Examiner
Art Unit 2675
/STEPHEN P COLEMAN/Primary Examiner, Art Unit 2675
Prosecution Insights