Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-8, 11-15 and 18-20 are rejected under 35 USC 103 as being unpatentable over TANG et al. (US 20220129682) in view of TELESENS (Object Detection and Classification using R-CNNs, IDS dated 9/19/2024 ).
Regarding claims 1, 8 and 15, TANG disclose a computer implemented method, a computing device and a non-transitory computer-readable medium with instruction (TANG, Figs. 1-4) comprising:
detecting a bystander in an initial image (TANG Fig. 4 block 420, receiving the digital image, block 426 classifying the respective object mask for each human that is to be removed from the digital image versus the respective object mask for each human to be maintained in the digital image, paragraphs 0018 and 0096 identifying unwanted humans they are to be removed. In the system of TANG classifying the respective object mask for each human that is to be removed from the digital image and identifying unwanted humans they are to be removed obviously corresponds to detecting a bystander in an initial image);
generating a bystander box that includes the bystander, wherein all pixels for the
bystander are within the bystander box (TANG Figs. 4-5 paragraphs 0096-0097 TANG disclose The digital image 402 includes, as objects that may be identified, first human 408, second human 410, and third human 412. In the present example, the first human 408 is classified as a wanted human instance, meaning the first human 408 is to be maintained in the digital image. In some examples, the first human 408 is identified by the execution device 110 as a main character (main human instance) in the digital image 402. The first human 408 is centered in the digital image and is facing frontward. The second human 410 and the third human 412 are classified as unwanted human instances, meaning they are to be removed and paragraph 0098 disclose the image instance segmentation module 101A of the execution device 110 generates a bounding box (not shown here) for each respective human instance and a bounding box (not shown here) for each detected human head instance. In some examples, the execution device 110 generates a bounding box (not shown here) for the additional objects and the background objects of the digital image. Furthermore paragraphs 0096-0098 list of digital image of wanted/unwanted humans mask and other objects and generating bounding box for all humans (wanted/unwanted human which corresponds to bystander) and other objects and the bounding box of digital image of unwanted humans would obviously include all the pixel of unwanted humans (bystander). All this obviously corresponds to generating a bystander box that includes the bystander, wherein all pixels for the for the bystander are within the bystander box);
generating localizer boxes that encompass the bystander and one or more objects that are attached to the bystander (TANG Fig. 4 paragraph 0098 discloses the image instance segmentation module 101A of the execution device 110 generates a bounding box (not shown here) for each respective human instance (Fig. 4 wanted and unwanted humans) and a bounding box (not shown here) for each detected human head instance. In some examples, the execution device 110 generates a bounding box (not shown here) for the additional objects and the background objects of the digital image 402. It is obvious that some of the objects in the background are attached to unwanted humans such as chair, bicycle, walking stick or animals or anything attached/overlapping to unwanted humans and therefor this disclosure of TANG obviously corresponds to generating localizer boxes that encompass the bystander and one or more objects that are attached to the bystander);
aggregating the bystander box and one or more of the localizer boxes to form an
aggregated box (TANG Fig. 4 paragraph 0098 discloses the image instance segmentation module 101A of the execution device 110 generates a bounding box (not shown here) for each respective human instance (Fig. 4 wanted and unwanted humans) and a bounding box (not shown here) for each detected human head instance. In some examples, the execution device 110 generates a bounding box (not shown here) for the additional objects and the background objects of the digital image 402. It is obvious that some of the objects in the background are attached to unwanted humans such as chair, bicycle, walking stick or animals or anything attached/overlapping to unwanted humans and therefor this disclosure of TANG obviously corresponds to generating localizer boxes that encompass the bystander and one or more objects that are attached to the bystander. Furthermore it would be also obvious to aggregate or combining and clustering bounding box of unwanted humans and bounding box of objects overlapping the bounding box of unwanted humans using the convention technique known in the art of the object segmentation);
applying a segmenter to the initial image, based on the aggregated box, to segment the bystander and the one or more objects from the initial image to generate a
bystander mask, wherein the bystander mask includes a subset of pixels within the
aggregated box (TANG Fig. 4 paragraph 0098 discloses the image instance segmentation module 101A of the execution device 110 generates a bounding box (not shown here) for each respective human instance (Fig. 4 wanted and unwanted humans) and a bounding box (not shown here) for each detected human head instance. In some examples, the execution device 110 generates a bounding box (not shown here) for the additional objects and the background objects of the digital image 402. It is obvious that some of the objects in the background are attached to unwanted humans such as chair, bicycle, walking stick or animals or anything attached/overlapping to unwanted humans and therefor this disclosure of TANG obviously corresponds to generating localizer boxes that encompass the bystander and one or more objects that are attached to the bystander. Furthermore it would be also obvious to aggregate or combining and clustering bounding box of unwanted humans and bounding box of objects in the background and overlapping the bounding box of unwanted humans using the convention technique known in the art of the object segmentation. TANG Fig. 4 and paragraph 0097 discloses the image instance segmentation module 101A of the execution device 110 generates a mask 406 for each respective human instance, in this example first mask 414 for first human 408, second mask 416 for second human 410, and third mask 418 for third human 412. In some examples, the execution device 110 generates a mask for the additional objects and the background objects of the digital image 402. Therefore in the system of TANG it would be obvious to applying a segmenter to the initial image, based on the aggregated box of unwanted human and any other objects overlapping , to segment the bystander and the one or more objects from the initial image to generate a bystander mask, wherein the bystander mask includes a subset of pixels within the aggregated box); and
generating an in-painted image, wherein all pixels within the bystander mask are replaced in the in-painted image with pixels that match a background in the initial image (TANG Fig. 4, blocks 426-404, paragraph 0099 discloses the wanted/unwanted human classification module 101D of the execution device 110 classifies the human instances as being wanted versus unwanted. For the unwanted human instances, the wanted/unwanted human classification module 101D generates a list of the masks of the unwanted human instances, in this example the second mask 416 for second human 410, and third mask 418 for third human 412 and paragraph 100 TANG discloses The image inpainting module 101C of the execution device 110 received the list of masks of the unwanted human instances, and generates an in-painted digital image 404, which is inpainting the second mask 416 (of the second human 410) and the third mask 418 (of the third human 412) of the digital image 402. In example embodiments, the execution device 110 uses the background objects and the additional objects for the inpainting. As shown in FIG. 3, the first human 408 is maintained in the in-painted digital image. All this obviously corresponds to generating an in-painted image, wherein all pixels within the bystander mask are replaced in the in-painted image with pixels that match a background in the initial image).
In the same field of endeavor of objects detection and classification TELESENS pages 4-5 with figures discloses aggregating i.e. combining or clustering the bound boxes.
Therefore it would have been obvious to one of ordinary skill in the art, before the claimed invention was filed to aggregating i.e. combining or clustering the bounding boxes of unwanted humans and other objects in the background with are attached or overlapping to the unwanted human as shown by TELESENS in the system of TANG because such a system and process provides boundaries of the objects for classification and segmentation.
Regarding claims 4 11 and 18 TANG disclose applying segmenter a foreground from background to distinguish the one object box attached the bystander from the background (TANG Fig. 4 paragraph 0098 discloses the image instance segmentation module 101A of the execution device 110 generates a bounding box (not shown here) for each respective human instance (Fig. 4 wanted and unwanted humans) and a bounding box (not shown here) for each detected human head instance. In some examples, the execution device 110 generates a bounding box (not shown here) for the additional objects and the background objects of the digital image 402. It would be obvious that some of the objects in the background are attached to unwanted humans such as chair, bicycle, walking stick or animals or anything attached/overlapping to unwanted humans and therefor this disclosure of TANG obviously e corresponds applying segmenter a foreground from background to distinguish the one object box attached the bystander from the background because of the bounding box and the mask overlarlapping)
Regarding claims 5, 12 and 19 TANG disclose determine if a subject of the initial image occludes the bystander and generating a subject mask and update the bystander mask remove pixels are within the subject mask (TANG Fig. 4 paragraph 0098 discloses the image instance segmentation module 101A of the execution device 110 generates a bounding box (not shown here) for each respective human instance (Fig. 4 wanted and unwanted humans) and a bounding box (not shown here) for each detected human head instance. In some examples, the execution device 110 generates a bounding box (not shown here) for the additional objects and the background objects of the digital image 402. It would be obvious in the system of TANG to detect overlapping bounding boxes and mask of wanted/unwanted humans therefore it would be obvious to detect if the subject (wanted humans) human unwanted human based on the overlapping bounding box. TANG in paragraph 0100 discloses inpainting the bounding box of unwanted humans with the background i.e. removing the mask of the unwanted humans. It would be obvious to remove unwanted human mask which are overlapping with wanted humans mask thereby updated wanted human mask/bounding box. Therefore it would be obvious in the system of TANG to determine if a subject of the initial image occludes the bystander and generating a subject mask and update the bystander mask remove pixels are within the subject mask.
Regarding claims 6, 13 and 20 TANG disclose the segmenter is trained machine learning model (TANG Fig. 4-5, 8, paragraphs 0097-0099 and 0110 disclose segmentation of wanted/unwanted humans and background and Fig. 8 and paragraph 0110 disclose segmenter is machine learning model).
Regarding claims 7 and 14 disclose localizer box include one or more of a subject the bystander or objects in the initial image and removing corresponding localalizer boxes that are associated with the subject (TANG Fig. 4, blocks 426-404, paragraph 0099 discloses the wanted/unwanted human classification module 101D of the execution device 110 classifies the human instances as being wanted versus unwanted. For the unwanted human instances, the wanted/unwanted human classification module 101D generates a list of the masks of the unwanted human instances, in this example the second mask 416 for second human 410, and third mask 418 for third human 412 and paragraph 100 TANG discloses The image inpainting module 101C of the execution device 110 received the list of masks of the unwanted human instances, and generates an in-painted digital image 404, which is inpainting the second mask 416 (of the second human 410) and the third mask 418 (of the third human 412) of the digital image 402. In example embodiments, the execution device 110 uses the background objects and the additional objects for the inpainting. As shown in FIG. 3, the first human 408 is maintained in the in-painted digital image. In the system of TANG it is obvious that localizer box include one or more of a subject the bystander or objects in the initial image and it would be obvious to remove any bounding box unwanted object related with the subject).
Allowable Subject Matter
Claims 2-3, 9-10 and 16-17 are objected as being dependent on rejected base claim but would be allowable if rewritten in the independent including limitations of the base claim and any intervening claim.
Communication
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHRAT I SHERALI whose telephone number is (571)272-7398. The examiner can normally be reached Monday-Friday 8:00AM -5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ISHRAT I. SHERALI
Examiner
Art Unit 2667
/ISHRAT I SHERALI/Primary Examiner, Art Unit 2667