Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 9, and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Grauman et al. US 2019/0355125 (hereinafter “Grauman”) in view of Kim et al. US 2022/0148284 (hereinafter “Kim”).
Regarding claim 1, Grauman discloses a method for image segmentation performed by at least one processor (see paragraph 0011, a method for segmenting generic objects in videos comprises processing by a processor)
PNG
media_image1.png
172
356
media_image1.png
Greyscale
the method comprising: acquiring a deep learning model trained through an image segmentation task (see figure 5, step 501 process an appearance stream of an image in a frame of the video using a first deep neural network, see paragraph 0070 the main idea Is to pre-train for object classification, then re-purpose the network to produce binary object segmentations, thus a deep learning model trained through an image segmentation task, see also paragraph 0041)
PNG
media_image2.png
430
704
media_image2.png
Greyscale
PNG
media_image3.png
184
348
media_image3.png
Greyscale
PNG
media_image4.png
108
348
media_image4.png
Greyscale
; extracting motion information associated with a current frame of a given image (see step 502 of figure 5, process a motion stream of an optical flow [motion information], see paragraph 0066)
PNG
media_image5.png
106
344
media_image5.png
Greyscale
; and performing image segmentation for the current frame by reflecting the extracted motion information into
PNG
media_image6.png
76
350
media_image6.png
Greyscale
Grauman does not explicitly disclose that the image segmentation reflects class specific feature maps.
Kim discloses a segmentation method that uses a current frame and an adjacent frame to determine a feature map based on temporal information between the frames [motion] and predicts a class of an object before performing segmentation of instances corresponding to object class (see paragraph 0005 and figure 1)
PNG
media_image7.png
178
342
media_image7.png
Greyscale
PNG
media_image8.png
546
496
media_image8.png
Greyscale
Grauman and Kim are analogous art because they are from the same field of endeavor of video segmentation.
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine Grauman and Kim to use class specific feature maps. The motivation would be to utilize the object class to improve the segmentation performance.
Regarding claim 2, Grauman discloses wherein the extracted motion information is not used in training of the deep learning model (see step 501 of figure 5, the extracted motion information is not used until step 502).
Regarding claim 4, Grauman discloses wherein the extracting the motion information comprises: determining a reference frame from among a plurality of frames included in the given image; and extracting the motion information associated with the current frame based on a difference between the current frame and the reference frame (see step 502 of figure 5 which takes the difference between frames, the first frame is interpreted as the reference frame).
Regarding claim 9, Kim discloses the extracted motion information includes motion information of a first object and motion information of a second object, the first object and the second object being within the current frame, and the performing the image segmentation comprises: reflecting the motion information of the first object into a feature map of a first class corresponding to the first object; and reflecting the motion information of the second object into a feature map of a second class corresponding to the second object (see figure 1 steps 130 and 140 in which each instance [separate object] has extracted features which are used to predict object class in step 140).
Claim 14 is similarly analyzed to claim 1.
Claim 15 is similarly analyzed to claim 1.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Grauman in view of Kim and further in view of Mittal US 11,450,104.
Regarding claim 3, as discussed above Grauman and Kim disclose the limitations of claim 1.
Grauman nor Kim do not explicitly disclose wherein the performing the image segmentation comprises performing the image segmentation for the current frame based on an amount of motion associated with the current frame being equal or greater than a threshold value, and wherein a result of image segmentation for a previous frame is used in performing the image segmentation for the current frame based on the amount of motion associated with the current frame being less than the threshold value.
Mittal discloses that new segmentation maps may be generated in response to the motion in a current frame exceeding a predetermined motion threshold (see col. 8 lines 11-26, here it is assumed that the former segmentation maps would be used if the threshold is not exceeded thus reading on the claim language)
PNG
media_image9.png
212
302
media_image9.png
Greyscale
Grauman, Kim, and Mittal are analogous art because they are from the same field of endeavor of video segmentation.
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to incorporate Mittal’s teaching of using a threshold to determine motion to determine whether to make a new segmentation map in view of a large amount of motion or maintain the previous mapping if it doesn’t surpass the threshold. The motivation would be to only do more processing when a threshold is surpassed thus improving the processing speed.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Grauman in view of Kim and further in view of Guruva Reddiar et al. US 2022/0109838 (hereinafter “Guruva Reddiar”).
Regarding claim 13, as discussed Grauman and Kim disclose the limitations of claim 1.
Grauman nor Kim do not explicitly disclose that the given image is an image of a user who participates in a video conference, and the method further comprises: applying a virtual background set by the user to the current frame using a result of the image segmentation for the current frame.
Guruva Reddiar discloses the missing limitation by disclosing a video frame segmenter circuitry that allows a user to select a virtual background for replacing detected background regions in a video conference (see figure 0004 and paragraph 0042)
PNG
media_image10.png
416
552
media_image10.png
Greyscale
PNG
media_image11.png
212
314
media_image11.png
Greyscale
Grauman, Kim, and Guruva Reddiar are analogous art because they are from the same field of endeavor of video segmentation.
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to use the segmentation technique as taught by Grauman and Kim for the application of allowing a user to select the background in a video conference. The motivation would be to permit a user to make changes to their video conference background.
Allowable Subject Matter
Claims 5-8, 10-12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN B STREGE whose telephone number is (571)272-7457. The examiner can normally be reached M-F 9-5 (PST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN B STREGE/Primary Examiner, Art Unit 2669