Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections
Claim 17 is objected to because of a minor informality. In the preceding amendment, a minor typographical error was introduced in the final limitation of this claim, which reads “stop stopping a display…” but was intended to simply read --stop a display…--. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10, 12-19, 21, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over McHugh et al. (U.S. Patent Application Publication No. 2019/0279407), referred herein as McHugh, in view of Yen (U.S. Patent Application Publication No. 2024/0177479), referred herein as Yen, and further in view of Storer (U.S. Patent Application Publication No. 2020/0314356), referred herein as Storer.
Regarding claim 1, McHugh teaches a method for controlling a display of an overlay in a video of a scene (figs 4, 6, 8, and 9; paragraph 94, lines 1-7; paragraph 95, lines 1-2), comprising:
defining the overlay to be displayed or stopped in the video, defining an overlay area including the overlay, and defining at least one foreground object of the scene in the video (figs 6A, B, and D, overlay 610, overlay area 605, foreground object 615; paragraph 96, lines 1-8 and the last 4 lines);
determining whether a spatial overlap is present in a video frame of the video between the foreground object and the overlay area such that a first portion of the overlay defined by the spatial overlap is blocked by the foreground object and a second portion of the overlay is not blocked by the foreground object, wherein the first portion and the second portion comprise an entirety of the overlay, and in response to determining the spatial overlap in the video frame, stopping a display of the entirety of the overlay within the overlay area, wherein said stopping results in pixels corresponding to the first portion and pixels corresponding to the second portion being transparent (fig 6D; paragraph 92, lines 1-7; paragraph 97, lines 1-5 and the last 14 lines; paragraph 108, lines 1-12; paragraph 109, lines 1-11; paragraphs 145 and 148; overlay 610 comprises a first portion blocked by the foreground object 615, and a second portion that is not blocked by the foreground object 615; when foreground object 615 spatially overlaps with overlay 610 in overlay area 605, the display of the entire overlay 610 is stopped, e.g. by fading out the overlay using a change in transparency/ alpha [which is consistent with the application’s specification and dependent claims 3 and 4, for example]).
McHugh teaches that the objects may become transparent (see, for example, paragraph 92, lines 1-7; paragraph 126, the last 12 lines; paragraph 127, lines 1-8), and also teaches fading objects out such that they become fully transparent at the edges (see, for example, paragraph 149, the last 8 lines). Additionally, as shown in figure 6D and its corresponding description, McHugh teaches that the first portion that is blocked by the foreground object is not visible. However, McHugh does not explicitly teach that the overlay becomes fully transparent or invisible.
However, in a similar field of endeavor, Yen teaches a method for controlling a display of an overlay in a scene, comprising defining an overlay in a video, defining an overlay area including the overlay, and defining at least one foreground object of the scene (figs 3A-3C and 3G; paragraph 31; paragraph 33, the last 8 lines; paragraph 48, the last 7 lines; the 3D letter overlays sitting atop the overlay area including the letter cards in the scene, and the car foreground object in the scene), and further comprising determining a spatial overlap occurring between the foreground object and the overlay area, and in response to the determination, a display of the entirety of the overlay stops, wherein said stopping results in pixels corresponding to the overlay being fully transparent or invisible (figs 3F-3I; paragraph 49, lines 1-7; when the determination result is positive, the foreground object emerges and overlaps a portion of the overlay area, and display of the 3D letter overlays then stops, resulting in pixels corresponding to the 3D letter overlays becoming fully transparent or invisible).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the overlay/foreground object interaction of Yen with that of McHugh, because this helps bring the foreground object into focus while temporarily removing pixel data related to the background overlay from having to be rendered, thereby improving the quality and focus of the object overlap display, while saving computational resources and avoiding a negative impact in the performance of the displaying (see, for example, Yen, paragraph 50).
McHugh in view of Yen teaches fading thresholds (McHugh, paragraphs 132 and 182) and modifying pixels values based on threshold distances (McHugh, paragraph 163), but does not explicitly teach determining a size of the spatial overlap and whether the size exceeds a first threshold, wherein the display of the overlay is changed in response to determining that the size of the spatial overlap exceeds the first threshold.
However, in a similar field of endeavor, Storer teaches a method for controlling the display of an overlay in a video of a scene, comprising determining a foreground object of the scene and a background, and adjusting alpha values accordingly (paragraph 12; paragraph 63; paragraphs 74 and 79), and further comprising determining a size of a spatial overlap and whether the size exceeds a first threshold, wherein a display of the overlay is changed in response to determining that the size of the spatial overlap exceeds the first threshold (paragraph 64; paragraph 74, lines 1-4 and the last 3 lines; paragraph 80, the last 18 lines; paragraph 81, lines 1-11; the overlap size can be determined by the number of pixels, a percentage amount of overlay, etc., and has a determined threshold; foreground objects are overlayed on the background in various ways that are based on the size determination).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the spatial overlap threshold of Storer with the image processing of McHugh in view of Yen because this provides a faster and more efficient way to evaluate the overlap and process the image accordingly, thereby increasing the efficiency of the processing while reducing bandwidth and storage requirements (see, for example, Storer, paragraph 14).
Regarding claim 2, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, wherein the size of the first threshold is related to a size of the overlay area (Storer, paragraph 64; paragraph 80, the last 18 lines; paragraph 81, lines 1-11; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 3, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, wherein stopping the display of the overlay includes changing an alpha value of pixels included in the overlay area (McHugh, paragraph 131, the last 6 lines; paragraph 132; paragraph 145; Storer, paragraph 79; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 4, McHugh in view of Yen, further in view of Storer teaches the method of claim 3, wherein the alpha value is changed at a pre-determined rate to achieve a fade-out effect (McHugh, paragraph 131, the last 6 lines; paragraph 132; paragraph 145; Storer, paragraph 79; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 5, McHugh in view of Yen, further in view of Storer teaches the method of claim 4, wherein the pre-determined rate is selectable by a user (McHugh, paragraphs 100 and 101; paragraph 132; paragraph 135, lines 16-31; Storer, paragraph 79; paragraph 80, the last 7 lines; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 6, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, further comprising: reintroducing the overlay when the size of the spatial overlap falls below a second threshold (McHugh, paragraph 93, the last 6 lines; paragraphs 112 and 145; Storer, paragraph 64; paragraph 80, the last 18 lines; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 7, McHugh in view of Yen, further in view of Storer teaches the method of claim 6, wherein the second threshold is the same as the first threshold (McHugh, paragraph 93, the last 6 lines; paragraphs 112 and 145; Storer, paragraph 64; paragraph 80, the last 18 lines; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 8, McHugh in view of Yen, further in view of Storer teaches the method of claim 6, further comprising starting a timer in response to the stopping of displaying the entire overlay within the overlay area, and wherein reintroducing the overlay occurs when the timer reaches a predetermined value (McHugh, paragraph 108, the last 16 lines; paragraphs 112 and 145; Storer, paragraph 62, lines 1-7; paragraph 63, the last 11 lines; paragraph 80, lines 5-20; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 9, McHugh in view of Yen, further in view of Storer teaches the method of claim 6, wherein reintroducing the overlay includes changing an alpha value of pixels included in the overlay (McHugh, paragraph 112; paragraph 131, the last 6 lines; paragraphs 132 and 145; Storer, paragraph 79; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 10, McHugh in view of Yen, further in view of Storer teaches the method of claim 9, wherein the alpha value is changed at a pre-determined rate to achieve a fade-in effect (McHugh, paragraph 91; paragraph 93, the last 6 lines; paragraph 97, the last 14 lines; paragraph 132; Storer, paragraph 79; paragraph 80, the last 12 lines; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 12, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, wherein determining the spatial overlap is done by determining a ratio between a number of pixels in the overlay area that belong to the foreground object and a total number of pixels in the overlay area (Storer, paragraph 64; paragraph 80, the last 18 lines; the percentage is a ratio; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 13, McHugh in view of Yen, further in view of Storer teaches the method of claim 12, wherein determining the spatial overlap further takes into account a position within the overlay area of the pixels belonging to the foreground object (McHugh, paragraph 163; Storer, paragraph 35, lines 1-4 and the last 9 lines; paragraphs 37 and 79; paragraph 80, the last 18 lines; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 14, McHugh in view of Yen, further in view of Storer teaches the method of any claim 1, wherein defining at least one foreground object in the video is done by using a motion-based foreground-background detection algorithm (Storer, paragraphs 35 & 72; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 15, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, wherein defining at least one foreground object in the video is done by using an object detection algorithm, and classifying pixels belonging to detected objects as foreground pixels and pixels not belonging to detected objects as background pixels (McHugh, paragraphs 71 and 100; Storer, paragraphs 37 and 72; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 16, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the medium, instructions, and computer, which are disclosed by McHugh, fig 2, medium 260 and its stored instructions, processor 240, and paragraphs 53 and 55); thus they are rejected on similar grounds.
Regarding claim 17, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the processor, memory, and instructions, which are disclosed by McHugh, fig 2, processor 240, memory 260 and its stored instructions, and paragraphs 53 and 55); thus they are rejected on similar grounds.
Regarding claim 18, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, wherein the first threshold comprises a percentage of pixels within the overlay area being covered by the foreground object (McHugh, paragraphs 150 and 152; Storer, paragraph 64, the last 3 lines; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 19, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, wherein the method comprises automatically setting the first threshold based on an appearance of the scene (Storer, paragraphs 64, 78, and 80; the threshold may be set based on positions, overlap, bounding boxes, etc. of objects in the scene; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Regarding claim 21, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, wherein the overlay is one of a plurality of overlays corresponding to a plurality of letters to enable a letter-by-letter display, further comprising: when a certain portion of a letter is covered by the foreground object, stopping display of said letter (McHugh, figs 5, 8C, and 8D; the letters in objects 515a, 515b, 808a, etc.; paragraph 91, lines 1-4 and the last 4 lines; paragraph 92, lines 1-7; paragraph 93; paragraph 129; paragraph 130, the last 7 lines; Yen, figs 3A-3C and 3G; paragraph 31; paragraph 33, the last 8 lines; paragraph 48, the last 7 lines; the 3D letter overlays sitting atop the letter cards in the scene, and the car foreground object; as shown in figs 3A-3C, each 3D overlay is one of a plurality of 3D overlays corresponding to a plurality of letters, and a letter-by-letter display is enabled; figs 3F-3I and paragraph 49, lines 1-7 then disclose that when the determination result is positive, the car foreground object emerges and covers a portion of one or more letters, and display of the 3D letter overlays then stops; the motivation to combine is similar to that discussed in the rejection of claim 1 ).
Regarding claim 22, McHugh in view of Yen, further in view of Storer teaches the method of claim 1, in response to determining that the size of the spatial overlap in the video frame does not exceed the first threshold, displaying the entire overlay within the overlay area (McHugh, paragraph 93, the last 6 lines; paragraphs 112 and 145; Storer, paragraph 64; paragraph 80, the last 18 lines; the motivation to combine is similar to that discussed above in the rejection of claim 1).
Response to Arguments
Applicant’s arguments with respect to the 112 rejection have been fully considered and are persuasive. The amendments have overcome this rejection; thus it is withdrawn.
Applicant’s arguments with respect to the 103 rejections have been fully considered, but are moot in view of new grounds of rejection presented above. As discussed in the rejection, although McHugh teaches that the entire overlay may become transparent when overlapped by the foreground object, and that edges may be fully transparent, the Examiner agrees that McHugh does not appear to explicitly teach that the entire overlay may be fully transparent or invisible. However, it is respectfully submitted that Yen teaches this limitation.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Chiba (U.S. Patent Application Publication No. 2013/0063736); Information processing apparatus, method, and program.
Zhang (U.S. Patent Application Publication No. 2024/0169541); Amodal instance segmentation using diffusion models.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights