Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to communication filed 10/25/2024. Claims 1-20 are pending for examination, the rejection cited as stated below.
Claim Rejections - 35 USC § 112
2. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
3. The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
4. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 1 recites “approximate first density of an underlying distribution of contrast differences” the scope of which cannot be definitely determined, lacking a readily ascertainable criterion in claim or in specification to determine “approximate first density of”. Applicant is required to clarify. For the sake of the examination, Examiner presumes that the claimed limitation reads “an underlying distribution of contrast differences”. Claims 2-20 are similarly rejected.
5. Claims 5, 11 and 19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 5 recites “the method of claim 4, wherein the blending factor is triggered based on the sum of contrast difference scores exceeding a selected threshold.” The recited “the blending factor” lacks sufficient antecedent basis, because neither claim 5 nor its parent claim 4 or grandparent claim 1 recites a blending factor. Applicant is required to clarify. For the sake of the examination, Examiner assumes anything being triggered. Claims 11 and 19 are similarly rejected.
6. Claims 3, 10 and 17-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 3 recites “the method of claim 2, further comprising: increasing, by the graphics computing system and based on the blending factor associated with the each individual one of the image pixels and the text pixels of the first layer, an opacity of the image pixels and the text pixels of the graphical overlay, within the video frame, to increase a readability of the text based on the brightness and contrast contributions of the blending factor for the image pixels and the text pixels.” It is unclear how the recited “to increase a readability of the text based on the brightness and contrast contributions of the blending factor for the image pixels and the text pixels” further limits the preceding limitation “increasing, by the graphics computing system and based on the blending factor associated with the each individual one of the image pixels and the text pixels of the first layer, an opacity of the image pixels and the text pixels of the graphical overlay, within the video frame.” Applicant is required to clarify. For the sake of the examination, Examiner interprets “to increase a readability of the text based on the brightness and contrast contributions of the blending factor for the image pixels and the text pixels” as an intended use which does not further limit the preceding limitation. Claims 10 and 17-19 are similarly rejected.
Claim Rejections - 35 USC § 103
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1, 4-5, 8, 11-12, and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta et al (US 2023/0007360) in view of Eschbach (US 2006/0126932) and further in view of Hoffman et al (US 20200193652).
As to claim 1, Gupta discloses a method for processing a video frame comprising:
determining, by a graphics computing system, a first graphic rendering area and position within the video frame of a graphical overlay located in a first layer of the video frame ([0012], “relocating an overlay to avoid overlapping information displayed in video content. As an overlay is rendered and positioned, there can be a check to make sure the overlay does not overlap an information box such as a sports score, stock ticker, news banner, or other information. If an overlap is determined, the overlay can be moved and rendered in a different position”; [0015], “An overlay engine may render different versions of text and/or graphics of an overlay such as a score board, score ticker, leaderboard, news banner, and/or stock ticker”. See Figures 1-3, wherein the overlays are in a first layer of the video frame. See Fig. 5A, “Receive an overlay image (e.g., network logo, caption, subtitle, etc.) and a second screen position. Does the second screen position overlap the first screen position”, indicates determining a first graphic rendering area and position within the video frame of a graphical overlay. See also [0094]-[0095]), wherein the graphical overlay comprises image pixels and text pixels ([0015], “An overlay engine may render different versions of text and/or graphics of an overlay such as a score board, score ticker, leaderboard, news banner, and/or stock ticker based on the size and resolution of a client device”; [0006], “A screen may be considered a two-dimensional coordinate plane, measured in pixels”);
calculating, by the graphics computing system, a location of video sub-blocks located in a second layer of the video frame (Figure 5A, “Receive a content item comprising a video image with an info box (e.g., score box, news, ticker, etc.), “Determine a first screen position for the info box”, wherein info box in the video image corresponds to sub-blocks in the second layer of the video frame behind the graphic overlay, see Figures 1-3 and [0092]-[0093]), wherein the first layer of the video frame is above the second layer within the video frame (see Figures 1-3, wherein the overlay layer is above the content layer) and the video sub-blocks at least is partially obfuscated by the graphical overlay during rendering (Fig. 5A, “Does the second screen position overlap the first screen position? Yes”; and Fig. 3, the graphic overlay “ALERT: YOUR BACK DOOR WAS OPENED AT 6:46 PM” obfuscate at least partially the underlay image); and
evaluating, by the graphics computing system, characteristics of the video sub-blocks aligned with the first graphic rendering area and position (see citation in rejection to the preceding limitations above, wherein being in the same screen area is considered an alignment, see e.g., Figure 2. See [0096]-[0098], wherein an overlapping degree based in part in the rendering area and position of the video sub-blocks are characteristics of the video sub-blocks. See also Figure 5B-5D and [0092]-[0093]);
when the first value of the character tics is above a threshold value, evaluating the video sub-blocks not aligned with the first graphic rendering area for a second value (it is to be noted that this limitation is not positively recited but instead is conditional upon the “when the first histogram value is above a threshold histogram value” condition which may not happen, therefore the limitation will not be given patentable weight. Applicant is suggested to amend to “upon determining that the first histogram value is above a threshold histogram value”. Alternatively, if the claim limitation were to be given patentable weight, see Figure 5A, “Does the second screen position overlap the first screen position”? Yes”; see also [0092]-[0098], wherein the overlapping degree based in part on the second screen position of the underlay image/sub-blocks is characteristics of the underlay sub-blocks, and that a value of “zero overlapping” (or “non-overlapping”) is a threshold value. As disclosed, when the overlapping degree is determined to be overlapping (which is above the threshold “zero overlapping” or “non-overlapping”), another display area is evaluated to find a non-overlapping zone, see particularly [0098]-[0100]);
selecting, based on the second value being not above the threshold value, a second graphic rendering area (see citation above, upon determining that the overlay does not overlap with a second graphic rendering area, the second graphic rendering area is selected to render the overlay, wherein non-overlapping is at the threshold itself, not above the threshold value); and
relocating, within the video frame, the graphic overlay to the second graphic rendering area (Figure 5A and [0101]).
However, Gupta does not expressly disclose a histogram of the video sub-blocks aligned with the first graphic rendering area and position, wherein the histogram provides a first histogram value indicating an approximate first density of an underlying distribution of contrast differences (see 112 rejection above and Examiner’s interpretation therein), or that the characteristics value of the underlay image used to determine where to position the graphic overlay is the histogram value, or that a histogram value corresponding to positioning the overlay is below a threshold.
Eschbach discloses a histogram of underlay video image sub-blocks aligned with a potential overlay area and position ([0004], “In order to overlay text or other objects onto an image, one has to avoid image areas that are of importance to an observer”; [0009], “Empirical evidence suggests that regions of images that are important are relatively high in contrast. In other words, the image region makes use of essentially the entire dynamic range that is possible. The dynamic range of an image can be characterized by a histogram on the image”; [0017], “step 140 checks the overlay region to assure that, at least with respect to the image, it is overlaid only in a region(s) of low interest (i.e., a region of low importance to an observer)”; [0019], “characterizing the level of interest to an observer would be accomplished by determining a contrast value for each region”, wherein the “low interest” region aligned with the overlay area and position is determined by a contrast map, see [0030], “The contrast map of FIG. 6 can now be used to define a preliminary area of "low interest" for the image of FIG. 3”, and wherein the contrast map is represented by a respective histogram, see [0022], “a global histogram of the luminance or Y-component of the pictorial image will be optionally derived”; [0024], “the image is divided into a set of local areas or regions 320 as depicted in FIG. 3 (from (0,0) to (m,n)), not necessarily identical in size, or ordered in any fashion, and histograms from each local area are also derived”; [0025], “The next step in the analysis process based upon image contrast compares the global histogram to a reference which in the example is a flat histogram. A flat histogram, as defined herein, is a reference signal, which provides a uniform number of pixel counts for each density or luminance possible within the image. The global histogram is compared to this flat histogram to give a global measure of contrast in the form of a variance.” Claim 8, “wherein determining a contrast value for each region comprises the steps of: for each region of the image, characterizing a local histogram signal representing the image signals within the region; comparing each local histogram signal to a reference histogram signal; and deriving a local contrast variance from the comparison, representing the flatness of the local histogram”; Also see abstract),
wherein the histogram provides a first histogram value indicating an underlying distribution of contrast differences (See citation in rejection to the preceding limitation, the respective histogram variance or contrast variance value. For example, [0027], “Generally speaking, with reference to the variance, the smaller the value, the flatter the histogram. The variance value is determined for each of the local histograms and is shown in FIG. 4”; [0030], “The contrast map of FIG. 6 can now be used to define a preliminary area of "low interest" for the image of FIG. 3. The preliminary area of low interest can be created or defined by weighting contrast attributes, or in other words giving high values to areas of low contrast.”. Hence, low contrast/histogram variance value are “low interest” areas that are candidate for placing the overlay, see [0017], “In order to correctly insert the variable data, step 140 checks the overlay region to assure that, at least with respect to the image, it is overlaid only in a region(s) of low interest (i.e., a region of low importance to an observer). As a result of the check and submit operation at step 140, at least a portion of the content is automatically inserted into a region of the image.” Therefore, a region with a lower contrast value or histogram variance is more likely (i.e., with a higher score of probability) to be determined as a “low interest” region for placing/positioning the overlay in the respective region),
and that the characteristics value of the underly image used to determine where to position the graphic overlay is the histogram value (see citation and Examiner’s explanation above, wherein the contrast value represented by the histogram variance in contrast map is used to determine a “low interest” region to place/position the overlay), and that the value corresponding to positioning the overlay needs to be “below” a threshold (see citation in rejection to the preceding limitation, wherein lower than a threshold is implied to determine low histogram variance or low contrast variance. This is also expressly disclosed, e.g., in claim 17, “applying a threshold to the contrast variance values to identifying those regions with the lowest contrast”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Gupta with Eschbach. The suggestion/motivation of the combination would have been to place overlays in a region of low interest to observers (Eschbach, abstract).
As discussed above, Gupta discloses the video sub-blocks at least is partially obfuscated by the graphical overlay during rendering (Fig. 5A, “Does the second screen position overlap the first screen position? Yes”; and Fig. 3, the graphic overlay “ALERT: YOUR BACK DOOR WAS OPENED AT 6:46 PM” obfuscate at least partially the underlay image); but does not expressly disclose the reverse way, e.g., that the underlay video sub-blocks at least partially obfuscate the graphical overlay during rendering. Hoffman discloses a situation n wherein an underlay video image sub-blocks at least partially obfuscate a graphical overlay during rendering ([0044], “the overlaid elements can still get lost in high noise areas of the image or image sections with colors that are close to the displayed text”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Gupta-Eschbach with Hoffman. The suggestion/motivation of the combination would have been to address issues when the overlay elements are obscured by underlay image(s) (Hoffman, [0044]).
As to claim 8, see similar rejection to claim 1.
As to claim 15, see similar rejection to claim 1.
As to claim 4, Gupta-Eschbach-Hoffman discloses the method of claim 1, wherein the evaluating the histogram further comprises calculating a sum of contrast difference scores of the video sub-blocks for the second rendering area (Eschbach, [0027], “The variance value is determined for each of the local histograms and is shown in FIG. 4”; [0028] The contrast map 400 is now used as the basis to find relatively large, connected regions…This contrast map 400 of FIG. 4 is first averaged to eliminate small local areas, producing a result such as the averaged map 500 depicted in FIG. 5” wherein “averaging” reflects calculating a sum).
As to claim 12, see similar rejection to claim 4.
As to claim 5, Gupta-Eschbach-Hoffman discloses the method of claim 4, wherein the blending factor is triggered based on the sum of contrast difference scores exceeding a selected threshold (see 112 rejection above and Examiner’s interpretation therein. See citation in rejection to claim 4 above, e.g., [0028], “This contrast map 400 of FIG. 4 is first averaged to eliminate small local areas, producing a result such as the averaged map 500 depicted in FIG. 5” wherein “averaging” reflects calculating a sum. The averaged contrast map is then compared to a threshold, and if exceeding the threshold, indicates the region(s) are not regions with “lowest contrast”, see claim 17, “applying a threshold to the contrast variance values to identifying those regions with the lowest contrast”. See citation in rejection to claim 1 above, disclosing that low contrast corresponds to low interest regions suitable for positioning the overlay, and that high contrast corresponds to high interest regions unsuitable for positioning the overly. Therefore, determining a region’s (averaged) contrast value(s) being above the threshold triggers classifying the region to be “high interest” region unsuitable for positioning the overlay on, in other words, not to blend the overlay and underlay contents, which can be considered a blending factor).
As to claim 11, Gupta-Eschbach-Hoffman discloses the graphics processing system of claim 9, wherein the at least one processor is further configured to trigger the generation of the blending factor based on the total contrast difference score exceeding a selected threshold (see similar rejection to claim 5).
As to claim 14, Gupta-Eschbach-Hoffman discloses the graphics processing system of claim 8, wherein the contrast difference is based on differences in brightness and contrast (Eschbach, see citation in rejection to claim 1, wherein the contrast difference is based on differences in contrast which reflects difference in brightness, see [0021], “luminance based contrast determination is carried out to identify regions or interest (or regions of non-interest)”. Also see [0019] and [0022], “a global histogram of the luminance or Y-component of the pictorial image will be optionally derived. The global histogram refers to the entire image”).
10. Claims 2, 7, 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta-Eschbach-Hoffman, as applied to claim 1 above, and further in view of Zhou et al (US 2023/0267581).
As to claim 2, Gupta-Eschbach-Hoffman discloses the claimed invention substantially as discussed in claim 1, but does not expressly disclose generating, as triggered by the histogram, a blending factor associated with each individual one of the image pixels and the text pixels of the first layer based on brightness and contrast contributions from a first pixel in the first layer and brightness and contrast contributions from a second pixel in the second layer, where the first pixel and second pixel are aligned within the second graphic rendering area. Zhou discloses generating, as triggered by a histogram, a blending factor associated with an image pixel or text pixel of a first layer based on brightness and contrast contributions from a first pixel in the first layer and brightness and contrast contributions from a second pixel in a second layer, where the first pixel and second pixel are aligned within a graphic rendering area ([0020], [0025]; [0028]; Figures 3-4; [0044], wherein the graphics of the overlay layer to be transparently composited with the underlying content layer).
Before the effective filing date, it would have been obvious for an ordinary skilled in the art to combine Gupta-Eschbach-Hoffman with Zhou. The suggestion/motivation of the combination would have been to avoid overlay being obscured (Zhou, abstract).
As to claim 9, see similar rejection to claim 2.
As to claim 16, see similar rejection to claim 2.
As to claim 7, Gupta-Eschbach-Hoffman discloses the method of claim 1, wherein the graphical overlay comprises a user interface providing media content control information for a viewer (Gupta, Fig. 2, the graphic overlay “Coming up on market talk” is a user display which is a type of user interface, that provides media content control information for a viewer, the media control information being information regarding presenting a next media item, i.e., “market talk”. It is to be noted that the claim does not require a specific type of media control information).
If the claimed term “media content control information” were to be construed narrower, then Gupta does not expressly teach media content control information. Zhou teaches an overlay comprise a user interface providing media content control information for a viewer ([0029], “an image to be displayed on the display device 118 includes HDR underlying content and an SDR overlay… HDR television content delivered over a cable television network where a set-top box provides SDR menus to be displayed over the HDR television content, or a media player ( e.g., blu-ray player) that displays HDR content from a physical media and also displays SDR menus, text, and/or graphics over the HDR content from the physical media”).
Before the effective filing date, it would have been obvious for an ordinary skilled in the art to combine Gupta-Eschbach-Hoffman with Zhou. The suggestion/motivation of the combination would have been to improve user friendliness.
11. Claims 3, 10 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta-Eschbach-Hoffman-Zhou, as applied to claim 2 above, and further in view of Saunders (US 2011/0063325).
As to claim 3, Gupta-Eschbach-Hoffman-Zhou discloses the claimed invention substantially as discussed in claim 2, including increasing, by the graphics computing system and based on the blending factor associated with the each individual one of the image pixels and the text pixels of the first layer, a characteristic value (such as brightness) of the image pixels and the text pixels of the graphical overlay, within the video frame, to increase a readability of the text based on the brightness and contrast contributions of the blending factor for the image pixels and the text pixels (See 112 rejection above and Examiner’s interpretation therefore. See Zhou, [0020], [0025]; [0028]; [0039]; [0042]; [0044], wherein the graphics of the overlay layer to be transparently composited with the underlying content layer. See [0028], “The compositor optionally analyzes one or more characteristic values of pixels of input content, (where input content includes underlying content as well as overlay content), and (in at least some situations) adjusts the brightnesses of one or more pixels of one or both of the underlying content and the overlay content such that, when blended, the overlay content has a different appearance than if no adjustments were made and the underlying content and overlay content were blended. This modification helps to improve contrast between SDR overlay and HDR underlying content so that the SDR overlay is more visible to a viewer”; [0042], “The adjustments made are based on a comparison between the alpha values of the SDR pixels and the calculated average alpha value. For pixels that have a lower alpha than the average value, a greater boost to luminance is applied”), but does not expressly disclose that the increased characterize value is an opacity. Saunders discloses a concept that an increased characteristic value of an overlay is opacity ([0085], “If, as determined at 404b, the brightness level is lower than the benchmark brightness level (i.e. the display image is dark), then the transparency level for the overlay will be set to a level that is lower than a default transparency level (i.e. increased opacity) associated with the benchmark brightness level at 404c”.
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Gupta-Eschbach-Hoffman-Zhou with Saunders. The suggestion/motivation of the combination would have been to adapt to back background (Saunders, [0085]).
As to claim 10, see similar rejection to claim 3.
As to claim 17, see similar rejection to claim 3.
As to claim 18, Gupta-Eschbach-Hoffman-Zhou-Saunders discloses the non-transitory computer-readable medium of claim 17, further configured to perform operations calculating a total contrast difference score of the video sub-blocks in the histogram (see citation and Examiner’s explanation in rejection to claim 4).
As to claim 19, Gupta-Eschbach-Hoffman-Zhou-Saunders discloses the non-transitory computer-readable medium of claim 17, further configured to perform operations to trigger the generation of the blending factor based on the total contrast difference score exceeding a selected threshold (see citation and Examiner’s explanation in rejection to claim 4).
12. Claims 6, 13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta-Eschbach-Hoffman, as applied to claim 1 above, and further in view of Wong (US 9355493).
As to claim 6, Gupta-Eschbach-Hoffman discloses the claimed invention substantially as discussed in claim 1, including wherein the calculating a location of video sub-blocks further comprises extracting, by the graphics computing system, pixel index information from a frame buffer corresponding to the second layer of the video frame aligned with the second graphic rendering area and position (Gupta, [0109], “the overlay engine extracts a video frame from the content item as an image for analysis”; [0110], “At step 566, the overlay engine analyzes the video image for features to identify info box position data. In some embodiments, an info box may be identified using feature analysis and identifying the box shape or text. In some embodiments, an info box may be identified by stationary text and using OCR. In some cases, score box 104 may be identified by comparing a plurality of frames and determining what shapes and/or text do not change from frame to frame”, wherein feature analysis on the extracted video frame implies extracting pixel index information from a frame buffer holding pixel information of the video frame), but does not expressly disclose that pixel index information is extracted from a frame buffer. Wong expressly disclose a concept of extracting pixel index information from a frame buffer (Col. 9, lines 44-48, “The alpha-blending is carried out in accordance with the transparency values, as noted above. Pixel locations corresponding to the mask (i.e., those pixel locations excluded from the blending operation), in the image buffer (frame buffer) are set to corresponding opaque pixels of the foreground plane”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Gupta-Eschbach-Hoffman with Wong. The motivation/suggestion of the combination would have been to set location as the opaque pixels (Wong, Col. 9, lines 44-48).
As to claim 13, see similar rejection to claim 6.
As to claim 20, see similar rejection to claim 6.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUA FAN whose telephone number is (571)270-5311. The examiner can normally be reached on 9-6.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Umar Cheema can be reached at 571-270-3037. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUA FAN/Primary Examiner, Art Unit 2458
Prosecution Insights