DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/14/2026, 06/03/2026 was filed. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 9 and 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over GUO (US 20150245004 A1, Pub Date: Aug. 27, 2015, hereinafter “Guo”) in view of KUNKEL (US 20180374192 A1, Pub Date: Dec. 27, 2018, hereinafter “Kunkel-4192”).
Regarding independent claim 1, Guo teaches: A method for processing data, the method comprising:
receiving a set of elements containing content to be composited into a displayable unit (Guo − [0025] The composition unit 210 may evaluate the dynamic range of the video content and the graphics/UI elements to display a frame optimized for and/or requested by the display device. The composition unit 210, which may be a hardware or software component, may receive video content 206 and graphics/UI content 204, evaluate the received content, determine where to place the graphics/UI content in relation to video content, and output a frame with the mixed content.)
and data representing positions or positioning rules in the displayable unit of each of the elements in the set of elements, the set of elements comprising all content to be displayed on the displayable unit; (Guo − [0025] The composition unit 210 may generate frames in a display-oriented manner, optimizing the frames based on specified conditions and/or ambient conditions of the display device. To mix the content, the composition unit 210 may consider dynamic ranges and luminance of the inputs, detected ambient conditions including lighting levels of the display device, a mode of operation specified by a user, metadata that may have been already provided with the content, and the viewer's position relative to the display panel (e.g. viewing distance, viewing angle).)
determining one or more types of content in each of the elements; ([0025] To mix the content, the composition unit 210 may consider dynamic ranges and luminance of the inputs, detected ambient conditions including lighting levels of the display device, a mode of operation specified by a user, metadata that may have been already provided with the content, and the viewer's position relative to the display panel (e.g. viewing distance, viewing angle).)
generating a set of metadata, from the set of elements, the set of metadata for use in creating composited content from the set of elements on the displayable unit, the set of metadata comprising (1) spatial data about each of the elements and (2) image metadata about at least some of the elements, (Guo − [0025] metadata that may have been already provided with the content, and the viewer's position relative to the display panel (e.g. viewing distance, viewing angle). For example, the metadata, which may be transmitted in a bitstream, may indicate whether, where, size, color, and/or brightness of captions and/or graphics/UI should be displayed on a frame. Information about multiple viewers may also be considered. The mode of operation may override the other considerations or limit the effects of the other considerations. [0047] As a further example, high dynamic range and/or wide color gamut content such as that described herein may be displayed on conventional 8-bit, 2.2 gamma, display pipe and/or displays by using spatial-temporal dither. The compositor 400 may also enhance the chromaticity and luminance of video samples, and optimize the frames for a low-light setting. One or more modes may be selected at any given time.)
and wherein the image metadata comprises color volume properties or image statistics for at least some of the elements in the set of elements; (Guo − [0047] For example, the desired effects may be brought about by increased precision (including a brighter reference), a different transfer function, and/or an extended-range representation (e.g., codings that exceed 1.0). As a further example, high dynamic range and/or wide color gamut content such as that described herein may be displayed on conventional 8-bit, 2.2 gamma, display pipe and/or displays by using spatial-temporal dither. The compositor 400 may also enhance the chromaticity and luminance of video samples, and optimize the frames for a low-light setting. One or more modes may be selected at any given time.)
and wherein the image statistics comprises one or more of: maximum luminance of an image; diffuse white level and white-point of an image, minimum luminance of an image; mean luminance of an image; or median luminance of an image; (Guo − [0047] For example, the desired effects may be brought about by increased precision (including a brighter reference), a different transfer function, and/or an extended-range representation (e.g., codings that exceed 1.0). As a further example, high dynamic range and/or wide color gamut content such as that described herein may be displayed on conventional 8-bit, 2.2 gamma, display pipe and/or displays by using spatial-temporal dither. The compositor 400 may also enhance the chromaticity and luminance of video samples, and optimize the frames for a low-light setting. One or more modes may be selected at any given time.)
and storing the generated set of metadata with an association to each of the elements for use during composition of the set of elements on the displayable unit. (Guo − [0015] The elements may come from different devices, for example, video footage may be provided by storage; [0016] the content may be stored on a storage device such as a DVD or flash memory; parsed, processed and rendered by a reader and graphics processing unit; and displayed on a display screen of a computer. Claim 10 storing the output frame in a non-transitory computer-readable medium)
Guo does not explicitly teach: wherein the spatial data comprises vector based spatial data that defines approximate boundaries on the displayable unit of each of the elements in the set of elements,
However, Kunkel-4192 teaches: wherein the spatial data comprises vector based spatial data that defines approximate boundaries on the displayable unit of each of the elements in the set of elements, (Kunkel-4192 – [0040-0043] [0042] Examples of image metadata whose spatial distributions may be captured/represented in spherical maps as described herein may include, but are not limited to only, any of: scalar values, vector values, matrix values, content mapping parameters, color mapping parameters, tone mapping parameters, prediction related parameters, lifts, gains, and gamma values in transfer functions, filter kernels, histograms of image codewords, statistical values of image codewords, luminance values, chroma values, primary color values, white point values, saliency maps, regions of interest (ROI), depth maps, spatial resolutions, light field maps, frame rates, movements, motion vectors, mappings between one of a perceptually quantized space and a linearly quantized space and another of one of a perceptually quantized space and a linearly quantized space, etc.)
and storing the generated set of metadata with an association to each of the elements for use during composition of the set of elements on the displayable unit. (Kunkel-4192 – [0152] In some embodiments, the data repository (710) represents one or more databases, one or more data storage units/modules/devices, etc., configured to support operations such as storing, updating, retrieving, deleting, etc., with respect to some or all of the spherical images, the spherical maps of image metadata, etc)
Accordingly it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the teachings of Guo and Kunkel-4192 because both references relate to processing and compositing image content using image metadata. Kunkel-4192 teaches representing spatial image metadata using vector-based spatial information associate with image content. One of ordinary skill in the art would have been motivated to incorporate Kunkel-4192’s vector-based spatial metadata into Guo’s metadata generation and compositing technique in order to provide a scalable and efficient representation of the spatial boundaries and locations of display elements during image composition.
Regarding dependent claim 2, depends on claim 1, Guo teaches: wherein the displayable unit is one of: (a) a page, sheet, folio, or other unit of content; or (b) a web page; or (c) a portion of or all of a screen of a display device or (d) content generated at least in part by a computer program. (Guo – [0015-0017] [0016] Video data may be transmitted over a network while UI data may be generated locally in the video player or display device. The video and other visual content may also be derived from local storage, without being transmitted over a network. For example, the content may be retrieved and displayed on a computer: the content may be stored on a storage device such as a DVD or flash memory; parsed, processed and rendered by a reader and graphics processing unit; and displayed on a display screen of a computer. [0025] The composition unit 210 may evaluate the dynamic range of the video content and the graphics/UI elements to display a frame optimized for and/or requested by the display device.)
Regarding dependent claim 3, depends on claim 2, Guo teaches: wherein the method further comprises: transmitting the set of elements and the generated set of metadata in response to a request for the web page, and wherein the data representing positions or positioning rules is contained in a description of the displayable unit in a hypertext markup language. (Guo – [0015] [0016] The video and other visual content may be delivered over a network, such as a local area network or the Internet, via channels 112, 114 [0025] graphics/UI content, metadata and generation of display frames from received content.)
Regarding dependent claim 4, depends on claim 1, Guo teaches: wherein the one or more types of content comprise at least one of: (a) high dynamic range (HDR) image content; (b) standard dynamic range (SDR) image content; (c) text content; or (d) user interface content for use in receiving inputs from a user. (Guo – [0003] The various visual elements to be mixed by the compositor may be of different dynamic ranges and color spaces. For example, video may be in a High Dynamic Range (HDR) and/or Wide Color Gamut (WCG) format, while graphics/UI elements may be in a standard dynamic range (SDR) or Standard Color Gamut (SCG) format.)
Regarding dependent claim 5, depends on claim 1, Guo teaches: wherein the set of metadata also describes a temporal change of the content over time, and wherein the set of metadata describes a rate of change associated with the temporal change of the content over time, the rate of change indicating a rate, over time, that metadata in the set of metadata changes from one scene to a next scene. (Guo – [0047] As a further example, high dynamic range and/or wide color gamut content such as that described herein may be displayed on conventional 8-bit, 2.2 gamma, display pipe and/or displays by using spatial-temporal dither. The compositor 400 may also enhance the chromaticity and luminance of video samples, and optimize the frames for a low-light setting. One or more modes may be selected at any given time.)
Regarding dependent claim 6, depends on claim 1, Guo teaches: wherein the image metadata comprises data about detected glare or data from which glare is detected in at least one of the elements in the set of elements, and wherein the detected glare is classified as one of disability glare or discomfort glare. (Guo − [0012] The composition unit automatically may adapt HDR inputs based on a display device's ability to render the frame and improve visibility in non-ideal viewing environments. The composition unit may output a frame that uses less backlight in the display device, which may save power in many viewing environments. In some instances, the output frame may make the display and/or backlight appear like paper to mimic ideal reading conditions to alleviate eyestrain. [0045] For instance, a camera may detect an eye distance and determine a field of view of that the display subtends. The sensor(s) may also perform 3D sensing such as time of flight, structured light. The 3D sensing may be used to determining a position of one or more viewers in a room and adapt the display to the positions of the viewer(s).)
Regarding dependent claim 9, depends on claim 1, Guo teaches: wherein the image metadata comprises a quantized representation of at least one of the elements in the set of elements. (Guo − [0025] metadata that may have been already provided with the content, and the viewer's position relative to the display panel (e.g. viewing distance, viewing angle). For example, the metadata, which may be transmitted in a bitstream, may indicate whether, where, size, color, and/or brightness of captions and/or graphics/UI should be displayed on a frame. Information about multiple viewers may also be considered. The mode of operation may override the other considerations or limit the effects of the other considerations. [0047] As a further example, high dynamic range and/or wide color gamut content such as that described herein may be displayed on conventional 8-bit, 2.2 gamma, display pipe and/or displays by using spatial-temporal dither. The compositor 400 may also enhance the chromaticity and luminance of video samples, and optimize the frames for a low-light setting. One or more modes may be selected at any given time.)
Regarding dependent claim 12, depends on claim 1, Guo teaches: A non-transitory machine readable medium storing executable program instructions which when executed by one or more data processing systems cause the one or more data processing systems to perform a method as in claim 1. (Guo − Claim 10 a non-transitory computer-readable medium)
Kunkel-4192 teaches: A non-transitory machine readable medium storing executable program instructions which when executed by one or more data processing systems cause the one or more data processing systems to perform a method as in claim 1. (Kunkel−4192 [0201] In an embodiment, a non-transitory computer readable storage medium stores software instructions, which when executed by one or more processors cause performance of a method as described herein.)
Accordingly it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the teachings of Guo and Kunkel-4192 because both references relate to processing and compositing image content using image metadata. Kunkel-4192 teaches representing spatial image metadata using vector-based spatial information associate with image content. One of ordinary skill in the art would have been motivated to incorporate Kunkel-4192’s vector-based spatial metadata into Guo’s metadata generation and compositing technique in order to provide a scalable and efficient representation of the spatial boundaries and locations of display elements during image composition.
Regarding dependent claim 13, depends on claim 1, Guo teaches: A data processing system configured to perform a method as in claim 1. (Guo – [abstract] A method and system for adaptively mixing video components with graphics/UI components, where the video components and graphics/UI components may be of different types, e.g., different dynamic ranges (such as HDR, SDR) and/or color gamut (such as WCG).)
Claim(s) 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Guo, Kunkel-4192 as applied to claim 1 above, and further in view of KUNKEL (US 20130251241 A1, Pub Date: Sep. 26, 2013, hereinafter “Kunkel-1241”).
Regarding dependent claim 7, depends on claim 1, Guo does not explicitly teach: texture abstraction of at least one of the elements in the set of elements
However Kunkel-1241 teaches: wherein the image metadata comprises a texture abstraction of at least one of the elements in the set of elements. (Kunkel − [0043,0047] 0043] Applied noises as described herein may refer to any kind of perceptible pattern or texture including film grains; image metadata, in addition to image data samples; the image metadata may specify/suggest a particular type of noise to be applied to a particular 3D image or frames. In another example, existing non-perceptual 2D noises may be removed and analyzed to determine a particular noise pattern to be combined with depth information for the purpose of applying the particular noise pattern as perceptually correct 3D noises simulate sensor noises or specific looks of a specific device. Noise application as described herein may use one or more of sampling techniques, modeling techniques, techniques based on matrix, textural patterns, computer-generated graphics, wood textures, other material textures, etc.)
Accordingly, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the teaching of Guo, Kunkel-4192, and Kunkel-1241 because each references relate to processing image content using image metadata. One of ordinary skill in the art would have been motivated to incorporate Kunkel-1241’s texture metadata into Guo’s metadata generation technique in order to provide additional image characteristics that may be used during image composition and rendering.
Regarding dependent claim 8, depends on claim 7, Guo does not explicitly teach: wherein the texture abstraction is derived from a Fourier analysis based representation of the at least one of the elements in the set of elements.
However Kunkel-1241 teaches: wherein the texture abstraction is derived from a Fourier analysis based representation of the at least one of the elements in the set of elements. (Kunkel-1241 − Fig. 4 [0047] [0063] FIG. 4 illustrates example adjustments of spatial frequency components of applied noise as a function of depth (z-axis). As illustrated, the closer a 3D object or image detail is to the viewer in the scene, the higher the spatial frequency of the applied noise. This may be accomplished by increasing or shifting spatial frequencies of the spatial frequency components and/or by increasing weights of high spatial frequency ones of the spatial frequency components.)
Accordingly, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the teaching of Guo, Kunkel-4192, and Kunkel-1241 because each references relate to processing image content using image metadata. One of ordinary skill in the art would have been motivated to incorporate Kunkel-1241’s texture metadata into Guo’s metadata generation technique in order to provide additional image characteristics that may be used during image composition and rendering.
Claim(s) 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Guo, Kunkel-4192 as applied to claim 1 above, and further in view of Lathrop (US 20070242082 A1, hereinafter “Lathrop”).
Regarding dependent claim 10, depends on claim 1, Guo does not explicitly teach: wherein an application programing interface (API) is used to cause the generation of the set of metadata, the API linking a metadata generation component in a data processing system with a web page creation software.
However, Lathrop teaches: wherein an application programing interface (API) is used to cause the generation of the set of metadata, the API linking a metadata generation component in a data processing system with a web page creation software. (Lathrop − [0028] Together, the components of System 100 provide the ability for a user to create graphical user interface (GUI) pages that are part of an application. According to one embodiment, the application is a web application that employs web services and the Web Services Definition Language (WSDL). WSDL defines the public interface for a web service or application. A client, such as a web application, connecting to a web service can read and interpret the WSDL to determine what functions and/or data are available for use within the web application. Alternatively, the components of System 100 may be used for any other purpose, such as designing web pages for a web site.)
Accordingly, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the teaching of Guo, Kunkel-4192, and Lathrop because Lathrop teaches using an application interface to facilitate communication between software components used for webpage creation. One of ordinary skill in the art would have incorporated such an interface into Guo’s metadata generation system to improve interoperability and integration with webpage creation software while retaining the image metadata generation.
Regarding dependent claim 11, depends on claim 1, Guo does not explicitly teach: wherein the set of metadata is stored in a scalable vector graphics format.
However, Lathrop teaches: wherein the set of metadata is stored in a scalable vector graphics format. (Lathrop − [0007] In accordance with one embodiment of the present invention, there is provided a method for providing Scalable Vector Graphics, Scalable Vector Graphics (SVG) is a standard for representing metadata-defined images, which may be incorporated into display pages of a web application.)
Accordingly, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined the teaching of Guo, Kunkel-4192, and Lathrop because Lathrop teaches using an application interface to facilitate communication between software components used for webpage creation. One of ordinary skill in the art would have incorporated such an interface into Guo’s metadata generation system to improve interoperability and integration with webpage creation software while retaining the image metadata generation.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARL E BARNES JR whose telephone number is (571)270-3395. The examiner can normally be reached Monday-Friday 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at (571) 272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARL E BARNES JR/Examiner, Art Unit 2178
/STEPHEN S HONG/Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights