DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20, as originally filed, are currently pending and have been considered below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oh, U.S. Publication No. 2016/0343348, hereinafter, “Oh”, and further in view of Kadu et al., U.S. Publication No. 2022/0058783, hereinafter, “Kadu”.
As per claim 1, Oh discloses a computer-implemented method for training a machine-learning algorithm for automatically converting video content, the computer-implemented method comprising:
receiving, by one or more processors, one or more source images having Standard Dynamic Range (SDR) video content (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; Oh, ¶0013, The input content may include a Standard Dynamic Range (SDR) content; Oh, ¶0066, The input interface 110 receives a content recorded on a recording medium such as a DVD or a BD and metadata of the corresponding content. Here, the content is an SDR content);
receiving, by one or more processors, a profile based on an identity corresponding to the one or more source images (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; and a processor configured to acquire image quality information applied to the content based on the metadata; Oh, ¶0014, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content; Oh, ¶0015, The HDR pixel values may correspond to the SDR pixel values of the image frames of the SDR content, and the processor may be further configured to convert the SDR pixel values of the image frames of the SDR content into the HDR pixel values based on the lookup table; Oh, ¶0074, as shown in FIG. 3, if an SDR content and metadata of the SDR content are input through the input interface 110, a data processor 150 decodes the SDR content based on the input metadata; Oh, ¶0075, a BD may include an SDR content that is encoded through a codec having a MPEG-2, H.264/MPEG-4 AVC, or VC-1 standard. Therefore, the data processor 150 may decode the input SDR content based on the encoding information of the metadata of the SDR content input through the input interface 110);
converting, by the one or more processors executing the machine-learning algorithm, the one or more source images to High Dynamic Range (HDR) video (Oh, ¶0023, a method of outputting a content from a content outputting apparatus to a display includes receiving an input content and metadata associated with the input content; acquiring image quality information applied to the input content based on the metadata; determining content conversion information related to the acquired image quality information; converting the input content into a converted content outputtable on the display using the content conversion information; and outputting the converted content to the display; Oh, ¶0024, The input content may include an SDR content, the image quality information applied to the input content may include gamma information, and the converted content outputtable on the display may include an HDR content; Oh, ¶0025, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content; Oh, ¶0026, The HDR pixel values may correspond to the SDR pixel values of the image frames of the SDR content, and the converting may further include converting the SDR pixel values of the image frames of the SDR content into the HDR pixel values based on the lookup table).
Oh does not explicitly disclose the following limitations as further recited however Kadu discloses
selecting, by the one or more processors, a machine-learning algorithm from among a plurality of machine-learning algorithms based on the profile (Kadu, ¶0045, Under techniques as described herein, a multitude of different ML prediction models/algorithms/methods can be learned for various specific user preferences. Then switching among a plurality of different user-preferred HDR looks can be as easily performed under these techniques as switching among the different ML prediction models/algorithms/methods. As a result, SDR content can be backward reshaped into HDR content with desired HDR looks for users ... Instead of manually color grading HDR images generated from a potentially vast amount of available SDR content, an individual user can provide a training dataset of pairs of SDR and corresponding (e.g., manually, etc.) color graded HDR images. Machine learned models/algorithms/methods under techniques as described herein can automatically transform any SDR content to corresponding HDR content by mimicking user preferences as embodied by the training dataset; Kadu, ¶0150-0151, Style Transfer with Encoder … as illustrated in FIG. 2F, different styles (e.g., a style preferring bluish images, a style preferring reddish images, a style of a first colorist preferring strong contrasts, a style of a different colorist preferring softer images, etc.) or different HDR looks can be transferred from an upstream video encoder to downstream video decoder(s) by way of dynamic composer metadata);
selecting, by the one or more processors, at least one of the one or more converted source images that exceed one or more image parameter thresholds (Kadu, ¶0019, Dynamic composer metadata generated at least in part through machine learning is used to derive one or more operational parameter values … The one or more operational parameter values of the image-specific backward reshaping mappings are used to backward reshape the SDR image into the mapped HDR image; Kadu, ¶0026, Image metadata transmitted with SDR video contents to a recipient device may include ML composer metadata; Kadu, ¶0027, One of the valuable services provided by the ML composer metadata generation (115) is to generate the HDR images from the SDR images; Kadu, ¶0036, In operational scenarios in which the receiver operates with (or is attached to) an HDR display 140-1 that supports a high dynamic range (e.g., 400 nits, 1000 nits, 4000 nits, 10000 nits or more, etc.), the receiver can extract the composer metadata from (e.g., the metadata container in, etc.) the coded bitstream (122) and use the composer metadata to compose HDR images (132); Kadu, ¶0039, The dynamic composer metadata (or ML-based composer metadata) can be infused with different user-defined (e.g., end user selectable, etc.) styles to modify the HDR look based individual users' respective preferences; Kadu, ¶0042, under the ML based approach of predicting or estimating HDR luma/chroma codewords, relevant image features (e.g., content dependent features, pixel value dependent features, etc.) can be extracted from SDR image data and used to train, predict and/or estimate (dynamic) ML-based composer metadata for constructing or reconstructing HDR image data from the SDR image data. In some operational scenarios, such construction or reconstruction can be further influenced by user input specifying a user selection of a user-intended visual style/mode; Kadu, ¶0153, The selected set of GPR models for the SDR image (282) may be selected from among different sets of GPR models such as 206-1 through 206-N, wherein N is a positive integer greater than one (1). The different sets of GPR models 206-1 through 206-N may be trained by different sets of training SDR-HDR image pairs in one or more different training datasets. For example, for a set of training SDR images, multiple sets of corresponding training HDR images may be generated with each set in the multiple sets of corresponding training HDR images representing a distinct user-desired style or HDR look among multiple user-desired styles or HDR looks represented in the multiple sets of corresponding training HDR images. As used herein, a user desired style or HDR look may refer to a style of images (e.g., HDR images, etc.) as preferred or intended by a user such as a colorist, a professional video creator, a studio, etc.; Kadu, ¶0154, a selected set of GPR models corresponding to a selected user desired style or HDR look may be a single set of GPR models selected (e.g., based on user preferences, system configuration information, etc.) from among the different sets of GPR models 206-1 through 206-N; Kadu, ¶0160, The different sets of SDR-HDR image pairs that support style transfer may comprise (training) HDR images of (or tailored to) different HDR looks or different user defined styles. For example, a first set of SDR-HDR image pairs in a first training dataset of the training datasets may correspond a first HDR look or a first user defined style, whereas a second different set of SDR-HDR image pairs in a second training dataset of the training datasets may correspond to a second different HDR look or a second different user defined style; Kadu, ¶0161-0165); and
storing, by the one or more processors, the at least one converted source image that exceeds the one or more parameter thresholds in a training dataset for a machine-learning model (Kadu, ¶0043, Training ML prediction models/algorithms/methods under techniques as described herein may be done with a training dataset comprising pairs of SDR and corresponding (e.g., user-desired, manually color graded, etc.) HDR images. The ML prediction models/algorithms/methods can learn (e.g., user-intended, etc.) SDR-to-HDR mappings during the training phase. Machine learnt optimal operational parameters for the ML prediction models/algorithms/methods can be stored persistently or in cache/memory).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Kadu and Oh because they are in the same field of endeavor. One skilled in the art would have been motivated to include the machine learning algorithms as taught by Kadu in the system of Oh in order to optimize and customize the output images (Kadu, ¶0028; ¶0039).
As per claim 2, Oh and Kadu disclose the computer-implemented method of claim 1, the computer-implemented method further comprising: generating, by the one or more processors, a profile-based model for the machine-learning algorithm based on the training dataset and data from the profile (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; and a processor configured to acquire image quality information applied to the content based on the metadata, to convert the input content into a converted content; Oh, ¶0023, a method of outputting a content from a content outputting apparatus to a display includes receiving an input content and metadata associated with the input content; acquiring image quality information applied to the input content based on the metadata; determining content conversion information related to the acquired image quality information; converting the input content into a converted content outputtable on the display using the content conversion information; and outputting the converted content to the display; Kadu, ¶0039, The dynamic composer metadata (or ML-based composer metadata) can be infused with different user-defined (e.g., end user selectable, etc.) styles to modify the HDR look based individual users' respective preferences; Kadu, ¶0043, Training ML prediction models/algorithms/methods under techniques as described herein may be done with a training dataset comprising pairs of SDR and corresponding (e.g., user-desired, manually color graded, etc.) HDR images. The ML prediction models/algorithms/methods can learn (e.g., user-intended, etc.) SDR-to-HDR mappings during the training phase. Machine learnt optimal operational parameters for the ML prediction models/algorithms/methods can be stored persistently or in cache/memory; Kadu, ¶0150-0151, Style Transfer with Encoder … as illustrated in FIG. 2F, different styles (e.g., a style preferring bluish images, a style preferring reddish images, a style of a first colorist preferring strong contrasts, a style of a different colorist preferring softer images, etc.) or different HDR looks can be transferred from an upstream video encoder to downstream video decoder(s) by way of dynamic composer metadata carried in a video signal or coded bitstream encoded with SDR images).
As per claim 3, Oh and Kadu disclose the computer-implemented method of claim 2, wherein the one or more image parameter thresholds correspond to at least one technical parameter included in the profile, wherein the profile corresponds to a Color Decision List (CDL), a color Look-Up Table (LUT), color scheme, and/or a genre (Oh, ¶0014, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content, wherein the display may be further configured to gamma-correct the converted content by using gamma information predetermined to be applied to the HDR content; Oh, ¶0106, SDR pixel values of each of R, G, and B pixels of an 8-bit SDR content may exist between 0 and 255, and first gamma information of a plurality of pieces of gamma information applicable to the SDR content may be applied to the SDR content. As shown in FIG. 5, SDR pixel values R0 through R255 corresponding to R pixels of the 8-bit SDR content, to which the first gamma information is applied, may be stored in a lookup table 410. HDR pixel values R′0 through R′255 respectively corresponding to the SDR pixel values R0 through R255 of the R pixels of the 8-bit SDR content may match with the SDR pixel values R0 through R255 corresponding to the R pixels and then may be stored in the lookup table 410. Here, the HDR pixel values R′0 through R′255 may be values for gamma-correcting an SDR convent converted to have a characteristic of an HDR content by using first gamma information of a plurality of pieces of gamma information applicable to the HDR content; Kadu, ¶0161, For each SDR image (282), the different luma and chroma backward reshaping mappings of different HDR looks or different user defined styles may be represented by different BLUTs and/or different sets of MMR coefficients generated based on the different sets of GPR (luma prediction/estimation) models (206-1 through 206-N) and/or by different chroma dictionaries (210-1 through 210-N). The different sets of GPR models and/or the different chroma dictionaries can be respectively trained on different sets of SDR-HDR image pairs of different HDR looks or different user defined styles (in the different training datasets) and then individually (e.g., a selected closest style or look based on a distance measure, etc.) applied to any (e.g., test, to-be-predicted, to-be-backward-reshaped, etc.) SDR image such as the SDR image (282)).
As per claim 4, Oh and Kadu disclose the computer-implemented method of claim 1, the computer-implemented method further comprising: generating, by the one or more processors, a generic model for the machine-learning algorithm based on the training dataset (Oh, ¶0090, the processor 130 may transmit content information including null information for gamma-correcting the corresponding content in the display apparatus 200 to the display apparatus 200 along with the content converted to have the characteristic of the HDR content. If the content information including the null information is received, the display apparatus 200 may gamma-correct the content converted to have the characteristic of the HDR content by using default gamma information, signal-process the gamma-corrected content in a form that may be output, and display the signal-processed content; Kadu, ¶0019, Example embodiments described herein relate to image metadata generation/optimization through machine learning. An SDR image to be backward reshaped into a corresponding mapped HDR image is decoded from a video signal. Dynamic composer metadata generated at least in part through machine learning is used to derive one or more operational parameter values of image-specific backward reshaping mappings is decoded from the video signal. The one or more operational parameter values of the image-specific backward reshaping mappings are used to backward reshape the SDR image into the mapped HDR image. A display image derived from the mapped HDR image is caused to be rendered with a display device).
As per claim 5, Oh and Kadu disclose the computer-implemented method of claim 4, wherein the image parameter thresholds include at least one technical parameter corresponding to Standard Dynamic Range (SDR) video content and/or High Dynamic Range (HDR) video content (Oh, ¶0011, Exemplary embodiments provide converting a Standard Dynamic Range (SDR) content into a content corresponding to a High Dynamic Range (HDR) content outputtable on a display apparatus; Kadu, ¶0018, Example embodiments described herein relate to image metadata generation/optimization through machine learning. A plurality of training image pairs comprising a plurality of training SDR image and a plurality of corresponding training HDR images is received. Each training image pair in the plurality of training image pairs comprises a training SDR image in the plurality of training SDR images and a corresponding training HDR image in the plurality of corresponding training HDR images. The training SDR image and the corresponding training HDR image in each such training image pair depict same visual content but with different luminance dynamic ranges. A plurality of training image feature vectors is extracted from a plurality of training SDR images in the plurality of training image pairs. A training image feature vector in the plurality of training image feature vectors is extracted from a training SDR image in a respective training image pair in the plurality of training image pairs. The plurality of training image feature vectors and ground truth derived with the plurality of corresponding training HDR images are used to train one or more backward reshaping metadata prediction models for predicting operational parameter values of backward reshaping mappings used to backward reshape SDR images into mapped HDR images).
As per claim 6, Oh and Kadu disclose the computer-implemented method of claim 1, wherein the profile is associated with a profile identifier including a code and/or an address (Oh, ¶0075, a BD may include an SDR content that is encoded through a codec having a MPEG-2, H.264/MPEG-4 AVC, or VC-1 standard. Therefore, the data processor 150 may decode the input SDR content based on the encoding information of the metadata of the SDR content input through the input interface 110; Oh, ¶0106, SDR pixel values of each of R, G, and B pixels of an 8-bit SDR content may exist between 0 and 255, and first gamma information of a plurality of pieces of gamma information applicable to the SDR content may be applied to the SDR content. As shown in FIG. 5, SDR pixel values R0 through R255 corresponding to R pixels of the 8-bit SDR content, to which the first gamma information is applied, may be stored in a lookup table 410. HDR pixel values R′0 through R′255 respectively corresponding to the SDR pixel values R0 through R255 of the R pixels of the 8-bit SDR content may match with the SDR pixel values R0 through R255 corresponding to the R pixels and then may be stored in the lookup table 410. Here, the HDR pixel values R′0 through R′255 may be values for gamma-correcting an SDR convent converted to have a characteristic of an HDR content by using first gamma information of a plurality of pieces of gamma information applicable to the HDR content; Kadu, ¶0171-0174, Style Transfer with Decoder … In some embodiments, the image metadata comprises dynamic composer metadata that carry or comprise one or more specific cluster indexes (e.g., one or more numbers, one or more values, one or more integers, etc.) identifying one or more specific chroma clusters to be combined/fused into a combined/fused chroma cluster for deriving MMR coefficients, which can then be used for performing chroma backward reshaping on the SDR image (282) ... As a result, the user desired style or the HDR look can be controlled at the decoder side).
As per claim 7, Oh and Kadu disclose the computer-implement method of claim 1, wherein converting the one or more source images to High Dynamic Range (HDR) video further comprises converting, by the one or more processors, the one or more source images from a first color space to a second color space based on the profile (Oh, ¶0013, The input content may include a Standard Dynamic Range (SDR) content, the image quality information applied to the input content may include gamma information, and the converted content outputtable on the display may include a High Dynamic Range (HDR) content; Oh, ¶0105-0106; Kadu, ¶00370038, Dynamic Composer Metadata Generation Through Machine Learning … Single Layer Inverse Display Management (SLiDM) or SDR+ can be used to enhance SDR content for rendering on HDR display devices. Luma and chroma channels (or color components) of SDR images may be mapped separately using image metadata to generate corresponding luma and chroma channels of HDR images; Kadu, ¶0042, under the ML based approach of predicting or estimating HDR luma/chroma codewords, relevant image features (e.g., content dependent features, pixel value dependent features, etc.) can be extracted from SDR image data and used to train, predict and/or estimate (dynamic) ML-based composer metadata for constructing or reconstructing HDR image data from the SDR image data. In some operational scenarios, such construction or reconstruction can be further influenced by user input specifying a user selection of a user-intended visual style/mode).
As per claim 8, Oh discloses a computer system training a machine-learning algorithm for automatically converting video content, the computer system comprising:
a memory having processor-readable instructions stored therein; and one or more processors configured to access the memory and execute the processor-readable instructions, which when executed by the one or more processors configures the one or more processors to perform a plurality of functions (Oh, ¶0065, the content outputting apparatus 100 includes an input interface 110, a communicator 120, a processor 130, and a storage 140), including functions for:
receiving, by one or more processors, one or more source images having Standard Dynamic Range (SDR) video content (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; Oh, ¶0013, The input content may include a Standard Dynamic Range (SDR) content; Oh, ¶0066, The input interface 110 receives a content recorded on a recording medium such as a DVD or a BD and metadata of the corresponding content. Here, the content is an SDR content);
receiving, by one or more processors, a profile based on an identity corresponding to the one or more source images (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; and a processor configured to acquire image quality information applied to the content based on the metadata; Oh, ¶0014, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content; Oh, ¶0015, The HDR pixel values may correspond to the SDR pixel values of the image frames of the SDR content, and the processor may be further configured to convert the SDR pixel values of the image frames of the SDR content into the HDR pixel values based on the lookup table; Oh, ¶0074, as shown in FIG. 3, if an SDR content and metadata of the SDR content are input through the input interface 110, a data processor 150 decodes the SDR content based on the input metadata; Oh, ¶0075, a BD may include an SDR content that is encoded through a codec having a MPEG-2, H.264/MPEG-4 AVC, or VC-1 standard. Therefore, the data processor 150 may decode the input SDR content based on the encoding information of the metadata of the SDR content input through the input interface 110);
converting, by the one or more processors executing the machine-learning algorithm, the one or more source images to High Dynamic Range (HDR) video (Oh, ¶0023, a method of outputting a content from a content outputting apparatus to a display includes receiving an input content and metadata associated with the input content; acquiring image quality information applied to the input content based on the metadata; determining content conversion information related to the acquired image quality information; converting the input content into a converted content outputtable on the display using the content conversion information; and outputting the converted content to the display; Oh, ¶0024, The input content may include an SDR content, the image quality information applied to the input content may include gamma information, and the converted content outputtable on the display may include an HDR content; Oh, ¶0025, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content; Oh, ¶0026, The HDR pixel values may correspond to the SDR pixel values of the image frames of the SDR content, and the converting may further include converting the SDR pixel values of the image frames of the SDR content into the HDR pixel values based on the lookup table).
Oh does not explicitly disclose the following limitations as further recited however Kadu discloses
selecting, by the one or more processors, a machine-learning algorithm from among a plurality of machine-learning algorithms based on the profile (Kadu, ¶0045, Under techniques as described herein, a multitude of different ML prediction models/algorithms/methods can be learned for various specific user preferences. Then switching among a plurality of different user-preferred HDR looks can be as easily performed under these techniques as switching among the different ML prediction models/algorithms/methods. As a result, SDR content can be backward reshaped into HDR content with desired HDR looks for users ... Instead of manually color grading HDR images generated from a potentially vast amount of available SDR content, an individual user can provide a training dataset of pairs of SDR and corresponding (e.g., manually, etc.) color graded HDR images. Machine learned models/algorithms/methods under techniques as described herein can automatically transform any SDR content to corresponding HDR content by mimicking user preferences as embodied by the training dataset; Kadu, ¶0150-0151, Style Transfer with Encoder … as illustrated in FIG. 2F, different styles (e.g., a style preferring bluish images, a style preferring reddish images, a style of a first colorist preferring strong contrasts, a style of a different colorist preferring softer images, etc.) or different HDR looks can be transferred from an upstream video encoder to downstream video decoder(s) by way of dynamic composer metadata);
selecting, by the one or more processors, at least one of the one or more converted source images that exceed one or more image parameter thresholds (Kadu, ¶0019, Dynamic composer metadata generated at least in part through machine learning is used to derive one or more operational parameter values … The one or more operational parameter values of the image-specific backward reshaping mappings are used to backward reshape the SDR image into the mapped HDR image; Kadu, ¶0026, Image metadata transmitted with SDR video contents to a recipient device may include ML composer metadata; Kadu, ¶0027, One of the valuable services provided by the ML composer metadata generation (115) is to generate the HDR images from the SDR images; Kadu, ¶0036, In operational scenarios in which the receiver operates with (or is attached to) an HDR display 140-1 that supports a high dynamic range (e.g., 400 nits, 1000 nits, 4000 nits, 10000 nits or more, etc.), the receiver can extract the composer metadata from (e.g., the metadata container in, etc.) the coded bitstream (122) and use the composer metadata to compose HDR images (132); Kadu, ¶0039, The dynamic composer metadata (or ML-based composer metadata) can be infused with different user-defined (e.g., end user selectable, etc.) styles to modify the HDR look based individual users' respective preferences; Kadu, ¶0042, under the ML based approach of predicting or estimating HDR luma/chroma codewords, relevant image features (e.g., content dependent features, pixel value dependent features, etc.) can be extracted from SDR image data and used to train, predict and/or estimate (dynamic) ML-based composer metadata for constructing or reconstructing HDR image data from the SDR image data. In some operational scenarios, such construction or reconstruction can be further influenced by user input specifying a user selection of a user-intended visual style/mode; Kadu, ¶0153, The selected set of GPR models for the SDR image (282) may be selected from among different sets of GPR models such as 206-1 through 206-N, wherein N is a positive integer greater than one (1). The different sets of GPR models 206-1 through 206-N may be trained by different sets of training SDR-HDR image pairs in one or more different training datasets. For example, for a set of training SDR images, multiple sets of corresponding training HDR images may be generated with each set in the multiple sets of corresponding training HDR images representing a distinct user-desired style or HDR look among multiple user-desired styles or HDR looks represented in the multiple sets of corresponding training HDR images. As used herein, a user desired style or HDR look may refer to a style of images (e.g., HDR images, etc.) as preferred or intended by a user such as a colorist, a professional video creator, a studio, etc.; Kadu, ¶0154, a selected set of GPR models corresponding to a selected user desired style or HDR look may be a single set of GPR models selected (e.g., based on user preferences, system configuration information, etc.) from among the different sets of GPR models 206-1 through 206-N; Kadu, ¶0160, The different sets of SDR-HDR image pairs that support style transfer may comprise (training) HDR images of (or tailored to) different HDR looks or different user defined styles. For example, a first set of SDR-HDR image pairs in a first training dataset of the training datasets may correspond a first HDR look or a first user defined style, whereas a second different set of SDR-HDR image pairs in a second training dataset of the training datasets may correspond to a second different HDR look or a second different user defined style; Kadu, ¶0161-0165); and
storing, by the one or more processors, the at least one converted source image that exceeds the one or more parameter thresholds in a training dataset for a machine-learning model (Kadu, ¶0043, Training ML prediction models/algorithms/methods under techniques as described herein may be done with a training dataset comprising pairs of SDR and corresponding (e.g., user-desired, manually color graded, etc.) HDR images. The ML prediction models/algorithms/methods can learn (e.g., user-intended, etc.) SDR-to-HDR mappings during the training phase. Machine learnt optimal operational parameters for the ML prediction models/algorithms/methods can be stored persistently or in cache/memory).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Kadu and Oh because they are in the same field of endeavor. One skilled in the art would have been motivated to include the machine learning algorithms as taught by Kadu in the system of Oh in order to optimize and customize the output images (Kadu, ¶0028; ¶0039).
As per claim 9, Oh and Kadu disclose the computer system of claim 8, the computer system further comprising: generating, by the one or more processors, a profile-based model for the machine-learning algorithm based on the training dataset and data from the profile (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; and a processor configured to acquire image quality information applied to the content based on the metadata, to convert the input content into a converted content; Oh, ¶0023, a method of outputting a content from a content outputting apparatus to a display includes receiving an input content and metadata associated with the input content; acquiring image quality information applied to the input content based on the metadata; determining content conversion information related to the acquired image quality information; converting the input content into a converted content outputtable on the display using the content conversion information; and outputting the converted content to the display; Kadu, ¶0039, The dynamic composer metadata (or ML-based composer metadata) can be infused with different user-defined (e.g., end user selectable, etc.) styles to modify the HDR look based individual users' respective preferences; Kadu, ¶0043, Training ML prediction models/algorithms/methods under techniques as described herein may be done with a training dataset comprising pairs of SDR and corresponding (e.g., user-desired, manually color graded, etc.) HDR images. The ML prediction models/algorithms/methods can learn (e.g., user-intended, etc.) SDR-to-HDR mappings during the training phase. Machine learnt optimal operational parameters for the ML prediction models/algorithms/methods can be stored persistently or in cache/memory; Kadu, ¶0150-0151, Style Transfer with Encoder … as illustrated in FIG. 2F, different styles (e.g., a style preferring bluish images, a style preferring reddish images, a style of a first colorist preferring strong contrasts, a style of a different colorist preferring softer images, etc.) or different HDR looks can be transferred from an upstream video encoder to downstream video decoder(s) by way of dynamic composer metadata carried in a video signal or coded bitstream encoded with SDR images).
As per claim 10, Oh and Kadu disclose the computer system of claim 9, wherein the one or more image parameter thresholds correspond to at least one technical parameter included in the profile, wherein the profile corresponds to a Color Decision List (CDL), a color Look-Up Table (LUT), color scheme, and/or a genre (Oh, ¶0014, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content, wherein the display may be further configured to gamma-correct the converted content by using gamma information predetermined to be applied to the HDR content; Oh, ¶0106, SDR pixel values of each of R, G, and B pixels of an 8-bit SDR content may exist between 0 and 255, and first gamma information of a plurality of pieces of gamma information applicable to the SDR content may be applied to the SDR content. As shown in FIG. 5, SDR pixel values R0 through R255 corresponding to R pixels of the 8-bit SDR content, to which the first gamma information is applied, may be stored in a lookup table 410. HDR pixel values R′0 through R′255 respectively corresponding to the SDR pixel values R0 through R255 of the R pixels of the 8-bit SDR content may match with the SDR pixel values R0 through R255 corresponding to the R pixels and then may be stored in the lookup table 410. Here, the HDR pixel values R′0 through R′255 may be values for gamma-correcting an SDR convent converted to have a characteristic of an HDR content by using first gamma information of a plurality of pieces of gamma information applicable to the HDR content; Kadu, ¶0161, For each SDR image (282), the different luma and chroma backward reshaping mappings of different HDR looks or different user defined styles may be represented by different BLUTs and/or different sets of MMR coefficients generated based on the different sets of GPR (luma prediction/estimation) models (206-1 through 206-N) and/or by different chroma dictionaries (210-1 through 210-N). The different sets of GPR models and/or the different chroma dictionaries can be respectively trained on different sets of SDR-HDR image pairs of different HDR looks or different user defined styles (in the different training datasets) and then individually (e.g., a selected closest style or look based on a distance measure, etc.) applied to any (e.g., test, to-be-predicted, to-be-backward-reshaped, etc.) SDR image such as the SDR image (282)).
As per claim 11, Oh and Kadu disclose the computer system of claim 8, the computer system further comprising: generating, by the one or more processors, a generic model for the machine-learning algorithm based on the training dataset (Oh, ¶0090, the processor 130 may transmit content information including null information for gamma-correcting the corresponding content in the display apparatus 200 to the display apparatus 200 along with the content converted to have the characteristic of the HDR content. If the content information including the null information is received, the display apparatus 200 may gamma-correct the content converted to have the characteristic of the HDR content by using default gamma information, signal-process the gamma-corrected content in a form that may be output, and display the signal-processed content; Kadu, ¶0019, Example embodiments described herein relate to image metadata generation/optimization through machine learning. An SDR image to be backward reshaped into a corresponding mapped HDR image is decoded from a video signal. Dynamic composer metadata generated at least in part through machine learning is used to derive one or more operational parameter values of image-specific backward reshaping mappings is decoded from the video signal. The one or more operational parameter values of the image-specific backward reshaping mappings are used to backward reshape the SDR image into the mapped HDR image. A display image derived from the mapped HDR image is caused to be rendered with a display device).
As per claim 12, Oh and Kadu disclose the computer of claim 11, wherein the image parameter thresholds include at least one technical parameter corresponding to Standard Dynamic Range (SDR) video content and/or High Dynamic Range (HDR) video content (Oh, ¶0011, Exemplary embodiments provide converting a Standard Dynamic Range (SDR) content into a content corresponding to a High Dynamic Range (HDR) content outputtable on a display apparatus; Kadu, ¶0018, Example embodiments described herein relate to image metadata generation/optimization through machine learning. A plurality of training image pairs comprising a plurality of training SDR image and a plurality of corresponding training HDR images is received. Each training image pair in the plurality of training image pairs comprises a training SDR image in the plurality of training SDR images and a corresponding training HDR image in the plurality of corresponding training HDR images. The training SDR image and the corresponding training HDR image in each such training image pair depict same visual content but with different luminance dynamic ranges. A plurality of training image feature vectors is extracted from a plurality of training SDR images in the plurality of training image pairs. A training image feature vector in the plurality of training image feature vectors is extracted from a training SDR image in a respective training image pair in the plurality of training image pairs. The plurality of training image feature vectors and ground truth derived with the plurality of corresponding training HDR images are used to train one or more backward reshaping metadata prediction models for predicting operational parameter values of backward reshaping mappings used to backward reshape SDR images into mapped HDR images).
As per claim 13, Oh and Kadu disclose the computer system of claim 8, wherein the profile is associated with a profile identifier including a code and/or an address (Oh, ¶0075, a BD may include an SDR content that is encoded through a codec having a MPEG-2, H.264/MPEG-4 AVC, or VC-1 standard. Therefore, the data processor 150 may decode the input SDR content based on the encoding information of the metadata of the SDR content input through the input interface 110; Oh, ¶0106, SDR pixel values of each of R, G, and B pixels of an 8-bit SDR content may exist between 0 and 255, and first gamma information of a plurality of pieces of gamma information applicable to the SDR content may be applied to the SDR content. As shown in FIG. 5, SDR pixel values R0 through R255 corresponding to R pixels of the 8-bit SDR content, to which the first gamma information is applied, may be stored in a lookup table 410. HDR pixel values R′0 through R′255 respectively corresponding to the SDR pixel values R0 through R255 of the R pixels of the 8-bit SDR content may match with the SDR pixel values R0 through R255 corresponding to the R pixels and then may be stored in the lookup table 410. Here, the HDR pixel values R′0 through R′255 may be values for gamma-correcting an SDR convent converted to have a characteristic of an HDR content by using first gamma information of a plurality of pieces of gamma information applicable to the HDR content; Kadu, ¶0171-0174, Style Transfer with Decoder … In some embodiments, the image metadata comprises dynamic composer metadata that carry or comprise one or more specific cluster indexes (e.g., one or more numbers, one or more values, one or more integers, etc.) identifying one or more specific chroma clusters to be combined/fused into a combined/fused chroma cluster for deriving MMR coefficients, which can then be used for performing chroma backward reshaping on the SDR image (282) ... As a result, the user desired style or the HDR look can be controlled at the decoder side).
As per claim 14, Oh and Kadu disclose the computer system of claim 8, wherein converting the one or more source images to High Dynamic Range (HDR) video further comprises: converting, by the one or more processors, the one or more source images from a first color space to a second color space based on the profile (Oh, ¶0013, The input content may include a Standard Dynamic Range (SDR) content, the image quality information applied to the input content may include gamma information, and the converted content outputtable on the display may include a High Dynamic Range (HDR) content; Oh, ¶0105-0106; Kadu, ¶00370038, Dynamic Composer Metadata Generation Through Machine Learning … Single Layer Inverse Display Management (SLiDM) or SDR+ can be used to enhance SDR content for rendering on HDR display devices. Luma and chroma channels (or color components) of SDR images may be mapped separately using image metadata to generate corresponding luma and chroma channels of HDR images; Kadu, ¶0042, under the ML based approach of predicting or estimating HDR luma/chroma codewords, relevant image features (e.g., content dependent features, pixel value dependent features, etc.) can be extracted from SDR image data and used to train, predict and/or estimate (dynamic) ML-based composer metadata for constructing or reconstructing HDR image data from the SDR image data. In some operational scenarios, such construction or reconstruction can be further influenced by user input specifying a user selection of a user-intended visual style/mode).
As per claim 15, Oh discloses a non-transitory computer-readable medium containing instructions for training a machine-learning algorithm for automatic conversion of video content (Oh, ¶0065, the content outputting apparatus 100 includes an input interface 110, a communicator 120, a processor 130, and a storage 140; Oh, ¶0146, an exemplary embodiment may be stored in program form on a non-transitory computer readable medium), the instructions comprising:
receiving, by one or more processors, one or more source images having Standard Dynamic Range (SDR) video content (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; Oh, ¶0013, The input content may include a Standard Dynamic Range (SDR) content; Oh, ¶0066, The input interface 110 receives a content recorded on a recording medium such as a DVD or a BD and metadata of the corresponding content. Here, the content is an SDR content);
receiving, by one or more processors, a profile based on an identity corresponding to the one or more source images (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; and a processor configured to acquire image quality information applied to the content based on the metadata; Oh, ¶0014, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content; Oh, ¶0015, The HDR pixel values may correspond to the SDR pixel values of the image frames of the SDR content, and the processor may be further configured to convert the SDR pixel values of the image frames of the SDR content into the HDR pixel values based on the lookup table; Oh, ¶0074, as shown in FIG. 3, if an SDR content and metadata of the SDR content are input through the input interface 110, a data processor 150 decodes the SDR content based on the input metadata; Oh, ¶0075, a BD may include an SDR content that is encoded through a codec having a MPEG-2, H.264/MPEG-4 AVC, or VC-1 standard. Therefore, the data processor 150 may decode the input SDR content based on the encoding information of the metadata of the SDR content input through the input interface 110);
converting, by the one or more processors executing the machine-learning algorithm, the one or more source images to High Dynamic Range (HDR) video (Oh, ¶0023, a method of outputting a content from a content outputting apparatus to a display includes receiving an input content and metadata associated with the input content; acquiring image quality information applied to the input content based on the metadata; determining content conversion information related to the acquired image quality information; converting the input content into a converted content outputtable on the display using the content conversion information; and outputting the converted content to the display; Oh, ¶0024, The input content may include an SDR content, the image quality information applied to the input content may include gamma information, and the converted content outputtable on the display may include an HDR content; Oh, ¶0025, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content; Oh, ¶0026, The HDR pixel values may correspond to the SDR pixel values of the image frames of the SDR content, and the converting may further include converting the SDR pixel values of the image frames of the SDR content into the HDR pixel values based on the lookup table).
Oh does not explicitly disclose the following limitations as further recited however Kadu discloses
selecting, by the one or more processors, a machine-learning algorithm from among a plurality of machine-learning algorithms based on the profile (Kadu, ¶0045, Under techniques as described herein, a multitude of different ML prediction models/algorithms/methods can be learned for various specific user preferences. Then switching among a plurality of different user-preferred HDR looks can be as easily performed under these techniques as switching among the different ML prediction models/algorithms/methods. As a result, SDR content can be backward reshaped into HDR content with desired HDR looks for users ... Instead of manually color grading HDR images generated from a potentially vast amount of available SDR content, an individual user can provide a training dataset of pairs of SDR and corresponding (e.g., manually, etc.) color graded HDR images. Machine learned models/algorithms/methods under techniques as described herein can automatically transform any SDR content to corresponding HDR content by mimicking user preferences as embodied by the training dataset; Kadu, ¶0150-0151, Style Transfer with Encoder … as illustrated in FIG. 2F, different styles (e.g., a style preferring bluish images, a style preferring reddish images, a style of a first colorist preferring strong contrasts, a style of a different colorist preferring softer images, etc.) or different HDR looks can be transferred from an upstream video encoder to downstream video decoder(s) by way of dynamic composer metadata);
selecting, by the one or more processors, at least one of the one or more converted source images that exceed one or more image parameter thresholds (Kadu, ¶0019, Dynamic composer metadata generated at least in part through machine learning is used to derive one or more operational parameter values … The one or more operational parameter values of the image-specific backward reshaping mappings are used to backward reshape the SDR image into the mapped HDR image; Kadu, ¶0026, Image metadata transmitted with SDR video contents to a recipient device may include ML composer metadata; Kadu, ¶0027, One of the valuable services provided by the ML composer metadata generation (115) is to generate the HDR images from the SDR images; Kadu, ¶0036, In operational scenarios in which the receiver operates with (or is attached to) an HDR display 140-1 that supports a high dynamic range (e.g., 400 nits, 1000 nits, 4000 nits, 10000 nits or more, etc.), the receiver can extract the composer metadata from (e.g., the metadata container in, etc.) the coded bitstream (122) and use the composer metadata to compose HDR images (132); Kadu, ¶0039, The dynamic composer metadata (or ML-based composer metadata) can be infused with different user-defined (e.g., end user selectable, etc.) styles to modify the HDR look based individual users' respective preferences; Kadu, ¶0042, under the ML based approach of predicting or estimating HDR luma/chroma codewords, relevant image features (e.g., content dependent features, pixel value dependent features, etc.) can be extracted from SDR image data and used to train, predict and/or estimate (dynamic) ML-based composer metadata for constructing or reconstructing HDR image data from the SDR image data. In some operational scenarios, such construction or reconstruction can be further influenced by user input specifying a user selection of a user-intended visual style/mode; Kadu, ¶0153, The selected set of GPR models for the SDR image (282) may be selected from among different sets of GPR models such as 206-1 through 206-N, wherein N is a positive integer greater than one (1). The different sets of GPR models 206-1 through 206-N may be trained by different sets of training SDR-HDR image pairs in one or more different training datasets. For example, for a set of training SDR images, multiple sets of corresponding training HDR images may be generated with each set in the multiple sets of corresponding training HDR images representing a distinct user-desired style or HDR look among multiple user-desired styles or HDR looks represented in the multiple sets of corresponding training HDR images. As used herein, a user desired style or HDR look may refer to a style of images (e.g., HDR images, etc.) as preferred or intended by a user such as a colorist, a professional video creator, a studio, etc.; Kadu, ¶0154, a selected set of GPR models corresponding to a selected user desired style or HDR look may be a single set of GPR models selected (e.g., based on user preferences, system configuration information, etc.) from among the different sets of GPR models 206-1 through 206-N; Kadu, ¶0160, The different sets of SDR-HDR image pairs that support style transfer may comprise (training) HDR images of (or tailored to) different HDR looks or different user defined styles. For example, a first set of SDR-HDR image pairs in a first training dataset of the training datasets may correspond a first HDR look or a first user defined style, whereas a second different set of SDR-HDR image pairs in a second training dataset of the training datasets may correspond to a second different HDR look or a second different user defined style; Kadu, ¶0161-0165); and
storing, by the one or more processors, the at least one converted source image that exceeds the one or more parameter thresholds in a training dataset for a machine-learning model (Kadu, ¶0043, Training ML prediction models/algorithms/methods under techniques as described herein may be done with a training dataset comprising pairs of SDR and corresponding (e.g., user-desired, manually color graded, etc.) HDR images. The ML prediction models/algorithms/methods can learn (e.g., user-intended, etc.) SDR-to-HDR mappings during the training phase. Machine learnt optimal operational parameters for the ML prediction models/algorithms/methods can be stored persistently or in cache/memory).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Kadu and Oh because they are in the same field of endeavor. One skilled in the art would have been motivated to include the machine learning algorithms as taught by Kadu in the system of Oh in order to optimize and customize the output images (Kadu, ¶0028; ¶0039).
As per claim 16, Oh and Kadu disclose the non-transitory computer-readable medium of claim 15, the non-transitory computer-readable medium further comprising: generating, by the one or more processors, a profile-based model for the machine-learning algorithm based on the training dataset and data from the profile (Oh, ¶0012, an input interface configured to receive an input content and metadata associated with the input content; and a processor configured to acquire image quality information applied to the content based on the metadata, to convert the input content into a converted content; Oh, ¶0023, a method of outputting a content from a content outputting apparatus to a display includes receiving an input content and metadata associated with the input content; acquiring image quality information applied to the input content based on the metadata; determining content conversion information related to the acquired image quality information; converting the input content into a converted content outputtable on the display using the content conversion information; and outputting the converted content to the display; Kadu, ¶0039, The dynamic composer metadata (or ML-based composer metadata) can be infused with different user-defined (e.g., end user selectable, etc.) styles to modify the HDR look based individual users' respective preferences; Kadu, ¶0043, Training ML prediction models/algorithms/methods under techniques as described herein may be done with a training dataset comprising pairs of SDR and corresponding (e.g., user-desired, manually color graded, etc.) HDR images. The ML prediction models/algorithms/methods can learn (e.g., user-intended, etc.) SDR-to-HDR mappings during the training phase. Machine learnt optimal operational parameters for the ML prediction models/algorithms/methods can be stored persistently or in cache/memory; Kadu, ¶0150-0151, Style Transfer with Encoder … as illustrated in FIG. 2F, different styles (e.g., a style preferring bluish images, a style preferring reddish images, a style of a first colorist preferring strong contrasts, a style of a different colorist preferring softer images, etc.) or different HDR looks can be transferred from an upstream video encoder to downstream video decoder(s) by way of dynamic composer metadata carried in a video signal or coded bitstream encoded with SDR images).
As per claim 17, Oh and Kadu disclose the non-transitory computer-readable medium of claim 16, wherein the one or more image parameter thresholds correspond to at least one technical parameter included in the profile, wherein the profile corresponds to a Color Decision List (CDL), a color Look-Up Table (LUT), color scheme, and/or a genre (Oh, ¶0014, The content conversion information may include a lookup table for converting SDR pixel values of image frames of the SDR content into HDR pixel values having a characteristic of the HDR content, wherein the display may be further configured to gamma-correct the converted content by using gamma information predetermined to be applied to the HDR content; Oh, ¶0106, SDR pixel values of each of R, G, and B pixels of an 8-bit SDR content may exist between 0 and 255, and first gamma information of a plurality of pieces of gamma information applicable to the SDR content may be applied to the SDR content. As shown in FIG. 5, SDR pixel values R0 through R255 corresponding to R pixels of the 8-bit SDR content, to which the first gamma information is applied, may be stored in a lookup table 410. HDR pixel values R′0 through R′255 respectively corresponding to the SDR pixel values R0 through R255 of the R pixels of the 8-bit SDR content may match with the SDR pixel values R0 through R255 corresponding to the R pixels and then may be stored in the lookup table 410. Here, the HDR pixel values R′0 through R′255 may be values for gamma-correcting an SDR convent converted to have a characteristic of an HDR content by using first gamma information of a plurality of pieces of gamma information applicable to the HDR content; Kadu, ¶0161, For each SDR image (282), the different luma and chroma backward reshaping mappings of different HDR looks or different user defined styles may be represented by different BLUTs and/or different sets of MMR coefficients generated based on the different sets of GPR (luma prediction/estimation) models (206-1 through 206-N) and/or by different chroma dictionaries (210-1 through 210-N). The different sets of GPR models and/or the different chroma dictionaries can be respectively trained on different sets of SDR-HDR image pairs of different HDR looks or different user defined styles (in the different training datasets) and then individually (e.g., a selected closest style or look based on a distance measure, etc.) applied to any (e.g., test, to-be-predicted, to-be-backward-reshaped, etc.) SDR image such as the SDR image (282)).
As per claim 18, Oh and Kadu disclose the non-transitory computer-readable medium of claim 15, the non-transitory computer-readable medium further comprising: generating, by the one or more processors, a generic model for the machine-learning algorithm based on the training dataset (Oh, ¶0090, the processor 130 may transmit content information including null information for gamma-correcting the corresponding content in the display apparatus 200 to the display apparatus 200 along with the content converted to have the characteristic of the HDR content. If the content information including the null information is received, the display apparatus 200 may gamma-correct the content converted to have the characteristic of the HDR content by using default gamma information, signal-process the gamma-corrected content in a form that may be output, and display the signal-processed content; Kadu, ¶0019, Example embodiments described herein relate to image metadata generation/optimization through machine learning. An SDR image to be backward reshaped into a corresponding mapped HDR image is decoded from a video signal. Dynamic composer metadata generated at least in part through machine learning is used to derive one or more operational parameter values of image-specific backward reshaping mappings is decoded from the video signal. The one or more operational parameter values of the image-specific backward reshaping mappings are used to backward reshape the SDR image into the mapped HDR image. A display image derived from the mapped HDR image is caused to be rendered with a display device).
As per claim 19, Oh and Kadu disclose the non-transitory computer-readable medium of claim 18, wherein the image parameter thresholds include at least one technical parameter corresponding to Standard Dynamic Range (SDR) video content and/or High Dynamic Range (HDR) video content (Oh, ¶0011, Exemplary embodiments provide converting a Standard Dynamic Range (SDR) content into a content corresponding to a High Dynamic Range (HDR) content outputtable on a display apparatus; Kadu, ¶0018, Example embodiments described herein relate to image metadata generation/optimization through machine learning. A plurality of training image pairs comprising a plurality of training SDR image and a plurality of corresponding training HDR images is received. Each training image pair in the plurality of training image pairs comprises a training SDR image in the plurality of training SDR images and a corresponding training HDR image in the plurality of corresponding training HDR images. The training SDR image and the corresponding training HDR image in each such training image pair depict same visual content but with different luminance dynamic ranges. A plurality of training image feature vectors is extracted from a plurality of training SDR images in the plurality of training image pairs. A training image feature vector in the plurality of training image feature vectors is extracted from a training SDR image in a respective training image pair in the plurality of training image pairs. The plurality of training image feature vectors and ground truth derived with the plurality of corresponding training HDR images are used to train one or more backward reshaping metadata prediction models for predicting operational parameter values of backward reshaping mappings used to backward reshape SDR images into mapped HDR images).
As per claim 20, Oh and Kadu disclose the non-transitory computer-readable medium of claim 15, wherein the profile is associated with a profile identifier including a code and/or an address (Oh, ¶0075, a BD may include an SDR content that is encoded through a codec having a MPEG-2, H.264/MPEG-4 AVC, or VC-1 standard. Therefore, the data processor 150 may decode the input SDR content based on the encoding information of the metadata of the SDR content input through the input interface 110; Oh, ¶0106, SDR pixel values of each of R, G, and B pixels of an 8-bit SDR content may exist between 0 and 255, and first gamma information of a plurality of pieces of gamma information applicable to the SDR content may be applied to the SDR content. As shown in FIG. 5, SDR pixel values R0 through R255 corresponding to R pixels of the 8-bit SDR content, to which the first gamma information is applied, may be stored in a lookup table 410. HDR pixel values R′0 through R′255 respectively corresponding to the SDR pixel values R0 through R255 of the R pixels of the 8-bit SDR content may match with the SDR pixel values R0 through R255 corresponding to the R pixels and then may be stored in the lookup table 410. Here, the HDR pixel values R′0 through R′255 may be values for gamma-correcting an SDR convent converted to have a characteristic of an HDR content by using first gamma information of a plurality of pieces of gamma information applicable to the HDR content; Kadu, ¶0171-0174, Style Transfer with Decoder … In some embodiments, the image metadata comprises dynamic composer metadata that carry or comprise one or more specific cluster indexes (e.g., one or more numbers, one or more values, one or more integers, etc.) identifying one or more specific chroma clusters to be combined/fused into a combined/fused chroma cluster for deriving MMR coefficients, which can then be used for performing chroma backward reshaping on the SDR image (282) ... As a result, the user desired style or the HDR look can be controlled at the decoder side).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-9 and 14 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 4-10, 12-14 and 16-20 of U.S. Patent No. 12,133,030. Although the claims at issue are not identical, they are not patentably distinct from each other because claims 1-9 and 14 of the current application are encompassed by the scope of claims 1, 2, 4-10, 12-14 and 16-20 of U.S. Patent No. 12,133,030 in that claims 1, 2, 4-10, 12-14 and 16-20 of U.S. Patent No. 12,133,030 contain similar limitations as claims 1-9 and 14 of the current application and are therefore not patentably distinct from claims 1, 2, 4-10, 12-14 and 16-20 of U.S. Patent No. 12,133,030.
Claims 1-9 and 14 of the current application recite similar limitations as claims 1, 2, 4-10, 12-14 and 16-20 of U.S. Patent No. 12,133,030 as follows:
Current Application No. 18/901,126
U.S. Patent No. 12,133,030
1. A computer-implemented method for training a machine-learning algorithm for automatically converting video content, the computer-implemented method comprising: receiving, by one or more processors, one or more source images having Standard Dynamic Range (SDR) video content; receiving, by one or more processors, a profile based on an identity corresponding to the one or more source images; selecting, by the one or more processors, a machine-learning algorithm from among a plurality of machine-learning algorithms based on the profile; converting, by the one or more processors executing the machine-learning algorithm, the one or more source images to High Dynamic Range (HDR) video; selecting, by the one or more processors, at least one of the one or more converted source images that exceed one or more image parameter thresholds; and storing, by the one or more processors, the at least one converted source image that exceeds the one or more parameter thresholds in a training dataset for a machine-learning model.
7. The computer-implement method of claim 1, wherein converting the one or more source images to High Dynamic Range (HDR) video further comprises converting, by the one or more processors, the one or more source images from a first color space to a second color space based on the profile.
1. A computer-implemented method for automatic conversion of video content, the method comprising: identifying, by one or more processors, a creative profile for source video content constrained to a first color space, wherein the creative profile specifies (i) a creative style associated with the source video content and (ii) a machine-learning algorithm associated with the creative style; selecting, by the one or more processors, the machine-learning algorithm from among a plurality of machine-learning algorithms based on the creative profile, wherein the machine-learning algorithm is trained to convert content from the first color space to a second color space according to the creative style; and converting, by the one or more processors executing the machine-learning algorithm, the source video content to converted video content constrained to the second color space, wherein the converted video content is consistent with the creative style.
2. The method of claim 1, wherein the source video content is Standard Dynamic Range (SDR) video content and the converted video content is High Dynamic Range (HDR) video content.
7. The method of claim 4, further comprising populating the training set with the source images, wherein the source images comprise source video content and wherein the corresponding converted images comprise HDR video content converted from the source video content.
8. The method of claim 7, wherein the source video content is selected from at least one of raw video content and SDR video content.
9. The method of claim 7, wherein the populating further comprises determining that each of the corresponding converted images is approved or meets or exceeds a parameter threshold for a technical parameter in the creative profile.
2. The computer-implemented method of claim 1, the computer-implemented method further comprising: generating, by the one or more processors, a profile-based model for the machine-learning algorithm based on the training dataset and data from the profile.
4. The method of claim 1, further comprising training the machine-learning algorithm over a training set comprising of source images and corresponding converted images matching the creative profile.
6. The method of claim 4, wherein the training set further comprises one or more machine-readable data of the creative profile.
3. The computer-implemented method of claim 2, wherein the one or more image parameter thresholds correspond to at least one technical parameter included in the profile, wherein the profile corresponds to a Color Decision List (CDL), a color Look-Up Table (LUT), color scheme, and/or a genre.
12. The method of claim 1, wherein the creative profile comprises machine-readable data associating the source video content with a genre of video content.
13. The method of claim 1, wherein the creative profile comprises machine-readable data associating the source video content with at least one of a Color Decision List (CDL) or a color Look-Up Table (LUT).
14. The method of claim 1, wherein the creative profile comprises machine-readable data associating the source video content with a scheme for at least one of, color tones, contrast ranges, or black level preferences.
4. The computer-implemented method of claim 1, the computer-implemented method further comprising: generating, by the one or more processors, a generic model for the machine-learning algorithm based on the training dataset.
5. The method of claim 4, further comprising, prior to the training over the training set, training the machine-learning algorithm over a generic training set including content matching multiple creative profiles.
5. The computer-implemented method of claim 4, wherein the image parameter thresholds include at least one technical parameter corresponding to Standard Dynamic Range (SDR) video content and/or High Dynamic Range (HDR) video content.
9. The method of claim 7, wherein the populating further comprises determining that each of the corresponding converted images is approved or meets or exceeds a parameter threshold for a technical parameter in the creative profile.
6. The computer-implemented method of claim 1, wherein the profile is associated with a profile identifier including a code and/or an address.
10. The method of claim 1, wherein the creative profile comprises machine-readable data associating the source video content with at least one personal identity.
Claims 16-19 of U.S. Patent No. 12,133,030 corresponds to claims 8 and 14 of the current application. Claim 20 of U.S. Patent No. 12,133,030 corresponds to claim 9 of the current application.
The table above shows that, although the corresponding claims are not identical, claims 1-9 and 14 of the current invention are not patentably distinct from claims 1, 2, 4-10, 12-14 and 16-20 of U.S. Patent No. 12,133,030.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Primary Examiner, Art Unit 2668
Prosecution Insights