DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-2, 4-11 and 13-16 are pending in the application. Claims 1, 5, 7-11, 13 and 16 have been amended and claims 3 and 12 have been canceled.
The amendment filed 9/26/25 overcomes objections applied to claims 5 and 10-11.
Claim interpretation under 35 USC 112(f) for claim elements in claims 7 and 10-11 is no longer applicable in view of claim amendment.
The amendment filed 9/26/25 overcomes rejections under 35 USC 101 applied to claim 16.
Claim Objections
The following claims are objected to.
Claim 5 should be dependent upon claim 1 rather than claim 3 since claim 3 has been canceled.
Claim 710th line “the RAW data” has no antecedent basis.
Response to Arguments
Applicant’s arguments, filed 9/26/25, with respect to art rejections applied to claim(s) 1 and 7 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Note due to claim amendment applied to claims 7 and 10-11, 112(f) claim interpretation is not applicable any longer. The scope of claims 7 and 10-11 has been changed.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 5-7, 9 and 13-16 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Brooks et al. (Brooks T, Mildenhall B, Xue T, Chen J, Sharlet D, Barron JT. Unprocessing images for learned raw denoising. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 (pp. 11036-11045), hereafter Brooks).
As per claim 1, Brooks teaches a data processing method (ABSTARCT), comprising:
obtaining reference data that comprises RGB image data and a plurality of device parameters of an image device (Brooks teaches a method for denoising RAW image data (Abstract). The noisy RAW image data is obtained by “unprocessing” images by inverting each step of an image processing pipeline, thereby allowing realistic raw sensor measurements to be synthesized from commonly available Internet photos (Abstract). As shown in Fig. 2, the “Unprocess” converts a reference RGB image into a RAW image. The “Unprocess” includes several steps, such as inverting tone mapping (described in Section 3.7), applying gamma decompression (see Section 3.6), applying the sRGB to camera RGB color correction matrix (see Section 3.5), and inverting white balance gains (see Section 3.4) and digital gain (see Section 3.3). Further a noisy Raw image is obtained by adding noise to an unprocessed Raw image (see Section 3.1). Brooks also describes an inverse demosaicing in Section 3.2. During the unprocess, a plurality of device parameters of an image sensor is used, such as sensor noise (Section 3.1), demosaicing (Section 3.2), digital gain (Section 3.3), white balance (Section 3.4), color correction matrix (CCM) (Section 3.5), gamma compression (Section 3.6) and tone mapping (Section 3.7));
determining, based on the plurality of device parameters, a search space corresponding to the plurality of device parameters and the image device (As described above, a plurality of device parameters of an image sensor is used, such as sensor noise (Section 3.1), demosaicing (Section 3.2), digital gain (Section 3.3), white balance (Section 3.4), color correction matrix (CCM) (Section 3.5), gamma compression (Section 3.6) and tone mapping (Section 3.7). The space corresponding to these device parameters is considered a search space, i.e., the search space corresponds to sensor noise space, demosaicing space, camera digital gain space, camera white balance space, camera sensor color correction space, sensor gamma curve space, and sensor tone mapping space.);
determining, from the determined search space, a plurality of conversion parameters that correspond to the plurality of device parameters for converting the RGB image data into RAW data (As described in Sections 3.1-3.7, when inverting the respective steps, the parameters used in the conversion are conversion parameters.); and
processing the RGB image data into the RAW data based on the plurality of conversion parameters, wherein the RAW data matches the plurality of device parameters of the image device (See Sections 3.1-3.7 for the conversion; FIG. 2 “Unprocess” and “Add Shot and Read Noise”).
As per claim 5, dependent upon claim 1, Brooks teaches a search space corresponding to the image device comprises a plurality of image processing modules (Fig. 2 “Unprocess” and “Add Shot and Read Noise”); and the image processing modules comprise one or more of the following: a noise addition module (Fig. 2 “Add Shot and Read Noise”; Section 3.1), a mosaic addition module (section 3.2), or a brightness adjustment module (Section 3.7).
As per claim 6, dependent upon claim 1, Brooks teaches the conversion parameters comprise one or more of the following: a noise addition parameter (Fig. 2 “Add Shot and Read Noise”; Section 3.1), a mosaic addition parameter (Section 3.2), a brightness adjustment parameter (Section 3.7), a gamma parameter (Section 3.6), a level adjustment parameter (Section 3.3), and a white balance adjustment parameter (Section 3.4).
As per claim 7, Brooks teaches a data processing system, comprising:
a memory (A memory is inherently taught since Brooks discloses a computer-implemented method); and
at least one processor (page 11041 right column Section 4.4 “Our models and ablations are trained to convergence over approximately 3.5 million steps on a single NVIDIA Tesla P100 GPU, which takes ∼3 days”; page 11042 Table 1 “Runtime”), coupled with the memory, configured to:
obtain reference data that comprises RGB image data and a device parameter of an image device (See rejections to corresponding limitations in claim 1);
determine a plurality of conversion parameters for converting the RGB image data into the RAW data (See rejections to corresponding limitations in claim 1); and
processing the RGB image data into the RAW data based on the plurality of conversion parameters, wherein the RAW data matches the device parameter of the image device (See rejections to corresponding limitations in claim 1).
As per claim 9, dependent upon claim 7, Brooks teaches outputting an image pair of the RGB image data and the RAW data (Fig. 2 “sRGB Training Image” and noisy “RAW Image”).
Claim 13, dependent upon claim 7, is rejected as applied to claim 5 above.
Claim 14, dependent upon claim 7, is rejected as applied to claim 6 above.
Claim 15, an apparatus claim, is rejected as applied to claim 1 and claim 7 above. As applied to claim 7, Brooks teaches a processor and memory for executing a computer-implemented method as recited in claim 1. A computer program is inherently taught.
Claim 16, a medium claim, is rejected as applied to claim 1 and claim 7 above. As applied to claim 7, Brooks teaches a processor and memory for executing a computer-implemented method as recited in claim 1. A computer-readable storage medium storing computer instructions is inherently taught.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Brooks, in view of Brown et al. (US Publication 2021/0297582 A1, hereafter Brown).
As per claim 2, Brooks discloses using generated RAW image data for training a machine learning model for denoising the RAW image data (Abstract; Fig. 2). The claimed limitations, however, are not taught.
Brown discloses a method for customizing camera parameters (Abstract). In order to determine preferred camera parameters for converting raw camera image into a preferred image, such as an edited image, an original image of the edited image is obtained, a raw RGB image corresponding to the original image is derived, and a mapping between the raw RGB image and the edited image is generated (Abstract; FIG. 1). Brown further teaches several methods for deriving the raw RGB image from an output displayed RGB image. When metadata indicative of processing performed by the image signal processor is not available, Brown discloses an artificial intelligence-based neural network for converting a RGB image into a raw RGB image (para. [0143]). Specifically, image pairs of RGB images (FIG. 5D 452-1- 452-q) and corresponding raw RGB images (FIG. 5D 454-1- 454-q) are used as training data for training a neural network (#450) for converting an input RGB image (#456; para. [0005] “The display-referred image may be encoded according to an industry standard color space, such as sRGB, Adobe RGB or ProPhoto RGB”) into a raw RGB image (#458). See description in para. [0103]-[0104]. The conversion parameters between the input RGB image and the raw RGB image are inherently taught.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teaching of Brooks to incorporate the teaching of Brown to apply a machine learning model to determine the plurality of conversion parameters. Doing so would allow conversion between an arbitrary input RGB image and its raw RGB image to be applicable as recognized by Brown (para. [0143]).
Claim 8, dependent upon claim 7, is rejected as applied to claim 2 above.
Claim(s) 4 and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Brooks, in view of Brown et al. (US Publication 2021/0297582 A1, hereafter Brown) and MATSUURA et al. (US Publication 2020/0311878 A1, hereafter MATSUURA).
As per claim 4, dependent upon claim 1, Brooks discloses using generated RAW image data for training a machine learning model for denoising the RAW image data (Abstract; Fig. 2). The claimed limitations, however, are not taught.
Brown discloses a method for customizing camera parameters (Abstract). In order to determine preferred camera parameters for converting raw camera image into a preferred image, such as an edited image, an original image of the edited image is obtained, a raw RGB image corresponding to the original image is derived, and a mapping between the raw RGB image and the edited image is generated (Abstract; FIG. 1). Brown further teaches several methods for deriving the raw RGB image from an output displayed RGB image. When metadata indicative of processing performed by the image signal processor is not available, Brown discloses an artificial intelligence-based neural network for converting a RGB image into a raw RGB image (para. [0143]). Specifically, Brown teaches:
constructing an image pair of RGB image data and RAW data;
inputting the image pair into a task processor for training, and determining a feedback signal, wherein the task processor is configured to process video or image data; and
updating, based on the feedback signal, the plurality of conversion parameters for converting the RGB image data into the RAW data (Brown FIG. 5D 452-1- 452-q and 454-1- 454-q are image pairs for training a neural network (#450) for converting an input RGB image (#456; para. [0005] “The display-referred image may be encoded according to an industry standard color space, such as sRGB, Adobe RGB or ProPhoto RGB”) into a raw RGB image (#458). See description in para. [0103]-[0104]; FIG. 2 showing a task processor; Determining a feedback signal and updating, based on the feedback signal, the plurality of conversion parameters for converting the RGB image data into the RAW data is inherently taught during training the neural network).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teaching of Brooks to incorporate the teaching of Brown to apply a machine learning model to determine and update the plurality of conversion parameters. Doing so would allow conversion between an arbitrary input RGB image and its raw RGB image to be applicable as recognized by Brown (para. [0143]).
Brooks in view of Brown does not further teach the feedback signal indicates construction quality of the image pair.
MATSUURA discloses a method for improving reconstructed medical image quality using a neural network (Abstract; FIG. 2B). Specifically, as shown in FIG. 2A and 2B, during training, an image pair, including an input image 153 (low-quality) and a target image 157 (high quality) is input into a neural network (DCNN) for training the neural network. The neural network produce images resembling the target images 153 from the input images 157. During training, a loss function (feedback signal), which represents the difference between an inferenced image and the target image (ground truth) is generated and backpropagated to optimized the parameters of the neural network by minimizing the mean-squared-error-based cost function using a (stochastic) gradient descent method (FIG. 4; para. [0080]-[0084]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Brooks and Brown to incorporate MATSUURA’s feedback signal indicative of construction quality of the image pair. Do so would allow a loss function to be applied to optimize the parameters of the neural network as recognized by MATSUURA (para. [0085]).
As per claim 10, dependent upon claim 9, Brooks in view of Brown and MATSUURA further teaches:
performing training on the image pair of the RGB image data and the RAW data that is output, and determine a feedback signal that indicates construction quality of the image pair (See rejections applied to claim 4 above).
As per claim 11, dependent upon claim 10, Brooks in view of Brown and MATSUURA further teaches:
receiving the feedback signal output, and adjusting a network parameter based on the feedback signal (MATSUURA FIG. 4 para. [0085]); and updating the plurality of conversion parameters based on the adjusted network parameter (See analysis applied to claim 4 above; Brown para. [0103]-[0104]; FIG. 5D).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUEMEI G CHEN whose telephone number is (571)270-3480. The examiner can normally be reached Monday-Friday 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XUEMEI G CHEN/Primary Examiner, Art Unit 2661
Prosecution Insights