DETAILED ACTION
This Office Action is in response to Applicant's Communication received on 04/24/2024 for application number 18/644,636.
Claims 1-11 are presented for examination. Claims 1, 10, and 11 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a first storage unit”, “a data conversion unit”, “a second storage unit”, and “a training unit” in Claim 10.
A review of the specification shows that the following appears to be the corresponding structure described in the specification for the 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph limitation: [0012] a deep learning model training system including: a first storage unit configured to store obtained visual data; a data conversion unit configured to generate training data for a plurality of visual intelligence tasks in a batch from visual data stored in the first storage unit; a second storage unit configured to store the generated training data; and a training unit configured to train a multi-task integrated deep learning model which performs a plurality of visual intelligence tasks by using the training data stored in the second storage unit; [0025] FIG. 1 is a view illustrating a configuration of a multi-task integrated deep learning model training system; the training system according to an embodiment may include a visual data repository 110, data conversion kernels 120, a multi-task training data repository 130, a data loading module 140, a multi-task integrated deep learning model 150, and a loss function calculation module 160; [0049] the computer-readable recording medium may be any data storage device that can be read by a computer and can store data; the computer-readable recording medium may be a read only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as failing to set forth the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant(s) regard as their invention.
Claim limitations “a data conversion unit” and “a training unit” invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function.
The claim limitations “a data conversion unit” and “a training unit” are not clearly defined in the specification.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Lin et al. (US 2022/0301298 A1 hereinafter Lin).
Regarding Claim 1, Lin teaches a deep learning model training method ([0042] the system performs the process 200 to train the image representation neural network to generate feature representations that can be used to achieve high-quality performance for each computer vision task in a set of multiple different computer vision tasks; [0021] the image representation neural network 110 can be a convolutional neural network, e.g., one that has a ResNet or Inception-based architecture) comprising:
obtaining visual data ([0043] the system obtains initial training data for training the image representation neural network; [0044] the initial training data includes multiple different training images; [0046] the initial training data can include a data set of unlabeled images);
generating training data for a plurality of visual intelligence tasks from the obtained visual data in a batch ([0048] the system generates, from the initial training data, final training data for training the image representation neural network; [0050] the system processes the training image using a teacher neural network for the given task to generate a pseudo-label and then uses the pseudo-label as the label for the training image for the given task; [0058] in the final training data, each training image from the initial training data is associated with a respective label for each of the multiple tasks; [0060] at each training step, the system samples a batch of training data from the final training data - thus, generating batches of training data); and
training a multi-task integrated deep learning model which performs a plurality of visual intelligence tasks by using the generated training data ([0059] the system trains the image representation neural network jointly with a respective output neural network head for each of the multiple tasks on the final training data; [0024]-[0030] the multiple different computer vision tasks can include any of a variety of tasks; the tasks can include one or more image classification task, object detection tasks, instance segmentation tasks, etc.).
Regarding Claim 11, Lin teaches a deep learning model training method ([0042] the system performs the process 200 to train the image representation neural network to generate feature representations that can be used to achieve high-quality performance for each computer vision task in a set of multiple different computer vision tasks; [0021] the image representation neural network 110 can be a convolutional neural network, e.g., one that has a ResNet or Inception-based architecture) comprising:
generating training data for a plurality of visual intelligence tasks from visual data in a batch ([0043] the system obtains initial training data for training the image representation neural network; [0044] the initial training data includes multiple different training images; [0046] the initial training data can include a data set of unlabeled images; [0048] the system generates, from the initial training data, final training data for training the image representation neural network; [0050] the system processes the training image using a teacher neural network for the given task to generate a pseudo-label and then uses the pseudo-label as the label for the training image for the given task; [0058] in the final training data, each training image from the initial training data is associated with a respective label for each of the multiple tasks; [0060] at each training step, the system samples a batch of training data from the final training data - thus, generating batches of training data); and
training a multi-task integrated deep learning model which performs a plurality of visual intelligence tasks by using the generated training data ([0059] the system trains the image representation neural network jointly with a respective output neural network head for each of the multiple tasks on the final training data; [0024]-[0030] the multiple different computer vision tasks can include any of a variety of tasks; the tasks can include one or more image classification task, object detection tasks, instance segmentation tasks, etc.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 2022/0301298 A1 hereinafter Lin).
Regarding Claim 10, Lin teaches a deep learning model training system ([0015] a system implemented as computer programs on one or more computers in one or more locations; [0084] the subject matter described in this specification can be implemented as one or more computer programs encoded on a tangible non transitory storage medium for execution by, or to control the operation of, data processing apparatus; [0042] the system performs the process 200 to train the image representation neural network to generate feature representations that can be used to achieve high-quality performance for each computer vision task in a set of multiple different computer vision tasks; [0021] the image representation neural network 110 can be a convolutional neural network, e.g., one that has a ResNet or Inception-based architecture) comprising:
a first storage unit configured to store obtained visual data ([0087] collection of data… can be stored on storage devices in one or more locations; [0090] one or more mass storage devices for storing data (i.e., first storage unit); [0043] the system obtains initial training data for training the image representation neural network; [0044] the initial training data includes multiple different training images; [0046] the initial training data can include a data set of unlabeled images);
a data conversion unit configured to generate training data for a plurality of visual intelligence tasks in a batch from visual data stored in the first storage unit ([0041] process performed by a system of one or more computers (i.e., data conversion unit) located in one or more locations; the training system appropriately programmed, can perform the process; [0048] the system generates, from the initial training data, final training data for training the image representation neural network; [0050] the system processes the training image using a teacher neural network for the given task to generate a pseudo-label and then uses the pseudo-label as the label for the training image for the given task; [0058] in the final training data, each training image from the initial training data is associated with a respective label for each of the multiple tasks; [0060] at each training step, the system samples a batch of training data from the final training data - thus, generating batches of training data);
a second storage unit configured to store the generated training data ([0060] at each training step, the system samples a batch of training data from the final training data; [0087] collection of data… can be stored on storage devices in one or more locations; [0090] one or more mass storage devices for storing data (i.e., second storage unit)); and
a training unit configured to train a multi-task integrated deep learning model which performs a plurality of visual intelligence tasks by using the training data stored in the second storage unit ([0041] process performed by a system of one or more computers located in one or more locations; the training system (i.e., training unit) appropriately programmed, can perform the process; [0059] the system trains the image representation neural network jointly with a respective output neural network head for each of the multiple tasks on the final training data; [0024]-[0030] the multiple different computer vision tasks can include any of a variety of tasks; the tasks can include one or more image classification task, object detection tasks, instance segmentation tasks, etc.).
Claims 2-6 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Saharia et al. (US 2023/0103638 A1 hereinafter Saharia).
As to dependent Claim 2, Lin teaches all the limitations of claim 1. However, Lin fails to expressly teach wherein generating visual data from the obtained visual data through corresponding kernels, respectively, before corresponding visual intelligence tasks are performed, and thereby generating input data of the multi-task integrated deep learning model, and wherein the obtained visual data is labeled data regarding the input data.
In the same field of endeavor, Saharia teaches wherein generating visual data from the obtained visual data through corresponding kernels, respectively, before corresponding visual intelligence tasks are performed, and thereby generating input data of the multi-task integrated deep learning model ([0043] the model described herein can perform operations including colorization, inpainting, and de-blurring; [0051] the model described herein referred to as “Palette; [0089] RGB parameterization may be used for colorization; the grayscale image may be used as the source image and Palette may be trained to predict the full RGB image - thus, a greyscale conversion (equivalent to colorization kernel) generates the source/ input image from the obtained color image before the colorization task is performed; [0090] for inpainting tasks, Palette may be trained on a combination of free-form and rectangular masks; the total area covered by the rectangular masks may be maintained between 10% to 40% of the image - thus, a masking operation (equivalent to inpainting kernel) generates masked input image; [0093] for JPEG restoration tasks, training may be performed on different quality factors; an exponential distribution may be used to sample the quality factor during training (the jpeg compression at a sampled quality factor equivalent to JPEG restoration kernel); [0053] additional conditioning of the source image via concatenation; [0064] p(y I x), where both x and y are images, such as a grayscale image 110, represented as x, and a color image, represented as y), and wherein the obtained visual data is labeled data regarding the input data ([0089] RGB parameterization may be used for colorization; the grayscale image may be used as the source image and Palette may be trained to predict the full RGB image; [0072] each pair comprises an image and at least one corresponding target version of the image (i.e., labeled data regarding input data); [0094] FIG. 2 illustrates example output images for different tasks; each row 2R1-2R4 shows corresponding input image 210, output image 220, and original image 230).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein generating visual data from the obtained visual data through corresponding kernels, respectively, before corresponding visual intelligence tasks are performed, and thereby generating input data of the multi-task integrated deep learning model, and wherein the obtained visual data is labeled data regarding the input data, as taught by Saharia into Lin. Doing so would be desirable because it would enable the image processing system to achieve a desired performance level over fewer training iterations than would be required by some existing systems, thereby enabling reduced consumption of computational resources during training (Saharia [0042]).
As to dependent Claim 3, Lin and Saharia teach all the limitations of claim 2. Lin further teaches wherein labeled data regarding the input data obtained from one piece of visual data is all the same ([0089] RGB parameterization may be used for colorization; the grayscale image may be used as the source image and Palette may be trained to predict the full RGB image; [0090] for inpainting tasks, Palette may be trained on a combination of free-form and rectangular masks; the total area covered by the rectangular masks may be maintained between 10% to 40% of the image; [0093] for JPEG restoration tasks, training may be performed on different quality factors; an exponential distribution may be used to sample the quality factor during training; [0094] for a input image, 210, Palette produces the output image, 220, shown, the estimate of the original image, 230 - thus, for each restoration task, the target is the original image from which the input was derived).
As to dependent Claim 4, Lin and Saharia teach all the limitations of claim 2. Shaharia further teaches wherein the visual intelligence tasks are selectable by a user ([0043] the image processing system can perform multiple image-to-image translation tasks; the model described herein can perform operations including colorization, inpainting, and de-blurring (or any other appropriate set of multiple tasks); [0044] user of the mobile computing device can view the input image and determine that the input image should be sharpened; the user can then provide the input image to the trained neural network; [0099] the task is inpainting, which requires user-specified masked regions of an image to be filled with realistic content).
As to dependent Claim 5, Lin and Saharia teach all the limitations of claim 2. Lin further teaches wherein obtaining, generating, and training are repeated for visual data obtained ([0043] the system obtains initial training data for training the image representation neural network; [0048] the system generates, from the initial training data, final training data for training the image representation neural network; [0049]-[0050] for each training image in the training data, the system determines whether the training image is missing a ground-truth label for any of the multiple tasks; if the training image is missing a ground-truth label for any given task, the system processes the training image using a teacher neural network for the given task to generate a pseudo-label and then uses the pseudo-label as the label for the training image for the given task; [0066] the system can repeatedly perform iterations of the process 300 to repeatedly update the image representation parameters. See fig. 3 - each iteration obtaining batch of training data and training the model; [0017] the image 112 can be any appropriate type of image; the image can be a point cloud generated by a LIDAR sensor; the image can be a medical image generating by a medical imaging device; computer tomography (CT) image, a magnetic resonance imaging (MM) image, an ultrasound image, an X-ray image, a mammogram image, a fluoroscopy image, or a positron-emission tomography (PET) image).
As to dependent Claim 6, Lin and Saharia teach all the limitations of claim 2. Saharia further teaches wherein a size of input data and a size of output data of the multi-task integrated deep learning model are the same ([0063] given a noisy image x and a denoised image y, 110, a diffusion model may generate a noisy version of the denoised image ỹ, and train a multi-task diffusion model 120, to denoise ỹ given input image x and a noise level indicator y; x may be iteratively downsampled through an encoder; the downsampling could be from a resolution of 128 × 128 to a resolution of 64 × 64 and then to a resolution of 8 × 8; an output from the downsampling process may be iteratively upsampled from a resolution of 8 × 8 to a resolution of 64 × 64 and then to a resolution of 128 × 128, through a decoder).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Saharia, further in view of Risser et al. (US 2019/0294931 A1 hereinafter Risser).
As to dependent Claim 7, Lin and Saharia teach all the limitations of claim 6. Saharia further teaches wherein the visual intelligence tasks comprise super-resolution, denoising, inpainting, colorization ([0058] the plurality of image-to-image translation tasks include one or more of a colorization task, an uncropping task, an inpainting task, a decompression artifact removal task, a super-resolution task, a de-noising task, or a panoramic image generation task).
However, Lin and Saharia fail to expressly teach wherein the visual intelligence tasks comprise dehazing, high dynamic range (HDR).
In the same field of endeavor, Risser teaches wherein the visual intelligence tasks comprise dehazing, high dynamic range (HDR) ([0080] GENs in accordance with numerous embodiments of the invention can be used in a variety of use cases, including increasing the bit depth of colors from standard definition (8 bits per color) to high dynamic range (10, 16 or 32 bits per color channel); [0082] monocular depth map estimation and image dehazing belong to a class of image-to-image generation problems; [0083] to address these use cases, GENs in accordance with a variety of embodiments of the invention can stack two GENs in sequence).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the visual intelligence tasks comprise dehazing, high dynamic range (HDR), as taught by Risser into Lin and Saharia. Doing so would be desirable because it would provide a novel, general purpose, image-to-image generative neural network method (Risser [0030]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Saharia, further in view of Zhou et al. (US 2023/0026811 A1 hereinafter Zhou), Abuolaim et al. (US 2023/0156342 A1 hereinafter Abuolaim), and Shi et al. (US 2018/0075581 A1 hereinafter Shi).
As to dependent Claim 8, Lin and Saharia teach all the limitations of claim 6. Saharia further teaches wherein the kernels used in generating the training data comprises: a denoising data conversion kernel configured to generate input data of the multi-task integrated deep learning model by adding a noise to visual data ([0058] the plurality of image-to-image translation tasks include one or more of a colorization task, an uncropping task, an inpainting task, a decompression artifact removal task, a super-resolution task, a de-noising task; [0072] applying a forward Gaussian diffusion process that adds Gaussian noise to a corresponding target version of each of the plurality of pairs of images to enable iterative denoising of the input image); an inpainting data conversion kernel configured to generate input data of the multi-task integrated deep learning model by masking a specific region in visual data ([0058] the plurality of image-to-image translation tasks include one or more of a colorization task, an uncropping task, an inpainting task, a decompression artifact removal task, a super-resolution task, a de-noising task; [0090] for inpainting tasks, Palette may be trained on a combination of free-form and rectangular masks); and a colorization data conversion kernel configured to generate input data of the multi-task integrated deep learning model by converting visual data into a gray image ([0058] the plurality of image-to-image translation tasks include one or more of a colorization task, an uncropping task, an inpainting task, a decompression artifact removal task, a super-resolution task, a de-noising task; [0089] RGB parameterization may be used for colorization; the grayscale image may be used as the source image and Palette may be trained to predict the full RGB image).
However, Lin and Saharia fail to expressly teach wherein a dehazing data conversion kernel configured to generate input data of the multi-task integrated deep learning model by adding a haze to visual data.
In the same field of endeavor, Zhou teaches wherein a dehazing data conversion kernel configured to generate input data of the multi-task integrated deep learning model by adding a haze to visual data ([0032]virtual hazy images can be generated from haze-free target images using the atmospheric physical model; [0069] training data generator 105 may apply a forward mode of the atmospheric physical model to target image 210 using the one or more randomly-generated atmospheric physical parameters, and may generate a virtual hazy image thereof; [0005] dehazing deep learning model is trained using the one or more hazy input images and the one or more target images).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein a dehazing data conversion kernel configured to generate input data of the multi-task integrated deep learning model by adding a haze to visual data, as taught by Zhou into Lin and Saharia. Doing so would be desirable because it would remove the haze effect on the remote sensing image (Zhou [0005]).
However, Lin, Saharia, and Zhou fail to expressly teach wherein a HDR data conversion kernel configured to generate input data of the multi-task integrated deep learning model by converting visual data into data of a low illuminance.
In the same field of endeavor, Abuolaim teaches wherein a HDR data conversion kernel configured to generate input data of the multi-task integrated deep learning model by converting visual data into data of a low illuminance ([0048] generating, from daytime or other high-illumination images, images with simulated low-light or nighttime illumination and simulated noise that can be used as pairs to train networks; [0050] a daytime image 402 may be captured, and nighttime illumination may be simulated on the image 402, as shown by image 404; noise may be simulated on the image 404, as shown in image 406; image 404 and 406 may be used as a training pair).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein a HDR data conversion kernel configured to generate input data of the multi-task integrated deep learning model by converting visual data into data of a low illuminance, as taught by Abuolaim into Lin, Saharia, and Zhou. Doing so would be desirable because it would provide improved method of generating a low-light image (Abuolaim [0003]-[0004]).
However, Lin, Saharia, Zhou, and Abuolaim fail to expressly teach wherein a super-resolution data conversion kernel configured to generate input data of the multi-task integrated deep learning model by converting visual data into data of a low resolution.
In the same field of endeavor, Shi teaches wherein a super-resolution data conversion kernel configured to generate input data of the multi-task integrated deep learning model by converting visual data into data of a low resolution ([0043] the high-resolution images can be provided during training of the networks; when training the networks ILR can be obtained by applying a Gaussian-filter to IHR followed by a downsampling operation with a downsampling factor r).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein a super-resolution data conversion kernel configured to generate input data of the multi-task integrated deep learning model by converting visual data into data of a low resolution, as taught by Shi into Lin, Saharia, Zhou, and Abuolaim. Doing so would be desirable because it would enable the network to recover photo-realistic textures from heavily downsampled images (Shi [0026).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Shi et al. (US 2018/0075581 A1 hereinafter Shi).
As to dependent Claim 9, Lin teaches all the limitations of claim 1. Lin further teaches wherein training the multi-task integrated deep learning model by using a weighted sum of a loss obtained considering a common loss of the multiple visual intelligence tasks through a loss function ([0023] the system trains the neural network on a loss function that measures the performance of representations generated from the feature maps on multiple different computer vision tasks; uses a multi-task learning loss function to measure the quality of those feature representations across multiple different computer vision tasks; [0078] an overall loss function that includes a respective task-specific loss function for each of the plurality of computer vision tasks; the overall loss function can be an average or a weighted average of the task-specific loss functions
However, Lin fails to expressly teach wherein the weighted sum of loss obtained considering characteristics of the multi-task integrated deep learning model and common loss.
In the same field of endeavor, Shi teaches wherein weighted sum of loss obtained considering characteristics of the multi-task integrated deep learning model and common loss ([0053] the definition of the perceptual loss function influences the performance of the generator network; [0054] given weighting parameters, the perceptual loss function can be defined as the weighted sum of individual loss functions - thus, the generator is trained with a weighted sum of loss components); the perceptual loss function can include a content loss function, an adversarial loss function; [0061]-[0062] adversarial loss; the generative component of the GAN can be added to the perceptual loss; the generative loss can be defined based on the probabilities of the discriminator - thus, adversarial loss defined from the discriminator of the GAN model (i.e., a loss obtained considering characteristics of the model; [0055]-[0056] content loss; the pixel-wise MSE loss can be calculated as - thus, content loss may be pixel-wise loss, construed as common loss).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein weighted sum of loss obtained considering characteristics of the multi-task integrated deep learning model and common loss, as taught by Shi into Lin. Doing so would be desirable because it would enable the network to recover photo-realistic textures from heavily downsampled images (Shi [0026).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 CFR § 1.111(c) to consider these references fully when responding to this action.
Kusumi et al. (US 2022/0405892 A1) teaches: the acquiring unit 101 b acquires one or more original images from the memory 101 a; if the training is to be performed at once in the machine learning model that sharpens all of these types of blur, a plurality of blurred images may be generated by using a plurality of types of blur that occur in the optical system 102 a; in the blurred image, signal values exceeding the second signal value are clipped so that luminance saturation is reproduced, the luminance saturation occurring during an imaging process of the captured image; if necessary, noise generated by the image sensor 102 b may be applied to the blurred image (see [0052]-[0053).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REJI KARTHOLY whose telephone number is (571)272-3432. The examiner can normally be reached on Monday - Thursday from 7:30 am to 3:30 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch, can be reached at telephone number 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/REJI KARTHOLY/Primary Examiner, Art Unit 2143