DETAILED ACTION
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12, and 15-20 are subject to the claim rejection under 35 U.S.C. 101 because the each claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more:
According to the 2024 Guidance Update on Patent Subject Matter Eligibility, Step 1 analysis: claims 1-12, are identified as identified as directed to a process, and claims 15-20 are directed to a machine, and manufacture of mater respectively;
In Step 2A, Prong One Analysis evaluates whether the claim recites a
judicial exception:
each independent each of claims 1, 15 and 19 similarly recites “acquiring a first image”; “determining restoration style information of the first image”; and “restoring the first image based on the restoration style information by using an artificial intelligence (AI) network to obtain a restored image”; which are as abstract idea of Mental processes, that can be mentally carried out solely by human, and without any particular algorithms or improvements, because “determining restoration style information…”, and “restoring the first image … to obtain a restored image” cane be mental processing, in which human obtains/observes an image, and forming a restored image” in mind, although claim recites “using an artificial intelligence (AI) network to obtain a restored image”; however, there is no specific algorithms or steps of how, what and/or specific configurations about an artificial intelligence (AI) network has been recited in claim, which would be considered as a generic computer, or program applied on an abstract idea. “An artificial intelligence (AI) network” is interpreted under the broadest reasonable interpretation (BRI) standard, as the of meaning of simulation of human mental processing.
Step (2) Analysis, each of independent claims 1, 15 and 19 recites abstract idea of Mental processes, those steps are human perception and mental processes. There is no limitation, or combination of another element, and there is no additional element.
Each of claims 15, and 19 recites “an electronic device”, and “non-transitory computer-readable storage media storing computer-executable instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations” to perform these abstract idea, but claims 15-20 do not particularly define, or specify the features, algorithms, or particular components, or any modification or improvement of the device to achieve the desired result or the stated purpose. Thus, the claimed “an electronic device” is considered as a generic computer applied to the idea abstract.
In Step 2B, Prong Two Analysis to evaluate whether the claimed additional elements amount to significantly more than the recited judicial exception itself:
There are no any additional elements recited in claim beyond the Mental processes, neither any practical application that above mentioned step as Mental processes can be integrated into has been recited in each of independent claims 1, 15 and 19. As no particular computer function, algorithm or configuration is practically improved, or any particular application, or specific utility, or improvement is defined, or to be brought into certain solution, or improvement in any meaningful practice. For example, there is specific limitation about how to restore the image, and what would be approved in the restored image, or how the artificial intelligence (AI) network have been configured to obtain the restored image? ..etc., Therefore, independent claims 1, 15, and 19 do not have additional elements amount to significantly more than the recited judicial exception itself.
Overall, Claims 1-12, and 15-20 do not limit what’s specialty, algorithms and characteristics of a method, or a device with any specific algorithms or improvements. Although Claims 3-12 recite additional elements, such as “image quality”, “restoration degree”; however still do not recite specific features of “image quality”, “restoration degree”, that would particularly associated with image restoration, and/or associated with “the artificial intelligence (AI) network to obtain a restored image” Therefore, Claims 1-12, and 15-20 are considered as human mental processing, and a conventional operation of device, and would be conventional practice of human observation and mental processing. Therefore above mentioned steps are considered as an abstract idea of Mental processes.
On the other hands, claims 13-14 recite specific implementation of “obtaining a first image feature based on the lightness information and/or the depth information;
performing feature extraction on the first image to obtain a second image feature; and
determining the image quality information of the first image based on the first image feature and the second image feature”; which is considered as an additional element require a computer or an electronic device to performing feature extraction on the first image to obtain a second image feature; and determining the image quality information of the first image based on the first image feature and the second image feature. Thus, claims 13-14 are eligible in Patent Subject Matter Eligibility analysis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1, 3-7, 15, and 17-19 are rejected under 35 U.S.C. 103(a) as being unpatentable over ZHANG (CN 110930333 A), and in view of YANG et al. (CN 111161161 B).
Re Claim 1, ZHANG a method performed by an electronic device (see ZHANG: e.g., -- an image repairing method, device, electronic device, and computer-readable storage medium, wherein the method comprises: obtaining the grey image and color information of the image to be repaired; the grey image input to the special repairing model; the grey image output after reducing, wherein special repairing model is obtained by training the repair model through a plurality of training samples belonging to image content category of the image to be repaired, wherein the training sample comprises: matching the original grey scale image and grey scale image is compressed; The greyscale image colour information and reducing to obtain the image after restoration--, in abstract), comprising the following:
acquiring a first image (see ZHANG: e.g., -- an image repairing method, device, electronic device, and computer-readable storage medium, wherein the method comprises: obtaining the grey image and color information of the image to be repaired; the grey image input to the special repairing model; the grey image output after reducing, wherein special repairing model is obtained by training the repair model through a plurality of training samples belonging to image content category of the image to be repaired, wherein the training sample comprises: matching the original grey scale image and grey scale image is compressed; The greyscale image colour information and reducing to obtain the image after restoration--, in abstract);
determining image content type of the image content category, and the colour information of the first image (see ZHANG: e.g., --to obtain the colour information and the image after repair.
…
identifying image content category of the image to be repaired;
The image content type identification to search the special repairing model corresponding to the image content category;-- ,in page 2/16 of English version of CN 110930333 A, as provided as NPL with the Office Action);
Zhang however does not explicitly disclose determining restoration style information of the first image;
YANG discloses determining restoration style information of the first image (see YANG: e.g., -- a colour-held characteristic fusion defogging method, comprising the following steps: (1) obtaining a total training set and a test set; the total training set comprises an outdoor training set and an indoor training set, the outdoor training set, indoor training set and test set are composed of clear image and a fog image; (2) realizing single image defogging based on the convolutional neural network convolutional neural network feature extraction, feature fusion and image recovery three parts; feature extraction comprises a content information extracting module and style information extracting module, the content information extracting module and style information extracting module respectively adopts seven cascaded residual blocks RB1 ~ RB7 and three cascaded dense residual blocks RDB1 ~ RDB3 as main body frame; the characteristic fusion part carries out weighted stacking to obtain the weighted content characteristic graph through the characteristic graph output by the residual error block; The image restoration part comprises a convolution layer for obtaining clear fog-free image.--, in abstract, and, -- (2) realizing single image defogging based on the convolutional neural network convolutional neural network
feature extraction, feature fusion and image recovery three parts; feature extraction comprises a content
information extracting module and style information extracting module, the content information extracting
module and style information extracting module respectively adopts seven cascaded residual blocks RB1 ~
RB7 and three cascaded dense residual blocks RDB1 ~ RDB3 as main body frame;
the characteristic fusion part carries out weighted stacking to obtain the weighted content characteristic
graph through the characteristic graph output by the residual error block;
The image restoration part comprises a convolution layer for obtaining clear fog-free image…..
the content information extracting module comprises three convolution layers and seven residual blocks; the three convolution layers are respectively cov1, cov2, cov3, the size of the convolution kernel is 3 * 3, the step length is 1, 1, 2, padding are set as 1, the output channel number is 64; the seven residual blocks are respectively RB1, RB2, RB3, RB4, RB5, RB6, RB7; the structure of each residual block is the same, each residual block comprises three convolution layers, the size of the convolution kernel is 3 * 3, the step length is 1, the padding is set as 1, the output channel number is 64;
the style information extracting module comprises two convolution layers and three dense residual blocks; the two convolution layers are respectively cov4, cov5, the size of the convolution kernel is 3 * 3, the step length is 1, the padding is set as 1, the output channel number is 16; the three dense residual blocks are RDB1, RDB2 and RDB3….
(1) Channel weighting: (a) channel weighting: outputting the output of the residual blocks RB1, RB5 and RB7 to obtain the weighted content characteristic diagram Vc, wherein the output of the dense residual block RDB3 is the style characteristic diagram Vs;
(b) fusion: stacking the content characteristic graph and style characteristic graph Vc and Vs, then obtaining the fusion characteristic graph Vf through reverse convolution layer and convolution layer operation. {herein determining/obtaining the “content characteristic style” and “fusion characteristic” read on claimed limitation of “determining restoration style information of the first image”}, --, in from the bottom of pages 2 to page 3 of English version of CN-111161161-B, as provided as NPL with the Office Action);
YANG and Zhang are combinable as they are in the same field of endeavor: image restoration, Therefore it would have been obvious to one of ordinary skill in the art at the time the invention was made to further modify ZHANG’s method using YANG’s teachings by including determining restoration style information of the first image to ZHANG’s identifying image content category of the image to be repaired in order to extract content information and style information of the image to be repaired for assisting in image restoration processing (see YANG: e.g., at abstract, and pages 2-3/8 of English version of CN-111161161-B, as provided as NPL with the Office Action); and
ZHANG as modified by Yang further disclose restoring the first image based on the restoration style information by using an artificial intelligence (AI) network to obtain a restored image (see ZHANG: e.g., -- an image repairing method, device, electronic device, and computer-readable storage medium, wherein the method comprises: obtaining the grey image and color information of the image to be repaired; the grey image input to the special repairing model; the grey image output after reducing, wherein special repairing model is obtained by training the repair model through a plurality of training samples belonging to image content category of the image to be repaired, wherein the training sample comprises: matching the original grey scale image and grey scale image is compressed; The greyscale image colour information and reducing to obtain the image after restoration--, in abstract also see YANG: e.g., -- a colour-held characteristic fusion defogging method, comprising the following steps: (1) obtaining a total training set and a test set; the total training set comprises an outdoor training set and an indoor training set, the outdoor training set, indoor training set and test set are composed of clear image and a fog image; (2) realizing single image defogging based on the convolutional neural network convolutional neural network feature extraction, feature fusion and image recovery three parts; feature extraction comprises a content information extracting module and style information extracting module, the content information extracting module and style information extracting module respectively adopts seven cascaded residual blocks RB1 ~ RB7 and three cascaded dense residual blocks RDB1 ~ RDB3 as main body frame; the characteristic fusion part carries out weighted stacking to obtain the weighted content characteristic graph through the characteristic graph output by the residual error block; The image restoration part comprises a convolution layer for obtaining clear fog-free image.--, in abstract).
Re Claim 3, ZHANG’333 as modified by Yang further disclose determining restoration degree information of the first image (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action);
wherein the restoring of the first image based on the restoration style information by using an AI network comprises:
restoring the first image based on the restoration style information and the restoration degree information by using an AI network (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action).
Re Claim 4, ZHANG’333 as modified by Yang further disclose wherein the restoration degree information is related to at least one of: image quality of the first image; or a restoration degree related instruction input by a user (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action).
Re Claim 5, ZHANG’333 as modified by Yang further disclose wherein the determining of the restoration degree information of the first image comprises: providing a restoration degree confirmation interface for the first image (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action); and
in response to a restoration degree related instruction input by a user through the restoration degree confirmation interface, determining second restoration degree information of the first image (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action).
Re Claim 6, ZHANG’333 as modified by Yang further disclose wherein the providing of the restoration degree confirmation interface for the first image comprises:
determining image quality information of the first image (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action),
determining corresponding first restoration degree information based on the image quality information (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action), and
providing a restoration degree confirmation interface for the first image based on the first restoration degree information (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action), and
wherein, in response to the restoration degree related instruction input by the user through the restoration degree confirmation interface, the determining of the second restoration degree information of the first image (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action) comprises :
in response to an adjustment instruction to the first restoration degree information input by the user through the restoration degree confirmation interface, determining second restoration degree information of the first image (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action).
Re Claim 7, ZHANG’333 as modified by Yang further disclose wherein the determining of the restoration degree information of the first image comprises: determining image quality information of the first image; and determining corresponding first restoration degree information based on the image quality information (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action).
Re Claims 15, and 17-18, claims 15, and 17-18 are the corresponding device claims to claims 1, and 3-4 respectively. Therefore, claims 15, and 17-18 are rejected by the similar reasons as for the rejections of claims 1, and 3-4 respectively. Furthermore, ZHANG’333 as modified by Yang further disclose electronic device comprising: memory storing one or more computer programs; and one or more processors communicatively coupled to the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to perform the method (see ZHANG: e.g., -- an image repairing method, device, electronic device, and computer-readable storage medium, wherein the method comprises: obtaining the grey image and color information of the image to be repaired; the grey image input to the special repairing model; the grey image output after reducing, wherein special repairing model is obtained by training the repair model through a plurality of training samples belonging to image content category of the image to be repaired, wherein the training sample comprises: matching the original grey scale image and grey scale image is compressed; The greyscale image colour information and reducing to obtain the image after restoration--, in abstract).
Re Claim 19, claim 19 is the corresponding medium claim to claim 1 respectively. Therefore, claim 19 is rejected by the similar reasons as for the rejections of claim 1 respectively. Furthermore, ZHANG’333 as modified by Yang further disclose One or more non-transitory computer-readable storage media storing computer-executable instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations (see ZHANG: e.g., -- an image repairing method, device, electronic device, and computer-readable storage medium, wherein the method comprises: obtaining the grey image and color information of the image to be repaired; the grey image input to the special repairing model; the grey image output after reducing, wherein special repairing model is obtained by training the repair model through a plurality of training samples belonging to image content category of the image to be repaired, wherein the training sample comprises: matching the original grey scale image and grey scale image is compressed; The greyscale image colour information and reducing to obtain the image after restoration--, in abstract).
Claims 2, 16, and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable over ZHANG’333 as modified by Yang, and further in view of ZHANG’728 et al. (CN 114219728 A, referred as ZHANG’728, different from ZHANG’333).
Re Claim 2, ZHANG as modified by Yang do not explicitly disclose wherein the restoration style information is related to attribute information of a target object in the first image,
ZHANG’728 disclose the restoration style information is related to attribute information of a target object in the first image (see ZHANG’728: Fig. 1, and Fig. 2, and, -- a method for repairing human face of one embodiment of the present invention of the flow chart. Including:
S11, extracting the high quality expression in the characteristic space by using the high quality characteristic extraction network for the input high quality image human faces
S12, using the low quality feature extraction network to obtain the low quality expression in the feature space of the input low quality human faces--, in page 5/9 of English version of CN-114219728-A, as provided as NPL with the Office Action);
ZHANG’333 (as modified by YANG) and Zhang’728 are combinable as they are in the same field of endeavor: image restoration, Therefore it would have been obvious to one of ordinary skill in the art at the time the invention was made to further modify ZHANG ZHANG’333 (as modified by YANG)’s method using Zhang’728’s teachings by including the restoration style information is related to attribute information of a target object in the first image to ZHANG’333 (as modified by YANG)’s extracting/determining the restoration style and identifying image content category of the image to be repaired in order to extract content information and style information of the image to be repaired for assisting in image restoration processing (see Zhang’728: e.g., Fig. 1, Fig. 2, and at abstract, page 5/9 of English version of CN-114219728-A, as provided as NPL with the Office Action).
. Re Claim 16, claim 16 is the corresponding device claims to claim 2 respectively. Therefore, claim 16 is rejected by the similar reasons as for the rejections of claim 2 respectively. Furthermore, ZHANG’333 as modified by Yang and Zhang’728 further disclose electronic device comprising: memory storing one or more computer programs; and one or more processors communicatively coupled to the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to perform the method (see ZHANG: e.g., -- an image repairing method, device, electronic device, and computer-readable storage medium, wherein the method comprises: obtaining the grey image and color information of the image to be repaired; the grey image input to the special repairing model; the grey image output after reducing, wherein special repairing model is obtained by training the repair model through a plurality of training samples belonging to image content category of the image to be repaired, wherein the training sample comprises: matching the original grey scale image and grey scale image is compressed; The greyscale image colour information and reducing to obtain the image after restoration--, in abstract).
Re Claim 20, claim 20 is the corresponding medium claim to claim 2 respectively. Therefore, claim 20 is rejected by the similar reasons as for the rejections of claim 2 respectively. Furthermore, ZHANG’333 as modified by Yang and Zhang’728 further disclose One or more non-transitory computer-readable storage media storing computer-executable instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations (see ZHANG: e.g., -- an image repairing method, device, electronic device, and computer-readable storage medium, wherein the method comprises: obtaining the grey image and color information of the image to be repaired; the grey image input to the special repairing model; the grey image output after reducing, wherein special repairing model is obtained by training the repair model through a plurality of training samples belonging to image content category of the image to be repaired, wherein the training sample comprises: matching the original grey scale image and grey scale image is compressed; The greyscale image colour information and reducing to obtain the image after restoration--, in abstract).
Claims 8-11 are rejected under 35 U.S.C. 103(a) as being unpatentable over ZHANG’333 as modified by Yang, and further in view of ZHANG’839 et al. (US 20200364839 A1, different from ZHANG’333).
Re Claim 8, ZHANG’333 as modified by Yang further disclose wherein the restoration degree confirmation interface comprises at least one of: a restoration degree confirmation interface (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action),
ZHANG’333 as modified by Yang still do not explicitly disclose a global restoration degree confirmation interface or at least one local restoration degree confirmation interface, wherein the at least one local restoration degree confirmation interface corresponds to at least one local region of a target object;
ZHANG’839 discloses a global restoration degree confirmation interface or at least one local restoration degree confirmation interface, wherein the at least one local restoration degree confirmation interface corresponds to at least one local region of a target object (see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]);
ZHANG’333 (as modified by YANG) and Zhang’839 are combinable as they are in the same field of endeavor: image restoration, Therefore it would have been obvious to one of ordinary skill in the art at the time the invention was made to further modify ZHANG ZHANG’333 (as modified by YANG)’s method using Zhang’’839’s teachings by including a global restoration degree confirmation interface or at least one local restoration degree confirmation interface, wherein the at least one local restoration degree confirmation interface corresponds to at least one local region of a target object to ZHANG’333 (as modified by YANG)’s extracting/determining the restoration style and identifying image content category of the image to be repaired in order to determine restoration overall or certain areas of the target image (see Zhang’’839: e.g.,[0031], and [0036]).
Re Claim 9, ZHANG’333 as modified by Yang and ZHANG’839 further disclose in response to the restoration degree related instruction input by the user through the restoration degree confirmation interface, the determining of the second restoration degree information of the first image comprises:
determining second restoration degree information of the first image in response to at least one of:
a global restoration degree related instruction input by the user through the global restoration degree confirmation interface (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and, see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]), or
at least one local restoration degree related instruction input by the user through the at least one local restoration degree confirmation interface (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]).
Re Claim 10, ZHANG’333 as modified by Yang and ZHANG’839 further disclose wherein the determining of the corresponding first restoration degree information based on the image quality information (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action) comprises:
determining first global restoration degree information based on the image quality information (see ZHANG’333: e.g., -- an image restoration method provided by this embodiment prior to the S201 further comprises the following S2011:
S2011, the set in response to a triggering event, executing the operation of obtaining the grey image and color information of the image to be repaired
event in this embodiment setting event may be that which instructs electronic device to perform image restoration method provided by this embodiment. Based on this, in one example, the display interface of the electronic device is provided with two modes selectable, one is normal mode, one is the high quality mode.
image based on the normal mode, the user electronic device to see that is one of image to be repaired. based on the high quality mode, the electronic device using image restoration method provided by this embodiment to image restoration, image based on this, see the user through the electronic device is a repaired image.
In one embodiment, the user can by means of mouse click or touch select the high quality mode, the user triggers the predetermined event. electronic device in response to the triggering of the setting time, and then performing the S201.
In this embodiment, the intelligence can further improve the image restoration method provided by this embodiment, based on this, which improves the reading experience of the user.--, in pages 10-11/16 of English version of CN 110930333 A, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]), and
determining, based on the image quality information and an image mask of at least one local region, first local restoration degree information corresponding to the at least one local region, wherein the providing of the restoration degree confirmation interface for the first image based on the first restoration degree information (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]) comprises:
providing a global restoration degree confirmation interface in the restoration degree confirmation interface based on the first global restoration degree information(see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]), and
determining, based on the first local restoration degree information corresponding to at least local region, at least one corresponding local restoration degree confirmation interface in the restoration degree confirmation interface, and
wherein, in response to an adjustment instruction to the first restoration degree information input by the user through the restoration degree confirmation interface, the determining of the second restoration degree information of the first image comprises: determining second restoration degree information of the first image in response to at least one of:
an adjustment instruction to the first global restoration degree information input by the user through the global restoration degree confirmation interface (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]), or
an adjustment instruction to at least one local restoration degree information input by the user through the at least one local restoration degree confirmation interface (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action). See the similar obviousness and motivation statements for the combination of cited references discussed above as addressed above for claim 8.
Re Claim 11, ZHANG’333 as modified by Yang and ZHANG’839 further disclose in response to the restoration degree related instruction input by the user through the restoration degree confirmation interface, the determining of the second restoration degree information of the first image (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]) comprises:
in response to the global restoration degree related instruction input by the user through the global restoration degree confirmation interface, determining second global restoration degree information (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]);
in response to the at least one local restoration degree related instruction input by the user through the at least one local restoration degree confirmation interface, determining at least one corresponding second local restoration degree information (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]); and
fusing the second global restoration degree information and the at least one second local restoration degree information to obtain the second restoration degree information (see ZHANG’333: e.g., Fig. 2, and, -- In this embodiment, special repairing model can only repair the restoring image of the corresponding image content classes. images, e.g., to be repaired, the special repair model is generated using the large amount of device image repairing model training of the word class, the special repairing model only capable of generating word category to the equipment for repair. Furthermore, special repairing model provided by this embodiment can be one, also can be several.
In one embodiment, when the special repairing model is one, the specific implementation of the S202 is an image content category in the grey image pair image content category of image to be repaired, and special repairing model corresponding to the same, the grey image input to the special repairing image, output the grey image after reduction. the grey image after the reduction compared to the gray scale image of the image to be repaired, is one grey image with high definition.
image content category corresponding to the grey image pair image content category of image to be repaired, and special repairing model corresponding to the different input, the gradation image in the special repairing image, output the grey image after reduction. the reducing the grey scale image is grey image after image to be repaired. that is, the special repairing model is not grey image of the input processing, and is directly output.--, in page 6/16 of English version of CN 110930333 A, as provided as NPL with the Office Action;
also see YANG: e.g., -- using PSNR (dB) /SSIM value for comparison, the two indexes can reflect the similarity degree and structure similarity of the restored haze-free image and original clear image. The data from Table 1 can be seen, PSNR and SSIM value of the method of the invention are higher than the comparison method, indicating the method of the invention recovery of fog-free image quality is higher.--, in the last para. of English version of CN-111161161-B, as provided as NPL with the Office Action; and see ZHANG’839: e.g., -- the style image is over deep in local texture to result in disharmony of an overall effect, therefore, the first to-be-processed area in the style image is required to be determined, where the first to-be-processed area refers to a part required to be processed in the style image, and thus, a phenomenon of style image disharmony is eliminated. Moreover, the second to-be-processed area at a corresponding position is extracted from the initial image, where the corresponding position refers to a coordinate, dimension and size of the first to-be-processed area in the style image, corresponding to a coordinate, dimension and size of the second to-be-processed area in the initial image. The remaining area except the first to-be-processed area in the style image may be not required to be processed. In practical application, a way of determining the first to-be-processed area by neural network model training has the advantages of high efficiency and wide application range; or a way of selecting a to-be-processed area on the style image by a user may be adopted, and the way has the advantage of high accuracy rate. However, for parts of types of images such as a landscape image, the user cannot determine positions where textures are over deep to result in disharmony of the overall effect of the style image, but for a style image of a human image, for example, it is easy for the user to determine that the texture of the face part is over deep to result in the disharmony phenomenon.--, in [0031], and, -- after the brightness component and the chroma component of the style image are processed, the restoration operation for the style image is affirmed to be completed, the processed style image with the brightness and chroma being separately represented may be converted into an output image with a preset format, may be converted into an output image with an RGB format recognized by eyes of a person and also may also be converted into an output image with other formats, such as an image signal for communication transmission, according to demands of practical application.--, in [0036]).
Claims 12-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over ZHANG’333 as modified by Yang, and further in view of YANG’432 et al. (KR 20230054432 A).
Re Claim 12, ZHANG’333 as modified by Yang however still don’t explicitly
discloses acquiring depth information and/or lightness information of the first image;
YANG’432 discloses acquiring depth information and/or lightness information of the first image (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action);
ZHANG’333 (as modified by YANG) and Yang’432 are combinable as they are in the same field of endeavor: image restoration, Therefore it would have been obvious to one of ordinary skill in the art at the time the invention was made to further modify ZHANG ZHANG’333 (as modified by YANG)’s method using Yang’432’s teachings by including acquiring depth information and/or lightness information of the first image to ZHANG’333 (as modified by YANG)’s extracting and identifying image content category of the image to be repaired in order to extract depth feature and lightness and luminance information of the image to be repaired for assisting in image restoration processing (see Yang’432: e.g., Fig. 1, Fig. 2, and at abstract, pages 4-5 of English version of KR 20230054432 A, as provided as NPL with the Office Action);
ZHANG’333 (as modified by YANG) and Yang’432 further disclose determining the image quality information of the first image based on lightness information and/or depth information (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action);
ZHANG’333 (as modified by YANG) and Yang’432 are combinable as they are in the same field of endeavor: image restoration, Therefore it would have been obvious to one of ordinary skill in the art at the time the invention was made to further modify ZHANG ZHANG’333 (as modified by YANG)’s method using Yang’432’s teachings by including acquiring depth information and/or lightness information of the first image to ZHANG’333 (as modified by YANG)’s extracting and identifying image content category of the image to be repaired in order to extract depth feature and lightness and luminance information of the image to be repaired for assisting in image restoration processing (see Yang’432: e.g., Fig. 1, Fig. 2, and at abstract, pages 4-5 of English version of KR 20230054432 A, as provided as NPL with the Office Action).
Re Claim 13, ZHANG’333 (as modified by YANG) and Yang’432 further disclose wherein the determining of the image quality information of the first image based on the lightness information and/or the depth information comprises:
obtaining a first image feature based on the lightness information and/or the depth information (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action);
performing feature extraction on the first image to obtain a second image feature (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action); and
determining the image quality information of the first image based on the first image feature and the second image feature (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action).
Re Claim 14, ZHANG’333 (as modified by YANG) and Yang’432 further disclose wherein the obtaining of the first image feature based on the lightness information and the depth information comprises:
performing feature extraction on the lightness information and the depth information to obtain a lightness feature and a depth feature, respectively (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action);
determining, based on the lightness feature, a first weight map for compensating the depth feature (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action);
weighting the depth feature according to the first weight map to obtain a compensated depth feature (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action); and
fusing the lightness feature and the compensated depth feature to obtain the first image feature (see YANG’432: e.g., -- the step of obtaining a target face image by restoring a person image based on the luminance channel includes inputting the luminance channel to a trained neural network model and
restoring a person image based on the luminance channel to restore the target face image. It includes the step of obtaining
Specifically, the person image is restored using the trained neural network model.--, in page 4/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action; and,
-- performing an encoding operation on the luminance channel using the first network, and obtaining a target feature map includes inputting the luminance channel to the first network and performing downsampling to obtain the first feature map. Obtaining a higher layer feature extraction for a first feature map using the fourth network, obtaining a higher layer feature map, and overlapping the first feature map and the higher layer feature map to obtain the target feature map. Including steps to get.
Here, the first feature map refers to a low-resolution feature map obtained by performing downsampling by a plurality of downsampling modules in the first network, and the high-layer feature map refers to a feature map obtained by performing depth feature extraction using the fourth network. . A target feature map is obtained by overlapping the first feature map and the upper layer feature map by shortcut connection. It should be understood that by overlapping the output of the first network and the output of the fourth network by means of a shortcut connection, overfitting of the neural network model can be prevented, while feature information can be enriched. The fourth network may be a residual block, and the residual block is a normal setting in the residual network, and is excellent in extracting depth features or extracting high-layer features.--, in from the last second para. of page 4/20 to the first para. page 5/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action;
and, -- corresponding pixel values in the rectangular area are replaced with pixel values in the target area to obtain the second face image, and the sample image pair is obtained by using the first face image and the second face image. make up
Here, the image quality of the first face image is first determined, and whether or not the image quality is deteriorated is determined. When the image quality is degraded, there is no need to perform deterioration processing, and a sample image pair is formed by the two first face images, and any one of the two is determined as the second face image. When the image quality is not deteriorated, degradation processing is performed, and concretely, this can be realized using a preset algorithm. When the first face image is input and the original first face image finally returns, the image quality of the first face image itself is degraded, and if it does not return, the first face image is subjected to atmospheric disturbance with a certain probability, Obtain a first deteriorated image, perform downsampling of the first deteriorated image by a factor of 0 to 8, obtain a low-resolution target deteriorated image, and perform corresponding upsampling of the target deteriorated image to obtain the same quality as the first deteriorated image. A second deteriorated image of resolution is obtained, noise is added to a luminance channel of the second deteriorated image, and non local average noise removal is performed to obtain a third deteriorated image.--, in page 9/20 of English version of KR 20230054432 A, as provided as NPL with the Office Action).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEIWEN YANG whose telephone number is (571)270-5670. The examiner can normally be reached on Monday-Friday 8:30am-4:30pm east.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEI WEN YANG/ Primary Examiner, Art Unit 2662
Prosecution Insights