DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 05/28/2026 has been fully considered and entered into record. Claims 1-20 remain pending in the application. Claims 1, 4, 6, 8, 11, 12, 15, 17, 19 and 20 have been amended.
Response to Arguments
Applicant's arguments with respect to claims 1-20 have been considered but are moot because the present rejection constitutes a new ground of rejection. The present rejection no longer relies only on Aggarwal and YANG for the disputed limitations, but instead relies on Luo, Aggarwal and YANG for teachings set forth in the rejection. Accordingly, Applicant’s arguments directed to the prior combination are not persuasive.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2,4, 6, 8-13, 17, 19 and 20 are rejected under 35 U.S.C. §103 as being unpatentable over Luo et al (US 20230362399 A1) [hereinafter " Luo "] in view of YANG et al. (US 12147583 B2) [hereinafter "YANG"] and in view of Aggarwal et al. ( US 6834344 B1) [hereinafter " Aggarwal "]
As per claim 1, Luo a computer-implemented method, comprising:
receiving, at an encoder (Luo , [Abstract]”computer programs encoded on a computer storage medium, for jointly training an encoder ”)comprising a first machine learning model, (Luo , [0004]” generating, using the encoder machine learning model to which the data item is provided as input,”)an image (Luo , [0004]”… to be encoded within a digital watermark to be embedded into a training image;”)and a digital watermark; (Luo , [0004]” a first digital watermark that encodes the data item”)
outputting, by the encoder, a watermarked image generated based on the image and the digital watermark; (Luo , [0085]” each first data item among the multiple data items is encoded by the encoder machine learning model into a digital watermark, which is then overlaid onto a particular training image (from among multiple training images) to obtain a respective watermarked training image”)
selecting, from a set of benign transforms, a benign transform; (Luo , [0071]” the distortion machine learning model 132b, the training process can distort the watermarked training images from the distortion model training dataset to generate distorted watermarked training images.”)
performing the benign transform on the watermarked image to generate a benign image; (Luo , [0113]” The server system 102 applies distortions to the watermarked image (580)… The distorted watermarked images are generated by adding one or more distortions such as vertical and horizontal scaling, image offset, splicing with other background images, JPEG compression, cropping to simulate real world image alteration that a possibly encoded image 302… can undergo”).
decoding, by a decoder comprising a second machine learning model, (Luo , [0004]” and decoding, using the decoder machine learning model,”) the benign image to a first predicted value of the digital watermark; (Luo , [0091]” the decoder machine learning model to generate a predicted first data item for each respective watermarked image and a respective second error value (referred to as Loss2 460)”).
decoding, by the decoder, the malicious image to a second predicted value of the digital watermark; (Luo , [0091]” a second error value (referred to as Loss2 460) is computed based on the predicted first data item 460 and the target first data item 410 that indicates the difference between the predicted value and the actual value of the first data item used to watermark the images. For example, Loss2 can be a sigmoid cross entropy loss”) and
adjusting(Luo , [0117]” The server system 102 adjusts parameters of the encoder and the decoder machine learning models (595)”) a plurality of weights at least one weight of the decoder during a learning phase of the decoder (Luo , [0111]” The server system 102 fixes the weights of the encoder machine learning model (570)”) by at least
learning a minimum amount of error between the first predicted value, (Luo , [0091]” a second error value (referred to as Loss2 460) is computed based on the predicted first data item 460 and the target first data item 410 that indicates the difference between the predicted value and the actual value of the first data item used to watermark the images. For example, Loss2 can be a sigmoid cross entropy loss.”).
learning a maximum amount of error between the second predicted value, (Luo , [0073], [0116]” After generating the universal pattern and the transformed pattern, the training process compares the two patterns to compute a third error value using a loss function (for e.g., L2 loss). Note that the third error value is a predicted measurement of distortions added to the watermarked training images” and “The third error value can sometimes take the form ∥T(U.sub.0)−U.sub.1∥.sup.2 where T refers to the transformation of the watermarked training images by adding one or more distortions, U.sub.0 is universal pattern and U.sub.1 is the transformed pattern.”)
Luo does not disclose a minimum amount of error [corresponds to the benign image, and the digital watermark to encourage retrieval of the digital watermark from images subjected to benign transforms, ] and a maximum amount of error [corresponds to the malicious image, and the digital watermark to discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile.],
selecting, from a set of malicious transforms, a malicious transform;
performing the malicious transform on the watermarked image to generate a malicious image;
However, YANG in the same field of endeavor discloses a minimum amount of error [corresponds to the benign image, ([YANG col.2 ln50-55]” the potentially perturbed image is determined to be the benign image when the potentially perturbed image includes the plurality of embedded bits matching the plurality of watermark bits”)and the digital watermark] ([YANG col.2 ln25-30]” the potentially perturbed image includes a plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image,”) a maximum amount of error [corresponds to the malicious image, ([YANG col.2 ln55-57]” the potentially perturbed image is determined to be the adversely modified image otherwise;”) and the digital watermark] ([YANG col.2 ln25-30]” the potentially perturbed image includes a plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image,”))
selecting, from a set of malicious transforms, a malicious transform; ([YANG col.11, ln15-20]” Various different types of adversarial attack methods have been developed to generate adversarial examples for inputs to deep-learning based models. Two main categories of adversarial attacks are gradient-based attacks and optimization-based attacks”)
performing the malicious transform on the watermarked image to generate a malicious image; ([YANG col.17 ln30-35]” The watermarked image can then be transmitted through a possibly adversarial environment. The watermarked image can be possibly attacked by adversaries” and “The detection rate can be defined as the percentage of adversely modified images φ(x.sub.wm) that are accepted by the detector as benign images. The detection rate can provide an indication of the sensitivity of the detector 106 (and the overall watermarking framework) to adversarial attacks.”).
discourage retrieval of the digital watermark from images subjected to malicious transforms, ([YANG col.12, ln 40-45]” If the evaluation of the received image indicates that the image has been adversely modified (e.g. the received image does not include the expected watermark), then the received image is prevented from being provided to the deep image classifier. If the evaluation of the received image indicates that the image has not been modified (e.g. the received image includes the expected watermark), then the received image can be provided to the deep image classifier.”) such that the digital watermark is semi-fragile.
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Luo to include a minimum amount of error [corresponds to the benign image, and the digital watermark]; a maximum amount of error [corresponds to the malicious image, and the digital watermark; selecting, from a set of malicious transforms, a malicious transform; performing the malicious transform on the watermarked image to generate a malicious image; discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile as suggested by YANG. One of ordinary skill in the art would have been motivated to do so because incorporating Yang’s watermark-based benign versus adversarial determination, which relies on bit error rate thresholds, would improve Luo’s encoder/decoder training by explicitly constraining predicted watermark values based on classification outcomes.
The combination of Luo and YANG fails to disclose encourage retrieval of the digital watermark from images subjected to benign transforms;
However, Aggarwal in the same field of endeavor discloses encourage retrieval of the digital watermark from images subjected to benign transforms, ([Aggarwal, [claim 16]” constructing a digest N…which survives benign modifications”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Luo to include a minimum amount of error [corresponds to the benign image, and the digital watermark]; a maximum amount of error [corresponds to the malicious image, and the digital watermark; selecting, from a set of malicious transforms, a malicious transform; performing the malicious transform on the watermarked image to generate a malicious image; discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile as suggested by YANG to further include encourage retrieval of the digital watermark from images subjected to benign transforms as taught by Aggarwal. One of ordinary skill in the art would have been motivated to do so because incorporating Aggarwal’s semi-fragile watermark technique , which preserves watermark retrieval after benign image modifications while allowing retrieval to fail following malicious modifications, would have predictably improved Luo’s machine-learning watermark training and complemented Yang’s benign-versus-adversarial watermark verification by increasing robustness to expected image processing without reducing the ability to detect maliciously transformed images.
As per claim 2, the combination of Aggarwal and YANG teaches the computer-implemented method of claim 1. YANG further discloses training, the encoder, to minimize an error between the image and the watermarked image, wherein the error comprises an image reconstruction loss and/or an adversarial loss. ([YANG, col.32, ln65-68]” Determining whether the potentially perturbed image includes the plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image can include: determining a bit error rate based on comparing the embedded bits with the watermark bits, where the bit error rate represents a percentage of the embedded bits that are distorted with respect to the corresponding watermark bits; and determining that the potentially perturbed image includes the plurality of embedded bits matching the plurality of watermark bits when the bit error rate is less than an error rate threshold.”).
As per claim 4, The references as combined above disclose the computer-implemented method of claim 1. Luo further discloses wherein the first machine learning model of the encoder comprises a convolutional neural network and/or a U-NET, ([Luo, [0062]]” The watermark detector machine learning model 350 is a CNN with UNet architecture”) and wherein the second machine learning model of the decoder comprises a convolutional neural network and/or a U-NET. ([Luo, [0080]]” the decoder machine learning model 134a can be deep convolutional neural network (CNN) with a UNet architecture that is trained to predict the predicted first data item”).
As per claim 6, the references as combined above disclose the computer-implemented method of claim 1. YANG further discloses wherein the malicious transform replaces an image portion of a subject of the watermarked image with another image portion, and/or wherein the malicious transform replaces at least a portion of a face image of the subject of the watermarked image with another face portion ([YANG, col.2, ln15-25]” watermarked image can be transmitted through a potentially adversarial environment. A potentially perturbed image intended for the deep neural network image classifier can be received from the potentially adversarial environment. The potentially perturbed image can be analyzed to determine whether the potentially perturbed image includes a plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image. The potentially perturbed image can be identified as an adversely modified image or benign image based on the comparison of the embedded bits and the expected watermark bits. The potentially perturbed image can be prevented from being provided to the deep neural network image classifier in response to determining that the potentially perturbed image is adverse. Benign images, on the other hand, can be provided as inputs to the deep neural network image classifier.”).
As per claim 8, the references as combined above disclose the computer-implemented method of claim 1. Aggarwal further discloses comparing the first predicted value of the digital watermark and/or the second predicted value to the digital watermark to determine whether the watermarked image has been maliciously transformed.([Aggarwal, Background [0007]” The principle is that if the image has been modified, the watermark alerts to this fact, and to some extent can localize where this modification has been done”).
As per claim 9, the references as combined above disclose the computer-implemented method of claim 1. Aggarwal further discloses wherein the digital watermark comprises an encrypted message, wherein the encrypted message is generated using a key and a message. ([Aggarwal, Background [0019]” A digital signature is a number that is obtained by encrypting a message or image through a digital signature algorithm and is mainly used to authenticate the integrity of the message or image. Digital watermarks are data added to the pixels of the image file. On the other hand, watermarks can have many uses, including, but not limited to integrity verification. For example, robust watermarks are used for claiming ownership. The present invention is to combine the benefits of robust watermarks (resistance to small modifications) with the benefits of fragile watermarks (detection of tampering of content)”). Claim 9 is rejected under the same rationale as claim 1 above.
As per claim 10, the references as combined above disclose the computer-implemented method of claim 1. YANG further discloses wherein the encoder receives a plurality of images to enable the learning phase of the decoder. ([YANG, col.12, ln30-35 ]]” The detector can then evaluate the received image to determine whether it has been adversely modified during transmission through the possibly adversarial environment. The evaluation of the received image can include determining whether the received image includes an expected watermark corresponding to the watermark embedded into the original image.”). Claim 10 is rejected under the same rationale as claim 1.
As per claim 11, the references as combined above disclose the computer-implemented method of claim 1. YANG further discloses wherein the adjusting further comprises adjusting at least one weight of the encoder during the learning phase. ([YANG, col.12, ln38-44,]”If the evaluation of the received image indicates that the image has been adversely modified (e.g. the received image does not include the expected watermark), then the received image is prevented from being provided to the deep image classifier. If the evaluation of the received image indicates that the image has not been modified (e.g. the received image includes the expected watermark), then the received image can be provided to the deep image classifier”).
As per claim 12, Luo discloses a system, comprising: at least one data processor; and at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising:
receiving, at an encoder (Luo , [Abstract]”computer programs encoded on a computer storage medium, for jointly training an encoder ”)comprising a first machine learning model, (Luo , [0004]” generating, using the encoder machine learning model to which the data item is provided as input,”)an image (Luo , [0004]”… to be encoded within a digital watermark to be embedded into a training image;”)and a digital watermark; (Luo , [0004]” a first digital watermark that encodes the data item”)
outputting, by the encoder, a watermarked image generated based on the image and the digital watermark; (Luo , [0085]” each first data item among the multiple data items is encoded by the encoder machine learning model into a digital watermark, which is then overlaid onto a particular training image (from among multiple training images) to obtain a respective watermarked training image”)
selecting, from a set of benign transforms, a benign transform; (Luo , [0071]” the distortion machine learning model 132b, the training process can distort the watermarked training images from the distortion model training dataset to generate distorted watermarked training images.”)
performing the benign transform on the watermarked image to generate a benign image; (Luo , [0113]” The server system 102 applies distortions to the watermarked image (580)… The distorted watermarked images are generated by adding one or more distortions such as vertical and horizontal scaling, image offset, splicing with other background images, JPEG compression, cropping to simulate real world image alteration that a possibly encoded image 302… can undergo”).
decoding, by a decoder comprising a second machine learning model, (Luo , [0004]” and decoding, using the decoder machine learning model,”) the benign image to a first predicted value of the digital watermark; (Luo , [0091]” the decoder machine learning model to generate a predicted first data item for each respective watermarked image and a respective second error value (referred to as Loss2 460)”).
decoding, by the decoder, the malicious image to a second predicted value of the digital watermark; (Luo , [0091]” a second error value (referred to as Loss2 460) is computed based on the predicted first data item 460 and the target first data item 410 that indicates the difference between the predicted value and the actual value of the first data item used to watermark the images. For example, Loss2 can be a sigmoid cross entropy loss”) and
adjusting(Luo , [0117]” The server system 102 adjusts parameters of the encoder and the decoder machine learning models (595)”) a plurality of weights at least one weight of the decoder during a learning phase of the decoder (Luo , [0111]” The server system 102 fixes the weights of the encoder machine learning model (570)”) by at least
learning a minimum amount of error between the first predicted value, (Luo , [0091]” a second error value (referred to as Loss2 460) is computed based on the predicted first data item 460 and the target first data item 410 that indicates the difference between the predicted value and the actual value of the first data item used to watermark the images. For example, Loss2 can be a sigmoid cross entropy loss.”).
learning a maximum amount of error between the second predicted value, (Luo , [0073], [0116]” After generating the universal pattern and the transformed pattern, the training process compares the two patterns to compute a third error value using a loss function (for e.g., L2 loss). Note that the third error value is a predicted measurement of distortions added to the watermarked training images” and “The third error value can sometimes take the form ∥T(U.sub.0)−U.sub.1∥.sup.2 where T refers to the transformation of the watermarked training images by adding one or more distortions, U.sub.0 is universal pattern and U.sub.1 is the transformed pattern.”)
Luo does not disclose a minimum amount of error [corresponds to the benign image, and the digital watermark to encourage retrieval of the digital watermark from images subjected to benign transforms, ] and a maximum amount of error [corresponds to the malicious image, and the digital watermark to discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile.],
selecting, from a set of malicious transforms, a malicious transform;
performing the malicious transform on the watermarked image to generate a malicious image;
However, YANG in the same field of endeavor discloses a minimum amount of error [corresponds to the benign image, ([YANG col.2 ln50-55]” the potentially perturbed image is determined to be the benign image when the potentially perturbed image includes the plurality of embedded bits matching the plurality of watermark bits”)and the digital watermark] ([YANG col.2 ln25-30]” the potentially perturbed image includes a plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image,”) a maximum amount of error [corresponds to the malicious image, ([YANG col.2 ln55-57]” the potentially perturbed image is determined to be the adversely modified image otherwise;”) and the digital watermark] ([YANG col.2 ln25-30]” the potentially perturbed image includes a plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image,”))
selecting, from a set of malicious transforms, a malicious transform; ([YANG col.11, ln15-20]” Various different types of adversarial attack methods have been developed to generate adversarial examples for inputs to deep-learning based models. Two main categories of adversarial attacks are gradient-based attacks and optimization-based attacks”)
performing the malicious transform on the watermarked image to generate a malicious image; ([YANG col.17 ln30-35]” The watermarked image can then be transmitted through a possibly adversarial environment. The watermarked image can be possibly attacked by adversaries” and “The detection rate can be defined as the percentage of adversely modified images φ(x.sub.wm) that are accepted by the detector as benign images. The detection rate can provide an indication of the sensitivity of the detector 106 (and the overall watermarking framework) to adversarial attacks.”).
discourage retrieval of the digital watermark from images subjected to malicious transforms, ([YANG col.12, ln 40-45]” If the evaluation of the received image indicates that the image has been adversely modified (e.g. the received image does not include the expected watermark), then the received image is prevented from being provided to the deep image classifier. If the evaluation of the received image indicates that the image has not been modified (e.g. the received image includes the expected watermark), then the received image can be provided to the deep image classifier.”) such that the digital watermark is semi-fragile.
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Luo to include a minimum amount of error [corresponds to the benign image, and the digital watermark]; a maximum amount of error [corresponds to the malicious image, and the digital watermark; selecting, from a set of malicious transforms, a malicious transform; performing the malicious transform on the watermarked image to generate a malicious image; discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile as suggested by YANG. One of ordinary skill in the art would have been motivated to do so because incorporating Yang’s watermark-based benign versus adversarial determination, which relies on bit error rate thresholds, would improve Luo’s encoder/decoder training by explicitly constraining predicted watermark values based on classification outcomes.
The combination of Luo and YANG fails to disclose encourage retrieval of the digital watermark from images subjected to benign transforms;
However, Aggarwal in the same field of endeavor discloses encourage retrieval of the digital watermark from images subjected to benign transforms, ([Aggarwal, [claim 16]” constructing a digest N…which survives benign modifications”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Luo to include a minimum amount of error [corresponds to the benign image, and the digital watermark]; a maximum amount of error [corresponds to the malicious image, and the digital watermark; selecting, from a set of malicious transforms, a malicious transform; performing the malicious transform on the watermarked image to generate a malicious image; discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile as suggested by YANG to further include encourage retrieval of the digital watermark from images subjected to benign transforms as taught by Aggarwal. One of ordinary skill in the art would have been motivated to do so because incorporating Aggarwal’s semi-fragile watermark technique , which preserves watermark retrieval after benign image modifications while allowing retrieval to fail following malicious modifications, would have predictably improved Luo’s machine-learning watermark training and complemented Yang’s benign-versus-adversarial watermark verification by increasing robustness to expected image processing without reducing the ability to detect maliciously transformed images.
As per claim 13, the references as combined above disclose the system of claim 12. YANG further discloses training, the encoder, to minimize an error between the image and the watermarked image, wherein the error comprises an image reconstruction loss and/or an adversarial loss. ([YANG, [0010]]” Determining whether the potentially perturbed image includes the plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image can include: determining a bit error rate based on comparing the embedded bits with the watermark bits, where the bit error rate represents a percentage of the embedded bits that are distorted with respect to the corresponding watermark bits; and determining that the potentially perturbed image includes the plurality of embedded bits matching the plurality of watermark bits when the bit error rate is less than an error rate threshold.”). Claim 13 is rejected under the same rationale as claim 12 above
As per claim 15, the substance of the claimed invention is identical or substantially similar to that of claim 4. Accordingly, this claim is rejected under the same rationale.
As per claim 17, the substance of the claimed invention is identical or substantially similar to that of claim 6. Accordingly, this claim is rejected under the same rationale.
As per claim 19, the references as combined above disclose the system of claim 12. Aggarwal discloses comparing the first predicted value of the digital watermark and/or the second predicted to the digital watermark to determine whether the watermarked image has been maliciously transformed, ([Aggarwal, [0007]” The principle is that if the image has been modified, the watermark alerts to this fact, and to some extent can localize where this modification has been done”)wherein the digital watermark comprises an encrypted message, wherein the encrypted message is generated using a key and a message, wherein the encoder receives a plurality of images to enable the learning phase of the decoder, wherein the adjusting further comprises adjusting at least one weight of an encoder during the learning phase. ([Aggarwal, Background[0019]” A digital signature is a number that is obtained by encrypting a message or image through a digital signature algorithm and is mainly used to authenticate the integrity of the message or image. Digital watermarks are data added to the pixels of the image file. On the other hand, watermarks can have many uses, including, but not limited to integrity verification. For example, robust watermarks are used for claiming ownership. The present invention is to combine the benefits of robust watermarks (resistance to small modifications) with the benefits of fragile watermarks (detection of tampering of content)”).
As per claim 20, Luo discloses a non-transitory computer-readable medium including instructions which, when executed by at least one data processor, cause operations comprising:
receiving, at an encoder (Luo , [Abstract]”computer programs encoded on a computer storage medium, for jointly training an encoder ”)comprising a first machine learning model, (Luo , [0004]” generating, using the encoder machine learning model to which the data item is provided as input,”)an image (Luo , [0004]”… to be encoded within a digital watermark to be embedded into a training image;”)and a digital watermark; (Luo , [0004]” a first digital watermark that encodes the data item”)
outputting, by the encoder, a watermarked image generated based on the image and the digital watermark; (Luo , [0085]” each first data item among the multiple data items is encoded by the encoder machine learning model into a digital watermark, which is then overlaid onto a particular training image (from among multiple training images) to obtain a respective watermarked training image”)
selecting, from a set of benign transforms, a benign transform; (Luo , [0071]” the distortion machine learning model 132b, the training process can distort the watermarked training images from the distortion model training dataset to generate distorted watermarked training images.”)
performing the benign transform on the watermarked image to generate a benign image; (Luo , [0113]” The server system 102 applies distortions to the watermarked image (580)… The distorted watermarked images are generated by adding one or more distortions such as vertical and horizontal scaling, image offset, splicing with other background images, JPEG compression, cropping to simulate real world image alteration that a possibly encoded image 302… can undergo”).
decoding, by a decoder comprising a second machine learning model, (Luo , [0004]” and decoding, using the decoder machine learning model,”) the benign image to a first predicted value of the digital watermark; (Luo , [0091]” the decoder machine learning model to generate a predicted first data item for each respective watermarked image and a respective second error value (referred to as Loss2 460)”).
decoding, by the decoder, the malicious image to a second predicted value of the digital watermark; (Luo , [0091]” a second error value (referred to as Loss2 460) is computed based on the predicted first data item 460 and the target first data item 410 that indicates the difference between the predicted value and the actual value of the first data item used to watermark the images. For example, Loss2 can be a sigmoid cross entropy loss”) and
adjusting(Luo , [0117]” The server system 102 adjusts parameters of the encoder and the decoder machine learning models (595)”) a plurality of weights at least one weight of the decoder during a learning phase of the decoder (Luo , [0111]” The server system 102 fixes the weights of the encoder machine learning model (570)”) by at least
learning a minimum amount of error between the first predicted value, (Luo , [0091]” a second error value (referred to as Loss2 460) is computed based on the predicted first data item 460 and the target first data item 410 that indicates the difference between the predicted value and the actual value of the first data item used to watermark the images. For example, Loss2 can be a sigmoid cross entropy loss.”).
learning a maximum amount of error between the second predicted value, (Luo , [0073], [0116]” After generating the universal pattern and the transformed pattern, the training process compares the two patterns to compute a third error value using a loss function (for e.g., L2 loss). Note that the third error value is a predicted measurement of distortions added to the watermarked training images” and “The third error value can sometimes take the form ∥T(U.sub.0)−U.sub.1∥.sup.2 where T refers to the transformation of the watermarked training images by adding one or more distortions, U.sub.0 is universal pattern and U.sub.1 is the transformed pattern.”)
Luo does not disclose a minimum amount of error [corresponds to the benign image, and the digital watermark to encourage retrieval of the digital watermark from images subjected to benign transforms, ] and a maximum amount of error [corresponds to the malicious image, and the digital watermark to discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile.],
selecting, from a set of malicious transforms, a malicious transform;
performing the malicious transform on the watermarked image to generate a malicious image;
However, YANG in the same field of endeavor discloses a minimum amount of error [corresponds to the benign image, ([YANG col.2 ln50-55]” the potentially perturbed image is determined to be the benign image when the potentially perturbed image includes the plurality of embedded bits matching the plurality of watermark bits”)and the digital watermark] ([YANG col.2 ln25-30]” the potentially perturbed image includes a plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image,”) a maximum amount of error [corresponds to the malicious image, ([YANG col.2 ln55-57]” the potentially perturbed image is determined to be the adversely modified image otherwise;”) and the digital watermark] ([YANG col.2 ln25-30]” the potentially perturbed image includes a plurality of embedded bits matching the plurality of watermark bits embedded into the original digital image,”))
selecting, from a set of malicious transforms, a malicious transform; ([YANG col.11, ln15-20]” Various different types of adversarial attack methods have been developed to generate adversarial examples for inputs to deep-learning based models. Two main categories of adversarial attacks are gradient-based attacks and optimization-based attacks”)
performing the malicious transform on the watermarked image to generate a malicious image; ([YANG col.17 ln30-35]” The watermarked image can then be transmitted through a possibly adversarial environment. The watermarked image can be possibly attacked by adversaries” and “The detection rate can be defined as the percentage of adversely modified images φ(x.sub.wm) that are accepted by the detector as benign images. The detection rate can provide an indication of the sensitivity of the detector 106 (and the overall watermarking framework) to adversarial attacks.”).
discourage retrieval of the digital watermark from images subjected to malicious transforms, ([YANG col.12, ln 40-45]” If the evaluation of the received image indicates that the image has been adversely modified (e.g. the received image does not include the expected watermark), then the received image is prevented from being provided to the deep image classifier. If the evaluation of the received image indicates that the image has not been modified (e.g. the received image includes the expected watermark), then the received image can be provided to the deep image classifier.”) such that the digital watermark is semi-fragile.
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Luo to include a minimum amount of error [corresponds to the benign image, and the digital watermark]; a maximum amount of error [corresponds to the malicious image, and the digital watermark; selecting, from a set of malicious transforms, a malicious transform; performing the malicious transform on the watermarked image to generate a malicious image; discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile as suggested by YANG. One of ordinary skill in the art would have been motivated to do so because incorporating Yang’s watermark-based benign versus adversarial determination, which relies on bit error rate thresholds, would improve Luo’s encoder/decoder training by explicitly constraining predicted watermark values based on classification outcomes.
The combination of Luo and YANG fails to disclose encourage retrieval of the digital watermark from images subjected to benign transforms;
However, Aggarwal in the same field of endeavor discloses encourage retrieval of the digital watermark from images subjected to benign transforms, ([Aggarwal, [claim 16]” constructing a digest N…which survives benign modifications”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Luo to include a minimum amount of error [corresponds to the benign image, and the digital watermark]; a maximum amount of error [corresponds to the malicious image, and the digital watermark; selecting, from a set of malicious transforms, a malicious transform; performing the malicious transform on the watermarked image to generate a malicious image; discourage retrieval of the digital watermark from images subjected to malicious transforms, such that the digital watermark is semi-fragile as suggested by YANG to further include encourage retrieval of the digital watermark from images subjected to benign transforms as taught by Aggarwal. One of ordinary skill in the art would have been motivated to do so because incorporating Aggarwal’s semi-fragile watermark technique , which preserves watermark retrieval after benign image modifications while allowing retrieval to fail following malicious modifications, would have predictably improved Luo’s machine-learning watermark training and complemented Yang’s benign-versus-adversarial watermark verification by increasing robustness to expected image processing without reducing the ability to detect maliciously transformed images.
Claims 3, 14-15 are rejected under 35 U.S.C. §103 as being unpatentable over Luo et al (US 20230362399 A1) [hereinafter " Luo "] in view of YANG et al. (US 12147583 B2) [hereinafter "YANG"] and in view of Aggarwal et al. ( US 6834344 B1) [hereinafter " Aggarwal "] as applied to claims 2, 1 and 13, 12 and further in view of Zhang et al. (“Invisible steganography via generative adversarial networks”, 2018) [hereinafter "Zhang"].
As per claim 3, the references as combined above disclose the computer-implemented method of claim 2. the combination of Aggarwal and YANG does not teach wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image. However, Zhang in the same field of endeavor discloses wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image. ([Zhang, abstract ]" We introduce the generative adversarial networks to strengthen the security by minimizing the divergence between the empirical probability distributions of stego images and natural images " ).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Aggarwal to include adjusting at least one weight of the decoder during a learning phase of the decoder by at least learning a minimum amount of error between the first predicted value, which corresponds to the benign image, and the digital watermark and learning a maximum amount of error between the second predicted value, which corresponds to the malicious image, and the digital watermark discloses as suggested by YANG to further include wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image as taught by Zhang . One of ordinary skill in the art would have been motivated to do so because Zhang teaches using a discriminator during training to compute an adversarial loss indicative of whether the watermark is recoverable, which directly informs adjusting decoder weights to distinguish benign from malicious transformations.
As per claim 14, the references as combined above disclose the system of claim 13. the combination of Aggarwal and YANG does not teach wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image. However, Zhang in the same field of endeavor discloses wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image. ([Zhang, , section 3.3]" The basic model can finish the entire hiding and revealing process, so we use the basic model as the generator, and introduce a CNN-based steganalysis model as the discriminator and the steganalyzer " ).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Aggarwal to include adjusting at least one weight of the decoder during a learning phase of the decoder by at least learning a minimum amount of error between the first predicted value, which corresponds to the benign image, and the digital watermark and learning a maximum amount of error between the second predicted value, which corresponds to the malicious image, and the digital watermark discloses as suggested by YANG to further include wherein the training to minimize the error between the image and the watermarked image further comprises using a discriminator to determine the adversarial loss indicative of whether the watermarked image is the image as taught by Zhang . One of ordinary skill in the art would have been motivated to do so because Zhang teaches using a discriminator during training to compute an adversarial loss indicative of whether the watermark is recoverable, which directly informs adjusting decoder weights to distinguish benign from malicious transformations.
Claims 5, 7, 16 and 18 are rejected under 35 U.S.C. §103 as being unpatentable over Luo et al (US 20230362399 A1) [hereinafter " Luo "] in view of YANG et al. (US 12147583 B2) [hereinafter "YANG"] and in view of Aggarwal et al. ( US 6834344 B1) [hereinafter " Aggarwal "] as applied to claims 1 and 12 and further in view of Goodfellow et al. (“EXPLAINING AND HARNESSING ADVERSARIAL EXAMPLES”, 2015) [hereinafter "Goodfellow"]
As per claim 5, the references as combined above disclose the computer-implemented method of claim 1. The combination of Aggarwal and YANG does not explicitly teaches wherein the benign transform is selected from a set of benign transforms comprising an image compression of the watermarked image, a color adjustment of the watermarked image, a lighting adjustment of the watermarked image, a contrast adjustment of the watermarked image, a downsizing of the watermarked image, an upsizing of the watermarked image transformation, a horizontal and/or vertical translation of the watermarked image, and/or a rotation of the watermarked image. However, Goodfellow in the same field of endeavor discloses wherein the benign transform is selected from a set of benign transforms comprising an image compression of the watermarked image, a color adjustment of the watermarked image, a lighting adjustment of the watermarked image, a contrast adjustment of the watermarked image, a downsizing of the watermarked image, an upsizing of the watermarked image transformation, a horizontal and/or vertical translation of the watermarked image, and/or a rotation of the watermarked image. ([Goodfellow, section 3 ]" In many problems, the precision of an individual input feature is limited. For example, digital images often use only 8 bits per pixel so they discard all information below 1/255 of the dynamic range. Because the precision of the features is limited, it is not rational for the classifier to respond differently to an input x than to an adversarial input x˜ = x + η if every element of the perturbation η is smaller than the precision of the features. Formally, for problems with well-separated classes, we expect the classifier to assign the same class to x and x˜ so long as ||η||∞ < c, where c is small enough to be discarded by the sensor or data storage apparatus associated with our problem "). Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Aggarwal to include adjusting at least one weight of the decoder during a learning phase of the decoder by at least learning a minimum amount of error between the first predicted value, which corresponds to the benign image, and the digital watermark and learning a maximum amount of error between the second predicted value, which corresponds to the malicious image, and the digital watermark discloses as taught by YANG to further include wherein the benign transform is selected from a set of benign transforms comprising an image compression of the watermarked image, a color adjustment of the watermarked image, a lighting adjustment of the watermarked image, a contrast adjustment of the watermarked image, a downsizing of the watermarked image, an upsizing of the watermarked image transformation, a horizontal and/or vertical translation of the watermarked image, and/or a rotation of the watermarked image as suggested by Goodfellow. One of ordinary skill in the art would have been motivated to do so because Goodfellow expressly teaches designing a watermarking system to be robust to a defined set of benign image transformations, including compression, resizing, and brightness and contrast adjustments, while remaining fragile to malicious manipulations.
As per claim 7, the references as combined above disclose the computer-implemented method of claim 1. The combination of Aggarwal and YANG does not teach wherein the malicious transform uses a mask that replaces at least a portion of the watermarked image with another image portion. However ,Goodfellow in the same field of endeavor discloses wherein the malicious transform uses a mask that replaces at least a portion of the watermarked image with another image portion([Goodfellow, abstract ]” adversarial examples—inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed in-put results in the model outputting an incorrect answer with high confidence”). Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Aggarwal to include adjusting at least one weight of the decoder during a learning phase of the decoder by at least learning a minimum amount of error between the first predicted value, which corresponds to the benign image, and the digital watermark and learning a maximum amount of error between the second predicted value, which corresponds to the malicious image, and the digital watermark discloses as taught by YANG to further include wherein the malicious transform uses a mask that replaces at least a portion of the watermarked image with another image portion as suggested by Goodfellow. One of ordinary skill in the art would have been motivated to do so because incorporating Goodfellow’s teaching that neural networks are vulnerable to intentionally applied worst-case perturbations that cause incorrect model outputs with high confidence.
As per claim 16, the substance of the claimed invention is identical to that of claim 5. Accordingly, this claim is rejected under the same rationale as claim 5.
As per claim 18, the substance of the claimed invention is identical to that of claim 7. Accordingly, this claim is rejected under the same rationale as claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Xi et al. (CN104700345A) discloses Method for improving detection rate of semi-fragile watermark authentication by establishing Benford's law threshold value library.
Yuan et al. ("Semi-Fragile Neural Network Watermarking Based on Adversarial Examples," in IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 8, no. 4, pp. 2775-2790) discloses Semi-Fragile Neural Network Watermarking Based on Adversarial Examples.
NORRIS et al. (US 20220084223 A1) discloses Focal Stack Camera As Secure Imaging Device And Image Manipulation Detection Method.
Tian et al.(US 20020076084 A1) discloses Measuring Quality Of Service Of Broadcast Multimedia Signals Using Digital Watermark Analyses
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Komi N. AMEVIGBE whose telephone number is (571)272-3381. The examiner can normally be reached Monday-Friday 2pm-10pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carl Colin can be reached at (571) 272-3862. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.N.A./Examiner, Art Unit 2493
/CARL G COLIN/Supervisory Patent Examiner, Art Unit 2493
Prosecution Insights