DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-19, all the claims pending in the application, are rejected.
Claim Interpretation
Independent claim 19 recites a “non-transient computer-readable storage medium”. The term “non-transient” is not expressly defined in the specification of the subject application; thus, in accordance with MPEP § 2111.01, the ordinary and customary meaning of the term is adopted. For example, Merriam-Webster defines the term “transient” as follows:
PNG
media_image1.png
234
798
media_image1.png
Greyscale
Logically, the claimed term “non-transient” is thus interpreted as equivalent to “non-transitory”. Accordingly, the subject matter of independent claim 19 does not encompass transitory signals and is therefore eligible subject matter under 35 USC § 101.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-4, 6, 9, 11-13, 15 and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2, in line 4, recites “a watermark recognition network”. It is unclear whether this is the same or different from the identically-recited “watermark recognition network” of claim 1 from which claim 2 depends. If they are the same, the Examiner recommends amending claim 2 to recite “[[a]]the watermark recognition network”. Otherwise, the Examiner recommends amending claim 2 to include a distinguishing modifier (e.g., “second” or “another”).
Claim 2, in line 5, recites “a watermark detection result”. It is unclear whether this is the same or different from the identically-recited “watermark detection result” of claim 1 from which claim 2 depends. If they are the same, the Examiner recommends amending claim 2 to recite “[[a]]the watermark detection result”. Otherwise, the Examiner recommends amending claim 2 to include a distinguishing modifier (e.g., “second” or “another”).
Claims 3-4 inherit the deficiencies of their parent claim 2.
Also, claim 4 recites “taking an identification of a generation algorithm, a confidence level corresponding to which is greater than a preset threshold”. It is unclear whether the “identification”, “generation algorithm”, “confidence level” and “present threshold” are the same or different from the identically recited limitations in claim 2 from which claim 4 depends.
Claim 6 recites “based on a watermark label” in line 10. It is unclear whether this “watermark label” is the same or different from the identically-recited “watermark label” recited earlier in the claim in line 3.
Claim 9 inherits the deficiencies of its parent claim 6.
Claims 11-13, 15 and 18 recite features nearly identical to those recited in claims 2-4, 6 and 9, respectively. Accordingly, claims 11-13, 15 and 18 are rejected for reasons analogous to those discussed above in conjunction with 2-4, 6 and 9, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 6, 9-10, 15, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over “Multi-Class Blind Steganalysis Using Deep Residual Networks” by Singhal et al. (hereinafter “Singhal”) in view of “Extracting Embedded Messages Using Adaptive Steganography Based on Optimal Syndrome-Trellis Decoding Paths” by Li et al. (hereinafter “Li”).
As to independent claim 1, Singhal discloses a method for watermark detection (Abstract discloses that Singhal is directed to a “multi class blind steganalysis technique for images” which has been successfully applied for detecting the “presence of hidden content” in “spatial and JPEG images”), comprising: acquiring an image to be detected; inputting the image to be detected into a watermark recognition network and acquiring a watermark detection result corresponding to the image to be detected, wherein the watermark detection result comprises presence of a hidden watermark (Sections 3-4 and Fig. 6 shows a “Deep Residual Network” which inputs an image and outputs a label of a predicted “steganographic algorithm” used to embed hidden content in the image, the algorithms including “spatial domain” algorithms (“WOW, S-UNIWARD, HILL, and MiPOD”), frequency/“JPEG domain” algorithms (“F5, Model Based, Steghide, and Outguess”), and no algorithm (“cover”; i.e., no hidden content)).
Singhal does not expressly disclose watermark extraction or extracting the hidden watermark in the image to be detected based on at least one extraction algorithm corresponding to the watermark detection result when detected.
Li, like Singhal, is directed to “steganalysis” and “embedded messages” in images (Abstract). Specifically, Li discloses that the “ultimate goal” of steganalysis “is to extract embedded messages” and thus outlines a decoding path for doing just that (Abstract and Section 3). Notably, Li discloses that the decoding can be applied to “secret images that were embedded with secret messages via HUGO, WOW and S-UNIWARD steganography algorithms” (Section 4), several of which overlap with the algorithms detected by Singhal’s multi-class steganalysis technique (Section 4 of Singhal).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Singhal to extract the embedded message in the image based on the steganography algorithm used to embed the message, as taught by Li, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have served the “ultimate goal” of steganalysis – namely, “to extract embedded messages”, as taught by Li (Abstract).
As to claim 6, Singhal as modified above further teaches training the watermark recognition network, comprising: acquiring sample data carrying a watermark label in a plurality of training stages, wherein the watermark label is used for characterizing an identification of a generation algorithm for generating a hidden watermark of the sample data; for a current training stage, inputting sample data of the current training stage into an intermediate recognition network obtained after a training in a previous training stage of the current training stage is completed to obtain an output result of the intermediate recognition network, wherein sample data of a first training stage is inputted into an initial recognition network; and based on a watermark label carried by the sample data in the current training stage and the output result, training the intermediate recognition network obtained after the training in the previous training stage of the current training stage is completed (Sections 3-4 and Fig. 5 of Singhal discloses a process of “training [the] used architecture” in which “weight and bias…are learnt by minimizing loss in softmax function” according to equation 7, wherein M sample training images are each given ground truth “labels” L which identify the “steganographic algorithms” used to embed hidden content in the corresponding training image, wherein each sample image is input to the Deep Residual Network which outputs – in response – a prediction/“outcome” oil of sample training image i as predicted label l, and the “Weight and bias during back propagation are updated” according to the loss function 7 which takes into account the prediction oil of sample training image i relative to its ground label L, and wherein the number of training stages/“epochs is set to 200”).
As to claim 9, Singhal as modified above further teaches that the sample data comprises video images and non-video images, and hidden watermarks of the video images and the non-video images are generated based on at least one selected from a group consisting of a video watermark generation algorithm and an image watermark generation algorithm (Abstract of Singhal discloses that the media in which the “steganography algorithm” embeds “hidden content” can be “image” or “video”).
Independent claim 10 recites a computer device, comprising: at least one processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the at least one processor; the at least one processor communicates with the memory through the bus upon running of the computer device, and the machine-readable instructions, upon being executed by the at least one processor, execute a method (Section 4 of Singhal discloses that the algorithm is implemented “in Python 3.6 using Spyder and NVidia Quadro K4200 GPU card”, wherein such “Python 3.6” software instructions must be stored in memory (for example “database” disclosed in Section 4) and must be communicated to the “GPU” processor by a bus in order to be executed) comprising the steps recited in the method of claim 1. Accordingly, claim 10 is rejected for reasons analogous to those discussed above in conjunction with claim 1.
Claims 15 and 18 recite features nearly identical to those recited in claims 6 and 9, respectively. Accordingly, claims 15 and 18 are rejected for reasons analogous to those discussed above in conjunction with claim 6 and 9, respectively.
Independent claim 19 recites a non-transient computer-readable storage medium storing computer programs, wherein the computer programs, upon being run by at least one processor, execute a method (Section 4 of Singhal discloses that the algorithm is implemented “in Python 3.6 using Spyder and NVidia Quadro K4200 GPU card”, wherein such “Python 3.6” software instructions must be stored in memory (for example “database” disclosed in Section 4) in order to be executed by the “GPU” processor) comprising the steps recited in the method of claim 1. Accordingly, claim 19 is rejected for reasons analogous to those discussed above in conjunction with claim 1.
Claims 2-4 and 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Singhal in view of Li and further in view of U.S. Patent Application Publication No. 2020/0394441 to Wen et al. (hereinafter “Wen”).
As to claim 2, Singhal as modified above further teaches that the watermark detection result comprises an identification of at least one target generation algorithm, and the target generation algorithm is used for generating the hidden watermark in the image to be detected (Sections 3-4 and Fig. 6 shows a “Deep Residual Network” which inputs an image and outputs a label of a predicted “steganographic algorithm” used to embed hidden content in the image); the inputting the image to be detected into a watermark recognition network and acquiring a watermark detection result corresponding to the image to be detected comprises: inputting the image to be detected into the watermark recognition network to obtain at least one confidence level corresponding to at least one generation algorithm outputted by the watermark recognition network, wherein the at least one confidence level corresponding to the at least one generation algorithm is used for characterizing a probability that the hidden watermark in the image to be detected is generated based on the at least one generation algorithm; and taking an identification of a generation algorithm as the identification of the target generation algorithm comprised in the watermark detection result (Sections 2-4 and Fig. 6 shows a “Deep Residual Network” which inputs an image and outputs a label of a predicted “steganographic algorithm” used to embed hidden content in the image, wherein the label is selected from “softmax” layer that “calculates the probability of each event over all possible events”).
The proposed combination of Singhal and Li does not expressly disclose that the generation algorithm, a confidence level corresponding to which is greater than a preset threshold, is selected.
Wen, like Singhal, is directed to a “convolutional neural network” CNN which inputs an image an outputs an “image classification”, wherein the CNN includes a softmax function that outputs “a likelihood that the image is properly classified in each possible classification that the work model is configured to determine” (Abstract and [0033, 0038-0039]). Wen discloses that the image classification system 118 may only present a recommendation of a particular classification or result to the end user if the corresponding likelihood calculated by the softmax function satisfies a certain threshold (e.g., display only if the probability is greater than 0.5)” ([0039]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Singhal’s CNN architecture such that the softmax function selects the final label only if its probability exceeds a threshold, as taught by Wen, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have avoided returning low confidence labels.
As to claim 3, Singhal as modified above further teaches that the watermark recognition network determines a plurality of confidence levels corresponding to a plurality of generation algorithms (Sections 2-4 and Fig. 6 of Singhal discloses a “Deep Residual Network” which inputs an image and outputs a label of a predicted “steganographic algorithm” used to embed hidden content in the image, wherein the label is selected from “softmax” layer that “calculates the probability of each event over all possible events”) based on following methods: extracting feature images with a plurality of sizes corresponding to the image to be detected; and determining the plurality of confidence levels corresponding to the plurality of generation algorithms based on the feature images with the plurality of sizes (Fig. 6 of Singhal shows that the output of each block is a plurality of feature maps/images with progressively smaller sizes (e.g., block 1 outputs 64 feature maps of size 128x128, block 2 outputs 128 feature maps of size 64x64, block 3 outputs 256 feature maps of size 32x32, block 4 outputs 512 feature maps of size 16x16, and block 5 outputs 1024 feature maps of size 8x8), wherein the last block 6 flattens the feature maps for input to the fully connected layer, then the softmax layer outputs the probabilities for the respective steganographic algorithms based on the upstream feature maps).
As to claim 4, Singhal does not expressly disclose that the taking an identification of a generation algorithm, a confidence level corresponding to which is greater than a preset threshold, as the identification of the target generation algorithm comprised in the watermark detection result comprises: determining candidate generation algorithms, confidence levels corresponding to which are greater than the preset threshold; and when a number of candidate generation algorithms exceeds a preset number, among the candidate generation algorithms, determining identifications of the preset number of target generation algorithms, confidence levels corresponding to which meet a preset condition.
Wen, like Singhal, is directed to a “convolutional neural network” CNN which inputs an image an outputs an “image classification”, wherein the CNN includes a softmax function that outputs “a likelihood that the image is properly classified in each possible classification that the work model is configured to determine” (Abstract and [0033, 0038-0039]). Wen discloses that the image classification system 118 may only present a recommendation of a particular classification or result to the end user if the corresponding likelihood calculated by the softmax function satisfies a certain threshold (e.g., display only if the probability is greater than 0.5)” ([0039]). Wen discloses an “additional…criteria” that “the image classification system 118 may only present a recommendation of a particular classification…when the corresponding likelihood is at least a threshold amount greater than the second-highest likelihood determined for the set of possible classifications” even when both (i.e., more than a preset number of 1) exceed the threshold ([0039]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Singhal’s CNN architecture such that the softmax function selects the final label only if, among classes that exceed a threshold, the one with the highest likelihood is at least a threshold amount greater than the likelihood of the other classes, as taught by Wen, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have avoided returning low confidence labels.
Claims 11-13 recite features nearly identical to those recited in claims 2-4, respectively. Accordingly, claims 11-13 are rejected for reasons analogous to those discussed above in conjunction with claims 2-4, respectively.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Singhal in view of Li and further in view of U.S. Patent Application Publication No. 2021/0027470 to Lin et al. (hereinafter “Lin”).
As to claim 5, Singhal as modified above further teaches that the watermark recognition network is obtained through training in a plurality of training stages (Section 4 of Singhal discloses that the training is performed over 200 epochs). Singhal as modified above does not expressly disclose, sample data used in different training stages is sample data obtained by processing based on different data enhancement methods, a complexity of a data enhancement method corresponding to sample data used in a current training stage is higher than a complexity of a data enhancement method in a previous training stage of the current training stage.
Lin, like Singhal, is directed to training a neural network (Abstract). Lin discloses that the training uses “an easy-to-hard data augmentation” strategy in the training process, wherein easy augmentations are applied to the training images in a first training, then harder augmentations are applied to the training images in a subsequent training, and finally the hardest/most complex augmentations are applied to the training images in a final training ([0025, 0038-0040, 0091]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Singhal and Li to apply progressively more complex augmentations to the training images at each stage of training, as taught by Lin, in order to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have “flexibly provide[d] extensive training to the…neural network even where the availability of training images is limited” ([0091] of Lin).
Claim 14 recites features nearly identical to those recited in claim 5. Accordingly, claim14 is rejected for reasons analogous to those discussed above in conjunction with claim 5.
Claims 7-8 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Singhal in view of Li and Lin and further in view of “Image Data Augmentation Approaches: A Comprehensive Survey and Future Directions” to Kumar et al. (hereinafter “Kumar”).
As to claim 7, the proposed combination of Singhal, Li, and Lin further teaches that the plurality of training stages comprise a first training stage for data enhancement processing, a second training stage for data enhancement processing, and a third training stage for data enhancement processing ([0025, 0038-0040, 0091] of Lin discloses that the training uses “an easy-to-hard data augmentation” strategy in the training process, wherein easy augmentations are applied to the training images in a first training, then harder augmentations are applied to the training images in a subsequent training, and finally the hardest/most complex augmentations are applied to the training images in a final training; the reasons for combining the references are the same as those discussed above in conjunction with claim 5). The proposed combination of Singhal, Li and Lin does not expressly disclose enhancement processing based on a pixel level, based on geometric transformation, and based on both the pixel level and the geometric transformation.
Kumar, like Lin, is directed to data augmentation techniques for training a neural network, wherein data augmentation is classified into basic and advanced branches, the latter encompassing “more complex techniques” (Abstract and Section 2). Kumar discloses “basic image data augmentations” including pixel level augmentations like “color manipulations…made by altering pixel values in the image” and “geometric manipulations” like rotation and translation, and “advanced data augmentations” including “image mixing data augmentations” (Section II).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Singhal, Li, and Lin to perform Lin’s easy-to-hard data augmentation stages based on a pixel level, based on geometric transformation, and based on both the pixel level and the geometric transformation, as taught by Kumar, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have prevented “overfitting problems” that result from “limited labeled data” (Abstract of Kumar).
As to claim 8, Singhal as modified above further teaches that a data enhancement processing in the third training stage further comprises image fusion (Fig. 2 and Section 2 of Kumar contemplates “multi-images mixing” as an “advanced image data augmentation”; the reasons for combining the references are the same as those discussed above in conjunction with claim 7).
Claims 16-17 recite features nearly identical to those recited in claims 7-8, respectively. Accordingly, claims 16-17 are rejected for reasons analogous to those discussed above in conjunction with claims 7-8, respectively.
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lwowski (U.S. Patent Application Publication No. 2021/0150312) is directed to detecting embedded data in a digital image using a plurality of steganography analyzers that detect the use of different steganography algorithms including spatial and frequency domain algorithms. These teachings are pertinent to the inventive concept of the subject application.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN M CONNER whose telephone number is (571)272-1486. The examiner can normally be reached 10 AM - 6 PM Monday through Friday, and some Saturday afternoons.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN M CONNER/Primary Examiner, Art Unit 2663
Prosecution Insights