Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments, filed 02 October 2025, have been fully considered but they are not persuasive. Applicant’s argument as understood by Examiner is paraphrased below:
Taking the amendments to independent claims 1, 14, and 23 into account, the disclosures of He, Rasmussen, and Rhoads, either alone or in combination, fail to teach, disclose, or suggest the newly-added limitation of mitigating the one or more distortions of the portion of the image prior to image decoding.
Respectfully, Examiner disagrees.
Rasmussen explicitly discloses distortion correction of two categories. Large distortions within the method and system of Rasmussen are explicitly handled by camera pose estimation for spatial correction using predetermined keypoints (pg. 2 section II B para. 2 and pg. 3 section II E , paras. 1-2). Furthermore, small-scale (pixel-wise) resolutions are injected within the training workflow to ensure that the model ignores these small corruptions within the method (pg. 2 section II B para. 2). Thus, the rejection under 35 U.S.C. § 103 of the independent claims and their respective dependents over He in view of Rasmussen and in further view of Rhoads is maintained.
Claim Interpretation
The prior action identified certain terms as invoking 112(f). Applicant does not respond to this interpretation, either in the form of traversal, acknowledgement, or amendment. The terms continue to invoke 112(f) as the nonce words would not be understood to have meaning beyond a reference back to the specification.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 14-18, and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over He et al. (US PG Pub 20210334929, hereinafter “He”), in view of Rasmussen et al. (“DeepMorph: A System for Hiding Bitstrings in Morphable Vector Drawings”, Signal Processing: Image Communication, 2021, hereinafter “Rasmussen”).
Regarding claim 1, He discloses a computer-implemented method, comprising: receiving an image (paras. 0006, 0014, wherein the image is obtained from a user’s computing device); determining that a digital watermark is embedded in a portion of the image (paras. 0005-0008, wherein the watermark is detected by analysis by a computing system; specifically, the computing system examines pixel values, patterns, and attributes of a source image to determine the potential presence of encoded data in the form of a digital watermark), wherein the digital watermark is not visually discernible by a human viewer (paras. 0003-0005, wherein the input image contains a digital watermark imperceptible to a human viewer); in response to determining that the digital watermark is embedded in the portion of the image, obtaining a first data item encoded within the digital watermark embedded in the portion of the image (paras. 0006-0012, wherein the identification of a digital watermark would lead to the decoding of the watermark by identification of data intrinsic to the watermark or the watermarked image), including predicting one or more distortions present in the portion of the image relative to an original version of the portion of the image (paras. 0015 and 0074, wherein imaging at many zoom levels and zoom resolutions, along with obscuring based on calculated binary pixel values, would involve image distortion, due to resolution changes potentially resulting in altered pixel values); modifying the portion of the image based on the predicted one or more distortions (paras. 0015, 0074, and 0080, wherein the multiple versions of the possibly encoded source image at varying scales and resolutions represent the modification of the portions of the image, wherein these images constitute the one or more distortions, and the possibly encoded source image still retains its predictability/watermarking pattern); and decoding the modified portion of the image to obtain the first data item encoded within the digital watermark (paras. 0074 and 0081-0082, wherein the disclosed decoding may comprise a variety of operations, including a nearest-neighbor based binarization algorithm); and validating an item depicted in the image based on the first data item (paras. 0075 and 0081-0084 and Fig. 3 element 316 wherein the extracted data is determined as a result of the decoding and wherein a database is accessed using the decoded, extracted data, to determine records).
Specifically, He discloses a method and system of detecting watermarks in captured source images, specifically directed to using a predictive model on different image zooms and resolutions of a potential encoding image to determine encoded values, prior to decoding the encoded values.
He does not explicitly disclose wherein the modifying of the portion of the image based on the predicted one or more distortions specifically comprises mitigating the one or more distortions of the portion of the image, wherein encoding and decoding of the watermarks are performed by an encoding machine learning network and a decoding machine learning network, wherein the decoder ML model and the encoder ML model are jointly trained as part of an end-to-end learning pipeline, or wherein a distortion detector machine learning model is used to determine distortions present in the received image portion compared to an original version of the image portion.
However, Rasmussen discloses wherein the modifying of the portion of the image based on the predicted one or more distortions specifically comprises mitigating the one or more distortions of the portion of the image (pg. 2 section II B para. 2 and pg. 3 section II E , paras. 1-2, wherein large distortions within the method and system of Rasmussen are explicitly handled by camera pose estimation for spatial correction using predetermined keypoints, wherein small-scale (pixel-wise) resolutions are injected within the training workflow to ensure that the model ignores these small corruptions within the method, and wherein the distortion mitigation mechanism of the method and system of Rasmussen occurs prior to the decoding mechanism), wherein the encoding and decoding of the watermarks was performed by an encoding machine learning network and a decoding machine network, respectively (pg. 4, Section III, subsections B and C, wherein sections B and C describe small, trainable convolutional neural networks/CNNs, which are a type of machine learning algorithm as the encoder model and decoder model of the watermarking system, respectively); distortion detection being performed by a machine learning distortion detector (Section II, subsection E, paras. 2-3, wherein the distortion detection methodology is a CNN derivative utilizing bounding boxes to identify and detect distortions); and wherein the decoder ML model and the encoder ML model are jointly trained as part of an end-to-end learning pipeline (pg. 4, Section III, subsections D and E, wherein section D describes a mechanism enabling end-to-end training as a method of jointly training the decoder and encoder ML models, and wherein section E describes the actual training process, including weight locking and unlocking and the sigmoid cross-entropy loss function).
Specifically, Rasmussen discloses a method and a system for hiding bitstrings in morphable vector drawings. The method of Rasmussen involves training an ML encoder network and an ML decoder network jointly (end-to-end) in order to obtain networks which can both encode bitstrings within images through image morphing and decode bitstrings from morphed images, similar to the principle of QR codes, but not limited to the shapes and lines of a QR code.
Therefore, both He and Rasmussen disclose methods and systems of encoding and decoding hidden image content generated by creating perturbations within images only perceptible using pixel-level detection processes.
Thus, it would have been obvious to one having ordinary skill in the art prior to the effective filing date of the claimed invention to have utilized the machine learning encoding and decoding models, and end-to-end training methods and system of Rasmussen within the method and system of He as the application of a known method to a known device ready for improvement, yielding the predictable result of a more efficient and more robust encoding and decoding framework for the operations of He with the use of Rasmussen’s dual paired CNN, and further yielding a far more efficient training method (Rasmussen Section III, subsection E, “Empirically we observe that training the full network from scratch is slow and prone to divergence. This is solved by applying a two-stage training schedule”).
Claims 14 and 23 are rejected, mutatis mutandis, for reasons similar to claim 1.
Regarding claim 23, Examiner is interpreting “data processing apparatus” to be a processor or a microprocessor suitable for the execution of a computer program, as specified within paras. 00152-00153 of the Specification.
Regarding claim 23, He further discloses a non-transitory computer-readable medium storing instructions (para. 0044, “defined by instructions stored in memory on one or more computer-readable storage devices”) that, when executed by one or more data processing apparatus (para. 0044, “one or more processors of the server system”), cause the one or more data processing apparatus to perform operations comprising the method of claim 1 and the system of claim 14.
Regarding claims 2, 15, and 24, He and Rasmussen disclose all limitations of claims 1, 14, and 23, respectively. He further discloses wherein determining the digital watermark is embedded in a portion of the image comprises generating a segmentation map of the image (paras. 0020-0022 and 0054-0056, wherein the binarization of the image amidst the nearest neighbors mapping and search would constitute generation of a segmentation map of the image, wherein image segments would consist of neighborhoods of pixels with the same classification); and identifying based on the segmentation map of the image, the portion of the image within which the digital watermark is embedded (para. 0055, wherein the watermark predictive model, which comprises the aforementioned binary classification model, is implemented to decide whether to decode, and subsequently decode, a potentially encoded image portion).
Regarding claims 3 and 16, He and Rasmussen disclose all limitations of claims 1 and 14, respectively. He further discloses wherein predicting one or more distortions present in the portion of the image comprises determining a zoom factor indicating an estimated level of zoom that the portion of the image has undergone relative to the original version of the portion of the image (paras. 0079-0081, wherein the zoom factors comprise different zoom levels based on determined and target resolutions determined over the course of generating different scaled version of the image); determining a vertical distortion factor indicating a vertical scaling that the image has undergone relative to the original version of the portion of the image; and determining a horizontal distortion factor indicting a horizontal scaling that the image has undergone relative to the original version of the portion of the image (paras. 0079-0082 for both horizontal and vertical distortion factors, wherein for multiple scaled versions of an image, the vertical and horizontal “distortion factors” comprise scale percentage changes within the select image scales which cause different resolutions and image contents to become visible, a key step in determining the presence of a digital watermark).
Regarding claims 4 and 17, He and Rasmussen disclose all limitations of claims 3 and 16, respectively. He further discloses wherein modifying the portion of the image based on the predicted one or more distortions while preserving the digital watermark embedded in the portion of the image comprises modifying the portion of the image based on the determined zoom factor, the horizontal distortion factor, and the vertical distortion factor (paras. 0079-0082, wherein the zoom factor (and the subsequently derived horizontal and vertical distortion factors) are utilized in order to determine the presence of a digital watermark and digitally-encoded content while “match[ing] the original size of the source image to ensure predictability in the pattern of the encoded and non-encoded pixels…generat[ing] a set of scaled versions of the possibly encoded source image” as stated in para. 0080).
Regarding claims 5 and 18, He and Rasmussen disclose all limitations of claims 4 and 17. He further discloses wherein modifying the portion of the image based on the determined zoom factor, the horizontal distortion factor, and the vertical distortion factor comprises zooming-in or zooming-out the portion of the image to adjust the estimated level of zoom that the image has undergone relative to the original version of the portion of the image as indicated by the vertical distortion factor (paras. 0079-0082, wherein different scaled versions at different zoom levels of the original portion of the original image are taken as a modification in order to further detect the potential presence of a digital watermark or encoded data, wherein scaling according to each distortion factor is performed as a function of original size and selected scale); and scaling the portion of the image as indicated by the horizontal distortion factor (paras. 0079-0082, wherein different scaled versions at different zoom levels of the original portion of the original image are taken as a modification in order to further detect the potential presence of a digital watermark or encoded data, wherein scaling according to each distortion factor is performed as a function of original size and selected scale).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over He in view of Rasmussen and in further view of Rhoads et al. (US PG Pub. 20200193553, hereafter referred to as Rhoads).
Regarding claim 6, He and Rasmussen disclose all limitations of claim 1. Rasmussen further discloses wherein the decoder machine learning model includes a decoder neural network that includes a first plurality of neural network layers, including a plurality of fully-connected convolutional layers and a max pooling layer; and the first plurality of neural network layers followed by a fully-connected convolutional layer and a pooling layer (pg. 4, section III, subsection C, wherein the decoder consists of a CNN, with 2 fully connected layers and a classification head). Thus, it would have been obvious to one having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the disclosures of He and Rasmussen according to the rationale of claim 1.
He and Rasmussen do not explicitly disclose the use of a pooling or a max pooling function within the CNN architecture of the decoder.
However, Rhoads discloses wherein a CNN consists of neural network layers followed by fully-connected convolutional layers and a pooling layer, which in a potential embodiment would comprise a max pooling layer (paras. 0401-0402, wherein a pooling layer is utilized to down-sample and aggregate data from the fully connected layers to reduce data dimensionality and drive an output, wherein max pooling is used to reduce feature map dimensions, and wherein the pooling layer would be critical to determining the output of a CNN). Specifically, Rhoads discloses a method and system for decoding steganographic digital watermark signals from nearby imagery or pixels using an optical scanner. Therefore, He and Rasmussen and Rhoads all disclose methods and systems of extracting digital watermark signals from larger areas of image content.
Thus, it would have been obvious to one having ordinary skill in the art prior to the effective filing date of the claimed invention to have utilized the pooling and max pooling layer of Rhoads within the method and system of He as modified by Rasmussen as an “obvious to try” methodology of obtaining results from a CNN; more specifically, the utilization of a (max) pooling layer within a CNN is one of a finite number of identified, predictable ways to reduce dimensionality of feature maps in a network to obtain actionable network output.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROHAN TEJAS MUKUNDHAN whose telephone number is (571)272-2368. The examiner can normally be reached Monday - Friday 9AM - 6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 5712723838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROHAN TEJAS MUKUNDHAN/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698