DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim 1 is cancelled and claims 2-21 were added as new by preliminary amendments.
Claim Objections
Claim 21 is objected to because of the following informalities: Claim 21 is a system claim but referring to claim 10 which is medium claim. It should be referring claim 16. Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Instant application 19/056193
US 12238322 B2
2. (New) A computer-implemented method, comprising:
receiving, from a client device and over a network, a first image;
determining, using a watermark detector model, that a first digital watermark is likely embedded in the first image;
decoding, using a watermark decoder machine learning model, a first version of the first image to obtain the first digital watermark, wherein the decoder machine learning model is jointly trained with an encoder machine learning model as part of an end-to-end learning pipeline, wherein the encoder machine learning model is trained to generate a digital watermark that is embedded into an image and the decoder machine learning model is trained to decode an image to obtained a digital watermark embedded in the image; and
validating a first source of the first image based on the first digital watermark.
Note: error value is difference between source image and watermarked image
1. A computer-implemented method for jointly training an encoder machine learning model that generates a digital watermark that is embedded into an image and a decoder machine learning model that decodes a first data item encoded within the digital watermark that is embedded into the image, wherein the training comprises:
obtaining a first plurality of training images and a plurality of data items, wherein each data item in the plurality of data items is a data item that is to be encoded within a digital watermark to be embedded into a training image; for each training image in the first plurality of training images: obtaining a data item from the plurality of data items; generating, using the encoder machine learning model to which the data item is provided as input, a first digital watermark that encodes the data item; tiling two or more instances of the first digital watermark to generate a second digital watermark; combining the second digital watermark with the training image to obtain a watermarked training image;
applying one or more distortions to the watermarked training image; predicting, using a distortion detector machine learning model, the one or more distortions present in the watermarked training image; modifying the watermarked training image based on the predicted one or more distortions while preserving the second digital watermark embedded in the distorted, watermarked training image; and
decoding, using the decoder machine learning model, the modified watermarked training image to obtain a decoded data item that is predicted to be embedded in the second digital watermark embedded in the distorted, watermarked training image;
determining a first error value based on the watermarked training image and the training image; determining a second error value based on the decoded data item and the data item; and adjusting one or more training parameters of the encoder machine learning model and the decoder machine learning model to minimize the first error value and the second error value.
Claims 2-21 are rejected on the ground of non-statutory double patenting as being unpatentable over claims 1-18of U.S. Patent No. US 12238322 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because of similar limitations with obvious variations .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2-3, 8, 10-11, 15-17, 21 are rejected under 35 U.S.C. 103 as being unpatentable over Gharaibeh et al(US 20180270388 A1) in view of JEONG et al(US 20220130019 A1).
With regards to claim 2, 10, 16 Gharaibeh discloses, A computer-implemented method, comprising:
receiving, from a client device and over a network, a first image ([0020] The client computer 104 can be any type of computing device that presents images and other content to one or more human users.The client computer 104 may include an application, such as a web browser application, that makes requests to and receives responses from the server system 102. The application may execute a response from the server system 102, such as web page code or other types of document files, to present the response to the one or more users of the client computer 104. In some implementations, the client computer 104 includes an electronic display device (e.g., an LCD or LED screen, a CRT monitor, a head-mounted virtual reality display, a head-mounted mixed-reality display), or is coupled to an electronic display device, that displays content from the rendered response to the one or more users of the client computer 104. The displayed content can include the selected source image 128a);
determining, using a watermark detector model, that a first digital watermark is likely embedded in the first image (FIG 5 502 and associated text; [0073]; As another alternative, the colors of the encoded pixel and its neighboring pixels may be provided as inputs to a machine-learning model that has been trained to classify encoded pixels as either encoding the first or second binary value based on the color of the encoded pixel and its neighboring pixels. In some instances, if the encoded pixel does not meet the matching criteria to be decoded to either the first or second binary value, the decoder module may arbitrarily assign either the first or second binary decoded value to that pixel. );
decoding, a first version of the first image to obtain the first digital watermark ([0074] During stage 712, the decoder module may employ various techniques to determine a decoded value for a given encoded pixel from the encoded source image. In some implementations, the decoder module uses a matching technique that involves determining a match with the color values of a majority of neighboring pixels. If the colors of a majority of the neighboring pixels of an encoding pixel match the color of the encoding pixel within a specified tolerance, then the encoded pixel is decoded to represent the second binary value (e.g., a value that represents a ‘white’ pixel in the original encoding image).), and
validating a first source of the first image based on the first digital watermark ([0005]; This specification describes systems, methods, devices, and other techniques for generating watermark images to supplement source images that are displayed, for example, as third-party content in an electronic document. In other aspects, this specification describes systems, methods, devices, and other techniques for recovering information encoded in an encoded source image that shows a semi-transparent watermark overlaid on a source image. In some implementations, a system uses information recovered from an encoded source image to identify a provider of the source image, other characteristics of the source image, or context about a specific impression of the source image.).
Gharaibeh does not but JEONG teaches, wherein the decoder machine learning model is jointly trained with an encoder machine learning model as part of an end-to-end learning pipeline, wherein the encoder machine learning model is trained to generate a digital watermark that is embedded into an image and the decoder machine learning model is trained to decode an image to obtained a digital watermark embedded in the image (0056] According to an embodiment, the processor 210 may recognize an area including at least one of an eye area, a wrist area, or a fingerprint area in the image. For example, the processor 210 may recognize the area including the at least one of the eye area, the wrist area, or the fingerprint area in the image to be transmitted to another device, based on a machine learning model. In addition to the above areas, any area on which image recognition can be performed may be included. Hereinafter, for convenience of explanation, the case of recognizing a face area will be described.); It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to modify Gharaibeh’s method with teaching of JEONG in order to protect the pieces of personal information included in the image(JEONG [0007])
With regards to claim 3, 11, 17 Gharaibeh further discloses, wherein the watermark detector model is a watermark detector machine learning model and wherein determining, using the watermark detector model, that the first watermark is likely embedded in the first image (FIG 4 412-414 and associated text; [0073]; As another alternative, the colors of the encoded pixel and its neighboring pixels may be provided as inputs to a machine-learning model that has been trained to classify encoded pixels as either encoding the first or second binary value based on the color of the encoded pixel and its neighboring pixels. In some instances, if the encoded pixel does not meet the matching criteria to be decoded to either the first or second binary value, the decoder module may arbitrarily assign either the first or second binary decoded value to that pixel.) comprises: processing different portions of the first image using the watermark detector machine learning model to identify a first portion of the first image where the first digital watermark is likely to be embedded (FIG 7 and associated text;).
With regards to claim 8, 15, 21 Gharaibeh further discloses, wherein the decoder machine learning model is trained using a set of training images that have been distorted or modified using a distortion generation model ([0010] Some implementations of the subject matter described herein can, in certain instances, realize one or more of the following advantages. First, a system may generate a watermark image to augment an arbitrary source image, even if the source image is not known when the watermark image is created. Because the watermark image may be overlaid on the source image for presentation to a user at a client computer remote from the system, the system need not directly obtain or modify the source image before the watermark image is transmitted to a client computer.[0073]; As another alternative, the colors of the encoded pixel and its neighboring pixels may be provided as inputs to a machine-learning model that has been trained to classify encoded pixels as either encoding the first or second binary value based on the color of the encoded pixel and its neighboring pixels. In some instances, if the encoded pixel does not meet the matching criteria to be decoded to either the first or second binary value, the decoder module may arbitrarily assign either the first or second binary decoded value to that pixel.).
Claim(s) 6-7, 14, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gharaibeh et al(US 20180270388 A1) in view of JEONG et al(US 20220130019 A1) and further in view of Stoddard et al(US 20220415683 A1).
With regards to claim 6, Gharibeh in view of JEONG do not but Stoddard teaches, wherein the decoder machine learning model is a convolutional neural network (CNN) with UNet architecture ([0075] The detailed model description in section 4.5.1 and 4.5.2 provide just one surrogate model concept which has demonstrated excellent performance for this task. There are several other model concepts that could also be used as the surrogate model for wafer bow corrective film pattern determination, including alternative UNet/Zernike-CNN designs, generative adversarial networks, or probabilistic encoder-decoder networks.). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to modify Gharaibeh in view of JEONG’s method with teaching of Stoddard in order to effectively remove bow signature and adequately reduce overlay error (Stoddard[0003]).
With regards to claim 7, Gharibeh in view of JEONG and Stoddard teaches, wherein, for an input image, the CNN generates a segmentation masks that identifies classifications for different image segments of the input image (Stoddard [0067] The architecture used for the forward model 142, 144 (predicting wafer shape transformation from film pattern) is a convolutional UNet (FIG. 5, 502). This structure is a specialized case of an encoder-decoder model where the encoder down-samples into the bottleneck and the decoder up-samples to the resulting output array. The encoder part functions similarly to a typical convolutional neural network (CNN) with a series of convolution operations to extract features from the inputs. FIG. 6 details the UNet architecture of the forward model in the surrogate model. It includes symmetric skip connections at each layer which enable low frequency information to pass through from input to output. In the encoder/down-sampling section, in the first three layers the number of features is doubled each layer. In the decoder section, each step up-samples the feature map followed by an up-convolution to reduce the number of feature channels and then concatenates with the skip connection from its sibling layer in the encoder section. Collectively, the UNet architecture has been proven to yield faster training and better performance with smaller datasets than alternative architectures for many tasks including image-to-image translation and image segmentation [22-24].).
Claim 14 and 20 are medium and system claims corresponding to method claims 6 and 7 combined, also rejected accordingly.
Allowable Subject Matter
Claims 4-5, 9, 12-13, 18-19 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101(DP), set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 20180225800 A1, US 10461931 B2, WO-2021045781-A1.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMED WALIULLAH whose telephone number is (571)270-7987. The examiner can normally be reached 8.30 to 430 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Yin-Chen Shaw can be reached on 1-571-272-8878. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMMED WALIULLAH/Primary Examiner, Art Unit 2498
Prosecution Insights