Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 3 is objected to because of the following informalities: ‘enerating’ should read ‘generating’. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness
Claims 1-3, 7, and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Levine (“(De)Randomized Smoothing for Certifiable Defense against Patch Attacks”) in view of Liu (“Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection”).
Regarding claim 1, Levine teaches a non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process ("We used NVIDIA 2080 Ti GPUs for our experiments" [page 18]) comprising: (“"Block smoothing:...Note that for an m×m adversarial patch, out of the h × w possible selections for blocks to use for classification, (m + s − 1)^2 of them will intersect the patch...Band smoothing:... For an m×m adversarial patch, out of the w possible selections for bands to use for classification, m + s − 1 of them will intersect the patch" [page 4] where the above equations successfully fulfill Applicant's proposed example of a 'missing rate' by indicating enough missingness to reduce the impact of the adversarial patch, as "this implies a substantially decreased probability of intersecting the adversarial patch" [page 4, see data provided in Fig. 2]), based on a second value representing a minimum size of the adversarial patch acquired ("we introduce a certifiable defense against patch attacks that guarantees for a given image and patch attack size, no patch adversarial examples exist" [page 1], "In this paper, we consider all attacks (image-specific or universal) on square patches of size m × m" [page 2]) generating a second image in which missingness exceeding the missing rate is added to the first image ("For both of these methods, it is tractable to use the base classifier to classify all possible ablated versions of an image" [page 4], see Fig. 4 for a supplied example of the generated versions with added missingness) comparing a first detection result obtained by inputting the first image into an object detection model with a second detection result obtained by inputting the second image into the object detection model ("The final smoothed classification is simply the plurality class returned " [page 5], where 'plurality class' is the most commonly detected class of all the outputs from the 'base classifier' and is thus a comparison module between raw images and partially ablated images, as "A large number of noisy images are then classified by a base classifier and then the consensus of these classifications is reported as the final classification result" [page 2]. Note that per Applicant’s disclosure, an acceptable example of ‘object detection’ is one which yields a ‘class output’, as is being done in the embodiment described above). Levine does not teach acquiring a first value representing a size of a region of an object included in a first image.
However, Liu teaches acquiring a first value representing a size of a region of an object included in a first image ("suppose that M is known to be an s x s patch" [page 4], [Fig. 2], where 'object' is e.g. the umpire and the accompanying patch M is of size s x s, or [Fig. 4], where 'object' is e.g. the bathtub and the accompanying patch M is of size s x s, also note "We present the APRICOT-Mask dataset 1, which provides segmentation masks and more accurate bounding boxes for adversarial patches in the APRICOT dataset" [page 8]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Levine to include the patch size detection capabilities and to set a region size based on the detected patch size from Liu so as to increase the functionality of Levine’s proposed patch detection method in cases where patch size is unknown, as well as to improve the accuracy of object detection by basing the size of missingness added on the detected size of the patch. Further, one would use that value to determine predicted patch sizes so as to more accurately apply missingness to regions of interest, resulting in more accurate patch detection and less likelihood for error.
Regarding claim 2, Levine in view of Liu teaches the non-transitory computer-readable recording medium of claim 1. Additionally, Levine teaches the generating includes generating the second image in which the missingness is added inside the region in the first image (“by removing (ablating) some of the pixels" [page 2] and "it is tractable to use the base classifier to classify all possible ablated versions of an image" [page 4], Fig. 4).
Regarding claim 3, Levine in view of Liu teaches the non-transitory computer-readable recording medium of claim 1. Additionally, Levine teaches the generating includes generating the second image in which a predetermined shape is added as the missingness (“sampling a single block or band” [page 5]) with a predetermined spacing ("we propose two structured ablation methods...Block Smoothing: In this method, we select a single s × s square block of pixels, and ablate the rest of the image...Band Smoothing: In this method, we select a single band (a column or a row) of pixels of width s, and ablate the rest of the image [page 4]), a predetermined orientation (note that the ‘blocks’ are always squares oriented with parallel edges to image edges, and the ‘bands’ are always either vertically or horizontally oriented, see e.g. Fig. 4), and a predetermined offset (as per Applicant’s disclosure at [0051], the offset can be equal to the spacing, and in this case because the ‘missingness’ is added at every possible location, the spacing and the offset will always be equal from frame to frame), and at a position according to the spacing (“for both of these methods, it is tractable to use the base classifier to classify all possible ablated versions of an image (i.e. hw and w possible ablations for block and column smoothing, respectively)" [page 4]. For the experiment, each image was tested using both methods (see comparative data in Fig. 5), which had a predetermined shape (either a block or a column), with predetermined spacing, orientation, and offset (each possible s x s block or single-band column or row).
Regarding claim 7, this claim is the method claim corresponding to the non-transitory computer-readable recording medium claim of claim 1 and is rejected accordingly. Note that a comparable method is proposed throughout Levine and summarized in the abstract at page 1. See above for relevant references.
Regarding claim 8, this claim is the device claim corresponding to the non-transitory computer-readable recording medium claim of claim 1 and is rejected accordingly. Note that a comparable device is proposed in Levine at page 18, as the proposed GPU can be configured to perform the tasks proposed by Applicant’s device. See above for relevant references.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Levine in view of Liu, further in view of Levine (“Robustness Certificates for Sparse Adversarial Attacks by Randomized Ablation”), henceforth referred to as Levine2.
Regarding claim 4, Levine in view of Liu teaches the non-transitory computer-readable recording medium according to claim 3. Levine further teaches setting the predetermined spacing, the predetermined orientation, the predetermined offset, and the position, within a predetermined range (“we now use that the adversarial patch will overlap with only (m + s − 1) columns [page 6], derived from Theorem 1, which states that the only case in which the adversarial patch will effect the image output if “both x is in the range between i − s + 1 and i + m − 1, inclusive, and y is in the range between j − s + 1 and j +m−1, inclusive” [page 11]. As such, the ‘predetermined range’ in this case defines exactly where, how many, and in which positions the ‘missingness’ needs to be added such that the adversarial patch will be undetected by the object detection algorithm. However, they both fall silent regarding randomly setting the aforementioned parameters.
However, Levine2 teaches randomly setting parameters applicable to those disclosed by Applicant ("we propose a novel smoothing method based on performing random ablations on the input image...In our proposed L0 smoothing method, for each sample generated from x, a majority of pixels are randomly dropped from the image before the image is given to the base classifier. If a relatively small number ρ of pixels have been adversarially corrupted (which is the case in sparse adversarial attacks), then it is highly likely that none of these pixels are present in a given ablated sample" [page 2]). Note that the random ablations of pixels taught in Levine2 are completely random, which apply to the orientation, offset, and position. They are ‘predetermined’ by the algorithm which executes the randomized samples on the images.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Levine in view of Liu further to perform random additions of missingness as described in Levine2 rather than each possible configuration as taught by Levine, since it would save processing time and cut costs, as exhausting all potential combinations of patch location requires more resources.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Levine in view of Liu, further in view of Hofman (“X-Detect: Explainable Adversarial Patch Detection for Object Detectors in Retail”).
Regarding claim 5, Levine in view of Liu teaches the non-transitory computer-readable recording medium according to claim 1, wherein the generating includes generating the second image with missingness added, but both fall silent regarding acquiring an edge of the object in the first image and the missingness is added outside a predetermined range from the edge of the first image.
However, Hofman teaches acquiring an edge of the object in the first image (“First, the OED uses an object extraction model to eliminate the background noise from the main object in s. As opposed to OD models, object extraction models use segmentation techniques that focus on the object’s shape rather than on other properties” [page 4], where under BRI, detecting the shape of the object and extracting it from its background necessitates that object edges were detected) and the missingness is added outside a predetermined range from the edge of the first image (“by using object extraction, the OED changes the assumed object surrounding by eliminating the scene’s background” [page 4]). Note that ‘OED’ refers to ‘Object Extraction Detector’.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Levine in view of Liu further to perform edge detection on objects in the images being analyzed before adding missingness as taught by Hofman, as preserving the objects as wholly as possible when attempting to detect adversarial patches improves the likelihood that the object will be detected correctly at the object detection step of the disclosed process.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Levine in view of Liu, further in view of Xiang (“DetectorGuard: Provably Securing Object Detectors against Localized Patch Hiding Attacks”).
Regarding claim 6, Levine in view of Liu teaches the non-transitory computer-readable recording medium according to claim 1, but both fall silent regarding issuing an alert when comparison between the first detection result and the second detection result indicates a mismatch.
However, Xiang teaches issuing an alert when comparison between the first detection result and the second detection result indicates a mismatch (“We consider our defense to be robust on an object if we can 1) detect the object on the clean image is correct and 2) detect part of the object or send out an attack alert on the adversarial image" [page 3]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include an alert system as described in Xiang so that operators would be aware of potential adversarial attacks to their property/products.
Grace Period Prior Art
Examiner notes that Hofman (X-Detect: Explainable Adversarial Patch Detection for Object Detectors in Retail) names at least one person not named as an Inventor on the incident disclosure and thus, is eligible prior art to the disclosure (see claim 5).
Regarding MPEP 2153.01:
Specifically, Office personnel may not apply a disclosure as prior art under AIA 35 U.S.C. 102(a)(1) if the disclosure: (1) was made one year or less before the effective filing date of the claimed invention; (2) names the inventor or a joint inventor as an author or an inventor; and (3) does not name additional persons as authors on a printed publication or joint inventors on a patent.
Note that regarding (3), at least one additional person is noted as an author on Hofman, and thus, the work fails to qualify as an exception to prior art using the grace period discussed in the MPEP.
Additional References
Prior art made of record and not relied upon that is considered pertinent to applicant’s disclosure:
Additionally cited references (see attached PTO-892) otherwise not relied upon
above have been made of record in view of the manner in which they evidence the general state of the art. Hofman (US20250285408A1) and Hofman (US12633088B2) contain at least one of named Inventors, and while not identical such that concern for Double Patenting is raised, provide context for relevant state of the art and have been noted for record. All other cited references may similarly/alternatively serve to anticipate at least the independent claims as recited, and/or provide examples of current art to Applicant as discovered and noted by Examiner during search.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA T RAMETTA whose telephone number is (571)272-0451. The examiner can normally be reached Monday- Friday, 8 a.m. 5 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JULIA T RAMETTA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights