DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (the English translation of CN 110263697) in view of He et al. ("Transreid: Transformer-based object re-identification." In Proceedings of the IEEE/CVF international conference on computer vision, pp. 15013-15022. 2021), and Ge et al. ("Robust contrastive learning using negative samples with diminished semantics." Advances in Neural Information Processing Systems 34 (2021))
Regarding claim 1:
Wang disclose: a person Re-identification (Re-ID) method (¶ [001] “…Pedestrian re-identification method, device and medium based on unsupervised learning”) comprising:
acquiring a data set, wherein pieces of data in the data set are unlabeled person images (¶ [0018] “…According to the migration learning method, the current visual classifier is used to learn the unlabeled target data set to obtain the matching probability and the spatiotemporal information; wherein the target data set is obtained from the camera network for performing pedestrian recognition. a set of pedestrian images”);
Wang does not specifically teach: performing block processing on each of the pieces of data in the data set to obtain blocked data, performing random ordering on each block of each piece of the blocked data to obtain out-of-order data corresponding to each of the pieces of data.
However, in the same field of endeavor, He teaches: performing block processing on each of the pieces of data in the data set to obtain blocked data (abstract, “we first encode an image as a sequence of patches and build a transformer-based strong baseline with a few critical improvements, which achieves competitive results on several ReID benchmarks with CNN-based methods. To further enhance the robust feature learning in the context of transformers, two novel modules are carefully designed. (i) The jigsaw patch module (JPM) is proposed to rearrange the patch embeddings”)
performing random ordering on each block of each piece of the blocked data to obtain out-of-order data corresponding to each of the pieces of data (abstract, “(i) The jigsaw patch module (JPM) is proposed to rearrange the patch embeddings via shift and patch shuffle operations which generates robust features with improved discrimination ability and more diversified coverage.”);
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the unsupervised pedestrian re-identification method of Wang to further include the patch-based processing and patch-shuffle operations taught by He. Wang is directed to improving pedestrian/person re-identification using an unlabeled target pedestrian-image dataset and unsupervised optimization of a pedestrian recognition model. He is similarly directed to re-identification and teaches encoding an image as a sequence of patches and applying a jigsaw patch module with shift and patch-shuffle operations to obtain more robust and discriminative feature representations in a re-identification system. A person of ordinary skill in the art would have been motivated to incorporate He’s patch-based block processing and patch rearrangement into Wang because doing so would have predictably improved Wang’s learned feature representation by increasing robustness to part misalignment, occlusion, and local variation in person images, while using a known technique for its established purpose in the same field of endeavor.
Wang further teaches: a negative sample pedestrian image in an unsupervised pedestrian re-identification training process (¶ [0025] “…wherein the triplet includes a positive sample pedestrian image, a negative sample pedestrian image… the negative sample pedestrian image being a sample pedestrian image ranked after the preset threshold”.
Wang does not expressly teach that the negative is generated from the original image and its shuffled (out of order) counterpart. He as applied above provide the shuffled (out of order) data but does not expressly teach generating negative samples therefrom.
However, in a related field, Ge teaches: generating negative sample from the input image using patches (blocks) (FIG. 1, “1 introduction”, “Inspired by the non-semantic features, we propose two methods to craft negative samples. The first method relies on texture synthesis tools from classic approaches [12, 52]. It generates realistic texture images based on two patches extracted from input images, as shown in Figure 1(d). For each image in ImageNet, we generate its texture version and form a dataset which we call ImageNet-Texture. The second method constructs non-semantic images by tiling randomly sampled patches of different sizes from the input image, as shown in Figure 1(e). Comparing the non-semantic negative samples with two semantic positive samples in Figure 1(b) and Figure 1(c), the dog from the input image is still recognizable in the positive samples but hard to understand from negative samples. Instead, local statistics such as the fur and color of the dog are preserved in the negative samples.”; “2 Negative Samples with Diminished Semantics”, “to generate negative samples for contrastive learning. Texture-based augmentation generates realistic images based on texture synthesis and patch-based augmentation exploits more comprehensive local features by sampling patches from input images.”) and
Wang as modified by He and further in view of Ge, teaches: performing unsupervised learning based on each of the pieces of data in the data set, the corresponding out-of-order data of each of the pieces of data, and the negative sample data of each of the pieces of data to obtain a corresponding identification (ID) network (Wang ¶ [0130] “…the unlabeled target data set is used to further optimize the visual classifier to improve the performance of the fusion model, so that the visual classifier and the fusion model can be mutually promoted and continuously optimized.”; ¶ [0170] “…The current visual classifier is re-executed from the migration learning step; detecting that the current model training optimization times is greater than the preset optimization threshold, obtaining the current visual classifier time and space information, as a pedestrian recognition model based on unsupervised learning”;
He, abstract, “(i) The jigsaw patch module (JPM) is proposed to rearrange the patch embeddings via shift and patch shuffle operations which generates robust features with improved discrimination ability and more diversified coverage.”;
Ge, “1 introduction”, “Inspired by the non-semantic features, we propose two methods to craft negative samples…. The second method constructs non-semantic images by tiling randomly sampled patches of different sizes from the input image, as shown in Figure 1(e).”), and
performing person Re-ID based on the corresponding ID network (Wang ¶ [0170] “…The current visual classifier is re-executed from the migration learning step; detecting that the current model training optimization times is greater than the preset optimization threshold, obtaining the current visual classifier time and space information, as a pedestrian recognition model based on unsupervised learning”; Wang ¶ [0158] “…and the optimized visual classifier in the unsupervised learning-based pedestrian re-recognition model combines spatio-temporal information, and the target image is compared with the comparison image, a similarity score is calculated, and the pedestrian in the target image is identified in the comparison image, and then the recognition result is output.”).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Wang in view of He to incorporate the teachings of Ge. Wang, as modified by He, would have provided an unsupervised person re-identification framework using unlabeled pedestrian images together with patch-based blocked and shuffled image representations. Ge teaches generating negative samples from input images using patch-based augmentations in unsupervised contrastive learning in order to improve the quality and discrimination of the learned representation. A person of ordinary skill in the art would have been motivated to apply Ge’s negative-sample generation technique to Wang in view He because Ge provides a known method for constructing informative patch-derived negatives from the same image data already being processed in the combined system, thereby predictably improving contrastive separation and robustness of the learned re-identification features. Using Ge’s patch-derived negative generation together with He’s shuffled patch representation in Wang’s unlabeled person re-identification framework would have been a straightforward substitution of one known training enhancement for another to achieve the expected benefit of improved feature discrimination and model robustness.
Regarding claims 19 and 20: the claims limitations are similar to those of claim 1, therefore, rejected in the same manner as applied above.
Allowable Subject Matter
Claim 2-17 and 21 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WASSIM MAHROUKA whose telephone number is (571)272-2945. The examiner can normally be reached Monday-Thursday 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WASSIM MAHROUKA/Primary Examiner, Art Unit 2665