DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s response, filed 11 December 2025, to the last office action has been entered and made of record.
In response to the amendments to the specification and claims, they are acknowledged, supported by the original disclosure, and no new matter is added.
In response to the amendments to the specification, the amended language has overcome the objection to the specification of the previous Office action, and the respective objection has been withdrawn.
In response to the amendments to the claims, specifically addressing the rejections of claims 6, 13, and 19 under 35 U.S.C. § 112 (b) / (pre-AIA ), second paragraph, of the previous Office action, the amended language has overcome the respective rejections, and the respective rejections have been withdrawn.
In response to the amendments to the claims, specifically addressing the double patenting rejections of claims 1, 8, and 14, of the previous Office action, the amended language has distinguished the instant claims from the claims of the conflicting co-pending Application, overcoming the respective rejections, and the respective rejections have been withdrawn.
Amendments to the independent claims 1, 8, and 14 have necessitated a new ground of rejection over the applied prior art. Please see below for the updated interpretations and rejections.
Response to Arguments
Applicant’s arguments with respect to amended independent claims 1, 8, and 14 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 5, 8, 12, 14, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over K. Zeng et al (“Hierarchical Clustering with Hard-batch Triplet Loss for Person Re-identification”), herein K. Zeng, in view of Yu et al. (“Deep metric learning with dynamic margin hard sampling loss for face verification”), herein Yu, and Song et al. (US 2020/0090039), herein Song.
Regarding claim 1, K. Zeng discloses a computer-implemented person re-identification method for a terminal device having a camera, comprising:
obtaining a person re-identification result by processing the obtained person re- identification task using a preset person re-identification model trained by taking a preset ratio-based triplet loss function that limits a ratio of a positive sample feature distance to a negative sample feature distance to less than a preset ratio threshold as a loss function (see K. Zeng sect. 1. Introduction, where person re-identification is performed using a hierarchical clustering with hard batch triplet loss (HCT) method; see K. Zeng Fig. 2, sect. 3. Our Method, sect. 3.3. Loss Function, and sect 3.4. Model Update, where a pre-trained ResNet-50 on ImageNet is trained according to the HCT method using a hard batch triplet loss with PK sampling to reduce the distance between similar samples and increase the distance between different samples, and includes a margin hyperparameter);
wherein the positive sample feature distance is a distance between a reference image feature and a positive sample image feature, and the negative sample feature distance is a distance between the reference image feature and a negative sample image feature (see K. Zeng sect. 3.3. Loss Function and Eq. (2), where the hard batch triplet loss computes the distances between anchor and positive samples and the anchor with negative samples).
While, K. Zeng teaches the use of a margin hyperparameter for calculating the hard batch triplet loss between distances between anchor with positive samples and anchor with negative samples, which reduces the distance between similar samples and increase the distance between different samples (see K. Zeng Fig. 2, sect. 3. Our Method, and sect. 3.3. Loss Function); K. Zeng does not explicitly disclose wherein the ratio threshold is a variable, and the ratio threshold is negatively correlated with the negative sample feature distance.
Yu teaches in a related and pertinent triplet loss based on deep neural network for end to end metric learning for face verification (see Yu Abstract), where a margin is set to limit the distance between positive and negative pairs, and a dynamic margin hard sampling loss (DMHSL) is applied to face verification with a dynamic margin which can adaptively set the margin based on the distribution of positive and negative pairs in the training set to limit the distances between positive and negative pairs (see Yu sect. 3.1 Dynamic margin hard sampling loss), and models trained with the proposed DMHSL can output a smaller intra-class variation and larger inter-class variation, where reducing intra-class variation and expanding inter-class variation can reduce the generalization error of training models (see Yu sect. 3.1 Dynamic margin hard sampling loss), where the dynamic margin controls the threshold of positive pairs and negative pairs (see Yu sect. 1. Introduction).
At the time of filing, one of ordinary skill in the art would have found it obvious to apply the teachings of Yu to the teachings of K. Zeng, such that the triplet loss function used in training the neural network model suggested by K. Zeng, would be further modified with using a dynamic margin based on the distribution of positive and negative pairs in the training set to limit the distances between positive and negative pairs and predictably result in training person re-identification models with reduced intra-class variation and expanded inter-class variation, suggesting that the dynamic margin is negatively correlated with the distance of negative sample pairs.
This modification is rationalized as an application of a known technique to a known method ready for improvement to yield predictable results.
In this instance, K. Zeng discloses a base method for performing person re-identification using a neural network based model trained using a disclosed hierarchical clustering with hard batch triplet loss method with triplet image samples of people, where a hard batch triplet loss is used with PK sampling to reduce the distance between similar samples and increase the distance between different samples, including a margin hyperparameter.
Yu teaches a known technique of applying a dynamic margin hard sampling loss to face verification which uses a dynamic margin which can adaptively set the margin based on the distribution of positive and negative pairs in the training set to limit the distances between positive and negative pairs to control the threshold of positive pairs and negative pairs, and reducing intra-class variation and expanding inter-class variation of trained models, which can reduce the generalization error of training models.
One of ordinary skill in the art would have recognized that by applying Yu’s techniques would allow for the method of K. Zeng to further modify the triplet loss function used in training the neural network model with a dynamic margin based on the distribution of positive and negative pairs in the training set to limit the distances between positive and negative pairs and predictably result in training person re-identification models with reduced intra-class variation and expanded inter-class variation, predictably leading to improved recognition performance for person reidentification.
K. Zeng and Yu do not explicitly disclose a terminal device having a camera; and obtaining, from the terminal device, a person re-identification task including at least a triplet image set generated based on at least an image captured by the camera of the terminal device.
Song teaches in a related and pertinent system and method to use a neural network to generate a unified machine learning model to identify particular data items associated with each of the plurality of object verticals (see Song Abstract), where the system performs learning operations to train the neural network to generate one or more specialized machine learning models (see Song [0036]-[0037]), where digital images captured using mobile devices/smartphones are learned using generated separate sub-network models (see Song [0038]), and that the system can train specialized embedding models to compute similarity features for object retrieval using a triplet loss function, where triplet includes an anchor image, a positive image, and a negative image, where the triplet loss function includes a margin enforced between positive and negative pairs (see Song [0048]-[0052]).
At the time of filing, one of ordinary skill in the art would have found it obvious to apply the teachings of Song to the teachings of K. Zeng, such that images are captured using a mobile devices / smartphones, where cameras of the mobile devices/smartphones are implied, and provided for performing the person re-identification and recognizing learned persons using the trained neural network based model using the hierarchical clustering with hard batch triplet loss method of K. Zeng for performing person re-identification.
This modification is rationalized as an application of a known technique to a known method ready for improvement to yield predictable results.
In this instance, K. Zeng discloses a base method for performing person re-identification using a neural network based model trained using a disclosed hierarchical clustering with hard batch triplet loss method with triplet image samples of people, where a hard batch triplet loss is used with PK sampling to reduce the distance between similar samples and increase the distance between different samples, including a margin hyperparameter.
Song teaches a known technique of training a neural network to generate one or more specialized machine learning models for object recognition and retrieval where digital images captured using mobile devices/smartphones are learned using generated separate sub-network, and that specialized embedding models are trained to compute similarity features for object retrieval using a triplet loss function, where triplet includes an anchor image, a positive image, and a negative image, where the triplet loss function includes a margin enforced between positive and negative pairs. One of ordinary skill in the art would have recognized that by applying Song’s techniques would allow for the method of K. Zeng to use images that are captured using a mobile devices / smartphones, implied to include cameras for capturing images, and provide the images for performing person re-identification and recognizing learned persons using the trained neural network based model using the hierarchical clustering with hard batch triplet loss method of K. Zeng for performing person re-identification, predictably leading to an improved method where user produced images captured using mobile devices / smartphones can be used to perform person re-identification.
Regarding claim 5, please see the above rejection of claim 1. K. Zeng, Yu, and Song disclose the method of claim 1, wherein the ratio-based triplet loss function is:
L
t
r
i
p
l
e
t
_
r
a
t
i
o
=
1
N
t
p
∑
i
=
1
P
∑
a
=
1
A
max
p
=
1
…
A
D
f
i
a
,
f
i
p
-
β
×
min
n
=
1
…
A
j
=
1
…
P
j
≠
i
D
f
i
a
,
f
j
n
+
;
where,
N
t
p
is a number of triples, i and j are serial numbers, fa, fP, and fn are the reference image feature, the positive sample image feature and the negative sample image feature, respectively, D is a function for calculating distance,
β
is the ratio threshold, {*}+ = max{*, 0} , max is a function for calculating maximum value, min is a function for calculating minimum value, P and A are a number of pedestrians in a training batch of the model and a number of images of each person to be re-identified, respectively, and is the ratio-based triplet loss function (see K. Zeng sect. 3.3. Loss Function and Eq. (2), where the hard batch triplet loss is disclosed to compute the max distances between anchor and positive samples and min distances between the anchor with negative samples across K instances from P identities for each mini-batch, and the loss includes a margin hyperparameter; see Yu sect. 3.1 Dynamic margin hard sampling loss, where a dynamic margin hard sampling loss is applied to face verification which uses a dynamic margin which can adaptively set the margin based on the distribution of positive and negative pairs in the training set to limit the distances between positive and negative pairs to control the threshold of positive pairs and negative pairs, and reducing intra-class variation and expanding inter-class variation of trained models, which can reduce the generalization error of training models; see also Song [0050] and Eq. (1), where triplet loss function is disclosed, where distance between a triplet of image feature pairs are computed and includes a margin enforced between the positive and negative pairs).
Regarding claim 8, it recites a non-transitory computer-readable storage medium for storing one or more computer programs performing the method of claim 1. K. Zeng, Yu, and Song teach a non-transitory computer-readable storage medium for storing one or more computer programs performing the method of claim 1 (see Song [0103]-[0111], where the disclosed teachings can be implemented by processors executing one or more computer programs stored on computer memory devices). Please see above for detailed claim analysis.
Please see the above rejection for claim 1, as the rationale to combine the teachings of K. Zeng, Yu, and Song are similar, mutatis mutandis.
Regarding claim 12, see above rejection for claim 8. It is a non-transitory computer-readable storage medium claim reciting similar subject matter as claim 5. Please see above claim 5 for detailed claim analysis as the limitations of claim 12 are similarly rejected.
Regarding claim 14, it recites a device performing the method of claim 1. K. Zeng, Yu, and Song teach a device performing the method of claim 1. Please see above for detailed claim analysis, with the exception to the following further limitations:
a camera, a processor (see Song [0038]), where digital images captured using mobile devices/smartphones are learned using generated separate sub-network models, suggesting that devices/smartphones includes at least a camera to capture the images; see Song [0109]-[0111], where a computer can be embedded in another device, e.g. a mobile telephone, and essential elements of a computer are a processor for performing actions according to instructions and data and memory devices for storing instructions and data);
a memory coupled to the processor (see Song [0109]-[0111], where a computer can be embedded in another device, e.g. a mobile telephone, and essential elements of a computer are a processor for performing actions according to instructions and data and memory devices for storing instructions and data); and
one or more computer programs stored in the memory and executable on the processor (see Song [0103]-[0111], where the disclosed teachings can be implemented by processors executing one or more computer programs stored on computer memory devices).
Please see the above rejection for claim 1, as the rationale to combine the teachings of K. Zeng, Yu, and Song are similar, mutatis mutandis.
Regarding claim 18, see above rejection for claim 14. It is a device claim reciting similar subject matter as claim 5. Please see above claim 5 for detailed claim analysis as the limitations of claim 18 are similarly rejected.
Allowable Subject Matter
Claims 6, 13, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding the subject matter of claims 6, 13, and 19, the prior art of record, alone or in combination fails to fairly teach or suggest the following limitations in combination with the other recited limitations, notably the claim subject matter of claims 5, 12, and 18 which they depend upon:
“wherein the ratio threshold is a variable satisfying an equation of:
β
=
F
D
f
i
a
,
f
j
n
=
β
1
D
f
i
a
,
f
j
n
<
D
1
a
n
k
×
D
f
i
a
,
f
j
n
+
b
D
1
a
n
≤
D
f
i
a
,
f
j
n
≤
β
2
D
f
i
a
,
f
j
n
>
D
2
a
n
D
2
a
n
;
Where,
k
=
β
1
-
β
2
D
1
a
n
-
D
2
a
n
,
b
=
β
2
-
β
1
-
β
2
D
1
a
n
-
D
2
a
n
×
D
2
a
n
,
β
1
is a preset upper limit of ratio threshold,
β
2
is a preset lower limit of ratio threshold,
D
1
a
n
is a preset lower limit of distance,
D
2
a
n
is a preset upper limit of distance, and
F
is a preset functional relationship”.
Previously cited W. Zeng was relied upon to teach a known technique of further modifying a pairwise metric triplet loss to develop a clustering-guided pairwise metric triplet loss for performing person re identification, where a clustering guided correction term is introduced by extracting sample pair distances and constrained through a hard sigmoid function, such that the cluster extracted from the features space effectively declines the influence of outliers in features space and improve model convergence and promote recognition performance (see W. Zeng sect. III. B. CPM-Triplet Loss, Eq. (12), and Fig. 3).
However, W. Zeng fails to further teach, alone or in combination, the disclosed equation for the ratio threshold as the clustering guided correction term is determined based on filtered positive sample pairs from the distance matrix and constrained using the sigmoid function, where the first term (i.e. 0, x<0) of the sigmoid function is not the upper limit of the of the function, and is not negatively correlated with the negative sample feature distance.
Further search and consideration of the prior art, failed to yield a fair teaching, alone or in combination, of the noted combination of claimed subject matter.
Wu et al. (“Attention Deep Model With Multi-Scale Deep Supervision for Person Re-Identification”) is pertinent in teaching a person reidentification deep model which is uses a ranked triplet loss where the objective function for nontrivial negative samples are weighted in order to push the distance between nontrivial negative samples larger than a threshold (see Wu sect. Ill. PROPOSED METHOD).
Qu et al. (“A Multi-Fault Detection Method With Improved Triplet Loss Based on Hard Sample Mining”) is pertinent in teaching a multi-fault detection method based on an improved triplet loss, where enhanced mapping is applied to the triplet loss to further bring the samples in the same class nearer and those in different classes further, and a small scaling factor is chosen to further enlarge the distance of anchor negative sample pairs (see Qu sect. IV. MULTI-FAULT DETECTION MODEL TRAINING).
Yang et al. (“SOFT RANKING THRESHOLD LOSSES FOR IMAGE RETRIEVAL”) is pertinent in teaching an image retrieval method based on ranking-based threshold losses of pairwise distances between samples in a training batch, including soft ranking threshold loss, hard thresholds, and hard and soft ranking margins (see Yang sect. 2. RANKING-BASED THRESHOLD LOSSES).
However, the above noted references fail to further teach, alone or in combination, that the ratio threshold is a variable satisfying an equation of:
β
=
F
D
f
i
a
,
f
j
n
=
β
1
D
f
i
a
,
f
j
n
<
D
1
a
n
k
×
D
f
i
a
,
f
j
n
+
b
D
1
a
n
≤
D
f
i
a
,
f
j
n
≤
β
2
D
f
i
a
,
f
j
n
>
D
2
a
n
D
2
a
n
;
where,
k
=
β
1
-
β
2
D
1
a
n
-
D
2
a
n
,
b
=
β
2
-
β
1
-
β
2
D
1
a
n
-
D
2
a
n
×
D
2
a
n
,
β
1
is a preset upper limit of ratio threshold,
β
2
is a preset lower limit of ratio threshold,
D
1
a
n
is a preset lower limit of distance,
D
2
a
n
is a preset upper limit of distance and
F
is a preset functional relationship.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIMOTHY WING HO CHOI whose telephone number is (571)270-3814. The examiner can normally be reached 9:00 AM to 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VINCENT RUDOLPH can be reached at (571) 272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIMOTHY CHOI/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671