DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restriction
Restriction to one of the following inventions is required under 35 U.S.C. 121:
I. Claims 1-4, 12, drawn to a system and method for two-stage ML training for identifying footwear outsoles with generating embeddings in a dataset, classified in CPC [G06V 10/82, G06V 10/764]
II. Claims 5-11, 13, drawn to a system and method which identify footwear outer soles by querying image with a search index, and using it to find nearest neighbor candidate footwear, classified in CPC [G06F 16/532, G06F 16/51].
The inventions are independent or distinct, each from the other because:
Inventions I and II are directed to related distinct product or distinct process. The related inventions are distinct if: (1) the inventions as claimed are either not capable of use together or can have a materially different design, mode of operation, function, or effect; (2) the inventions do not overlap in scope, i.e., are mutually exclusive; and (3) the inventions as claimed are not obvious variants. See MPEP § 806.05(j). In the instant case, the inventions claimed in Group II can be used to practice a method that is different from claimed invention in Group I. Furthermore, the inventions as claimed do not encompass overlapping subject matter and there is nothing of record to show them to be obvious variants.
Restriction for examination purposes as indicated is proper because all the inventions listed in this action are independent or distinct for the reasons given above and there would be a serious search and/or examination burden if restriction were not required because one or more of the following reasons apply:
Group I requires a search at least CPC G06V 10/82, G06V 10/764 along with a unique text search. Group II would not be searched as above and would instead require a search in at least CPC G06F 16/51, G06F 16/532 along with a unique text search.
Applicant is advised that the reply to this requirement to be complete must include (i) an election of an invention to be examined even though the requirement may be traversed (37 CFR 1.143) and (ii) identification of the claims encompassing the elected invention.
The election of an invention may be made with or without traverse. To reserve a right to petition, the election must be made with traverse. If the reply does not distinctly and specifically point out supposed errors in the restriction requirement, the election shall be treated as an election without traverse. Traversal must be presented at the time of election in order to be considered timely. Failure to timely traverse the requirement will result in the loss of right to petition under 37 CFR 1.144. If claims are added after the election, applicant must indicate which of these claims are readable upon the elected invention.
Should applicant traverse on the ground that the inventions are not patentably distinct, applicant should submit evidence or identify such evidence now of record showing the inventions to be obvious variants or clearly admit on the record that this is the case. In either instance, if the examiner finds one of the inventions unpatentable over the prior art, the evidence or admission may be used in a rejection under 35 U.S.C. 103 or pre-AIA 35 U.S.C. 103(a) of the other invention.
A telephone call and voice mail was made to Mr. Andrew Zidel (Reg No. 45256) on 09/25/26 to request an election to the above restriction requirement. Mr. Zidel called back on 09/25/2026 and 09/28/2026 respectively, and elected invention II without traverse. Mr. Zidel emailed the examiner with proposed claim amendments on 09/28/2026.
Applicant is reminded that upon the cancelation of claims to a non-elected invention, the inventorship must be corrected in compliance with 37 CFR 1.48(a) if one or more of the currently named inventors is no longer an inventor of at least one claim remaining in the application. A request to correct inventorship under 37 CFR 1.48(a) must be accompanied by an application data sheet in accordance with 37 CFR 1.76 that identifies each inventor by his or her legal name and by the processing fee required under 37 CFR 1.17(i).
The examiner has required restriction between product or apparatus claims and process claims. Where applicant elects claims directed to the product/apparatus, and all product/apparatus claims are subsequently found allowable, withdrawn process claims that include all the limitations of the allowable product/apparatus claims should be considered for rejoinder. All claims directed to a nonelected process invention must include all the limitations of an allowable product/apparatus claim for that process invention to be rejoined.
In the event of rejoinder, the requirement for restriction between the product/apparatus claims and the rejoined process claims will be withdrawn, and the rejoined process claims will be fully examined for patentability in accordance with 37 CFR 1.104. Thus, to be allowable, the rejoined claims must meet all criteria for patentability including the requirements of 35 U.S.C. 101, 102, 103 and 112. Until all claims to the elected product/apparatus are found allowable, an otherwise proper restriction requirement between product/apparatus claims and process claims may be maintained. Withdrawn process claims that are not commensurate in scope with an allowable product/apparatus claim will not be rejoined. See MPEP § 821.04. Additionally, in order for rejoinder to occur, applicant is advised that the process claims should be amended during prosecution to require the limitations of the product/apparatus claims. Failure to do so may result in no rejoinder. Further, note that the prohibition against double patenting rejections of 35 U.S.C. 121 does not apply where the restriction requirement is withdrawn by the examiner before the patent issues. See MPEP § 804.01.
EXAMINER'S AMENDMENT
An examiner’s amendment to the record appears below. Should the changes and/or additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the payment of the issue fee.
Authorization for this examiner’s amendment was given in a phone conversation with Mr. Andrew Zidel (Reg No. 45256) and an email from him both on 09/28/2026.
The application has been amended as follows:
IN THE CLAIMS
1. (withdrawn) A system for training a machine learning model for identification of footwear outsoles, the system comprising:
a first training unit operable to train a descriptor classification model to identify a presence of one or more descriptors within a footwear outsole image;
a separation unit operable to separate a descriptor extractor portion of the descriptor classification model;
a second training unit operable to add a neural network on top of the descriptor extractor portion, and train these layers to identify a footwear outsole to generate a trained identification model;
an embeddings generation unit operable to use the trained identification model to generate embeddings for entries in a footwear reference database; and
a tree generation unit operable to construct a search index using the generated embeddings.
2. (withdrawn) The system of claim 1, wherein the descriptor classification model is a neural network.
3. (withdrawn) The system of claim 1, wherein the second training unit is operable to add an L2-normalised embeddings output layer.
4. (withdrawn) The system of claim 1, wherein the second training unit is operable to use a triplet loss function when performing the training.
5. (currently amended) A system for identifying footwear outsoles, the system comprising:
an image obtaining unit operable to obtain a query image, the image comprising at least a portion of a footwear outsole;
an inputting unit operable to input the query image to a trained machine learning model trained using a system for training the machine learning model for identification of footwear outsoles , the training system comprising:
a first training unit operable to train a descriptor classification model to identify a presence of one or more descriptors within a footwear outsole image;
a separation unit operable to separate a descriptor extractor portion of the descriptor classification model;
a second training unit operable to add a neural network on top of the descriptor extractor portion, and train these layers to identify a footwear outsole to generate a trained identification model;
an embeddings generation unit operable to use the trained identification model to generate embeddings for entries in a footwear reference database; and
a tree generation unit operable to construct a search index using the generated embeddings;
a candidate obtaining unit operable to obtain one or more candidate identifications of the footwear using the trained machine learning model, the candidates being entries in the footwear reference database; and
a candidate selection unit operable to select one of the candidate identifications as the identity of the footwear outsole of the query image.
6. (previously presented) The system of claim 5, further comprising an image modification unit operable to apply one or more modifications to the query image before it is used.
7. (previously presented) The system of claim 6, wherein the modifications include one or more of reorientation, recolouring, contrast adjustment, cropping, and padding.
8. (previously presented) The system of claim 5, wherein the query image comprises at least one of an inked impression of an outsole, a lifted impression of an outsole, an image of a three-dimensional impression of an outsole, or a scan of an outsole.
9. (previously presented) The system of claim 5, wherein the trained machine learning model is trained upon a data set comprising a footwear database and one or more augmented versions of entries in the footwear database, the augmented versions comprising images which have been cropped, distorted, and/or otherwise modified to obscure features of the original entry in the footwear database.
10. (previously presented) The system of claim 5, wherein the trained machine learning model comprises a k-dimensional search tree of embeddings for images in the data set.
11. (previously presented) The system of claim 10, wherein the candidate obtaining unit is operable to obtain n nearest neighbours of embeddings associated with the query image, where n is a positive integer.
12. (withdrawn) A method for training a machine learning model and generating a search tree for identification of footwear outsoles, the method comprising:
training a descriptor classification model to identify a presence of one or more descriptors within a footwear outsole image;
separating a descriptor extractor portion of the descriptor classification model;
adding a neural network on top of the descriptor extractor portion;
training these layers to identify a footwear outsole to generate a trained identification model;
using the trained identification model to generate embeddings for entries in a footwear reference database; and
constructing a search index using the generated embeddings.
13. (previously presented) A method for identifying footwear outsoles, the method comprising:
obtaining a query image, the image comprising at least a portion of a footwear outsole;
inputting the query image to a machine learning model trained using a descriptor classification model to identify a presence of one or more descriptors within a footwear outsole image;
obtaining one or more candidate identifications of the footwear using the trained machine learning model, the candidates being entries in footwear reference database; and
selecting one of the candidate identifications as the identity of the footwear outsole of the query image.
14-15. (cancelled)
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/27/2025 has/have been considered by the examiner.
Claim Objections
Claims 5 and 13 are objected to because of the following informalities:
In claims 5 and 13, “… one or more candidate identifications of the footwear using …” should read -- one or more candidate identifications of the footwear outsole using --.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 5, the phrase "train these layers" renders the claim indefinite because it is unclear whether these layers are in the neural network or in the descriptor extractor, and what these layers are. See MPEP § 2173.05(d). For examination purpose, the phrase has been interpreted as train the neural network.
Claim 9 recites the limitation "the original entry" in line 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 10 recites the limitation "the data set" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claims 6-8 and 11 are also 11 rejected under 35 U.S.C. 112(b) as being dependent upon a rejected base claim.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 13 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li et al (IEEE Internet of Things Journal, IEEE, Vol. 9, No. 19, pp.18829-18838, 2022), hereinafter Li.
-Regarding claim 13, Li discloses a method for identifying footwear outsoles, the method comprising (Abstract; FIGS. 1-9; Algorithms 1-2): obtaining a query image (FIG. 1, query image), the image comprising at least a portion of a footwear outsole (FIGS. 2, 5, 7-8); inputting the query image to a machine learning model trained using a descriptor classification model to identify a presence of one or more descriptors within a footwear outsole image (FIGS. 1-2, 5; Algorithm 2); obtaining one or more candidate identifications of the footwear using the trained machine learning model, the candidates being entries in footwear reference database (FIG. 1, Feature Set, SPI retrieval algorithm, Similarity measure; Algorithm 2, part2, Step 1, gallery image set
X
G
, hash code
H
G
); and selecting one of the candidate identifications as the identity of the footwear outsole of the query image (FIG. 1, Results, Query; Algorithm 2, part2, steps 2-4).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5-6 and 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al (IEEE Internet of Things Journal, IEEE, Vol. 9, No. 19, pp.18829-18838, 2022), hereinafter Li in view of Grivei et al (IGARSS 2019), hereinafter Grivei.
-Regarding claim 5, Li discloses a system for identifying footwear outsoles, the system comprising (Abstract; FIGS. 1-9; Algorithms 1-2): an image obtaining unit operable to obtain a query image (FIG. 1, query image), the image comprising at least a portion of a footwear outsole (FIGS. 2, 5, 7-8); an inputting unit operable to input the query image to a trained machine learning model trained using a system for training the machine learning model for identification of footwear outsoles, the training system comprising (FIGS. 1-2, 5
PNG
media_image1.png
329
686
media_image1.png
Greyscale
): a first training unit operable to train a descriptor classification model to identify a presence of one or more descriptors within a footwear outsole image (FIG. 5; Algorithm 1; Sec III, subsection B); a separation unit operable to separate a descriptor extractor portion of the descriptor classification model (FIGS. 2, 5; Page 18831, Sec. B.; ResNet50 in teacher module as shown in FIGS. 2, 5 is used as feature extractor; Note: using ResNet50 or other neural networks as feature extractors is a common practice in the fields See also Ma et al (IEEE Access-Special Section on Al-Driven Big Data Processing: Theory, Methodology, and Applications, March 30 2019), Page 1, 2nd Col., 1st paragraph and Kong et al (arXiv:1804.02367v2 1 Oct 2018), Page 7, 2nd Col., 2nd – 3rd paragraphs); a second training unit operable to add a neural network on top of the descriptor extractor portion, and train these layers to identify a footwear outsole to generate a trained identification model (FIG. 5; Sec III, subsection. C; Algorithm 2, part 1; FIGS. 1, 3-4); an embeddings generation unit operable to use the trained identification model to generate embeddings for entries in a footwear reference database (FIGS. 4-5; Algorithm 2, part2, Step 1, gallery image set
X
G
, hash code
H
G
); a candidate obtaining unit operable to obtain one or more candidate identifications of the footwear using the trained machine learning model, the candidates being entries in the footwear reference database (FIG. 1, Feature Set, SPI retrieval algorithm, Similarity measure; Algorithm 2, part2, Step 1, gallery image set
X
G
, hash code
H
G
); and a candidate selection unit operable to select one of the candidate identifications as the identity of the footwear outsole of the query image (FIG. 1, Results, Query; Algorithm 2, part2, steps 2-4).
Li does not disclose a tree generation unit operable to construct a search index using the generated embeddings.
In the same field of endeavor, Grivei teaches a method that combines feature extraction, hashing methods and optimized indexing to convert the images characteristics into hash codes in an effort to speed up the search process (Grivei: Abstract; FIGS. 3-4, Table 1). Grivei further teaches a tree generation unit operable to construct a search index using the generated embeddings (Grivei: FIG. 3
PNG
media_image2.png
379
756
media_image2.png
Greyscale
; Page 5910, Sec. 3., 2nd paragraph, “ … The resulted hashes are used to build a binary index tree which further improves the query times …”).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Li with the teaching of Grivei by constructing a search index using the generated embeddings in order to speed up the search process of query image (Grivei: Abstract).
-Regarding claim 6, Li in view of Grivei teaches the system of claim 5. The combination further teaches comprising an image modification unit operable to apply one or more modifications to the query image before it is used (Li: Page 18835, Sec. B., 1st paragraph, “ … uniformly adjust the input shoeprint image size to 128×256 …”).
-Regarding claim 10, Li in view of Grivei teaches the system of claim 5.
Li does not disclose wherein the trained machine learning model comprises a k-dimensional search tree of embeddings for images in the data set.
In the same field of endeavor, Grivei teaches a method that combines feature extraction, hashing methods and optimized indexing to convert the images characteristics into hash codes in an effort to speed up the search process (Grivei: Abstract; FIGS. 3-4, Table 1). Grivei further teaches wherein the trained machine learning model comprises a k-dimensional search tree of embeddings for images in the data set (Grivei: FIG. 3; Page 5910, Sec. 3., 2nd paragraph, “ … The resulted hashes are used to build a binary index tree which further improves the query times …”; Page 5911, 1st Col., 3rd paragraph).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Li with the teaching of Grivei by constructing a search index using the generated embeddings in order to speed up the search process of query image (Grivei: Abstract).
-Regarding claim 11, Li in view of Grivei teaches the system of claim 5. The combination further teaches wherein the candidate obtaining unit is operable to obtain the n nearest neighbours of embeddings associated with the query image, where n is a positive integer (Li: Algorithm 2, part 2; See also Grivei: Page 5911, 1st Col., 3rd paragraph).
Claim(s) 7-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al (IEEE Internet of Things Journal, IEEE, Vol. 9, No. 19, pp.18829-18838, 2022), hereinafter Li in view of Grivei et al (IGARSS 2019), hereinafter Grivei, and further in view of Chiu et al (CISP-BMEI 2019), hereinafter Chiu.
-Regarding claim 7, Li in view of Grivei teaches the system of claim 6.
Li in view of Grivei does not teach wherein the modifications include one or more of reorientation, recolouring, contrast adjustment, cropping, and padding.
However, Chiu is an analogous art pertinent to the problem to be solved in this application and teaches a shoeprint retrieval system that normalizes the size of shoeprints and is efficient in retrieving full and partial print shoeprints (Chiu: Abstract; FIGS. 1-8). Chiu further teaches wherein the modifications include one or more of reorientation, recolouring, contrast adjustment, cropping, and padding (Chiu: FIGS. 2, 8; Page 3, 2nd Col., 1st paragraph, “ color … transformed … various orientations … align … zeros- padding …”).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify the teaching of Li in view of Grivei with the teaching of Chiu by modifying query image in order to normalizes the size of shoeprints for efficient retrieval.
-Regarding claim 8, Li in view of Grivei teaches the system of claim 5.
Li in view of Grivei does not teach wherein the query image comprises at least one of an inked impression of an outsole, a lifted impression of an outsole, an image of a three-dimensional impression of an outsole, or a scan of an outsole.
However, Chiu is an analogous art pertinent to the problem to be solved in this application and teaches a shoeprint retrieval system that normalizes the size of shoeprints and is efficient in retrieving full and partial print shoeprints (Chiu: Abstract; FIGS. 1-8). Chiu further teaches wherein the query image comprises at least one of an inked impression of an outsole, a lifted impression of an outsole, an image of a three-dimensional impression of an outsole, or a scan of an outsole (Chiu: FIG. 2; Page 2, Sec. III.; Page 4, Sec. V.).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify the teaching of Li in view of Grivei with the teaching of Chiu by using query image that comprises at least one of an inked impression of an outsole, a lifted impression of an outsole, an image of a three-dimensional impression of an outsole, or a scan of an outsole in order to retrieve shoeprint with various kinds of query images.
-Regarding claim 9, Li in view of Grivei teaches the system of claim 5.
Li in view of Grivei does not teach wherein the trained machine learning model is trained upon a data set comprising a footwear database and one or more augmented versions of entries in the footwear database, the augmented versions comprising images which have been cropped, distorted, and/or otherwise modified to obscure the features of the original entry in the footwear database.
However, Chiu is an analogous art pertinent to the problem to be solved in this application and teaches a shoeprint retrieval system that normalizes the size of shoeprints and is efficient in retrieving full and partial print shoeprints (Chiu: Abstract; FIGS. 1-8). Chiu further teaches wherein the trained machine learning model is trained upon a data set comprising a footwear database and one or more augmented versions of entries in the footwear database, the augmented versions comprising images which have been cropped, distorted, and/or otherwise modified to obscure the features of the original entry in the footwear database (Chiu: Page 3, Sec. A.; Sec. B., 1st paragraph, “each full-print shoeprint image in the database or each partial-print shoeprint image in query data is divided into overlapped blocks by a sliding window … After the block dividing operation, each full-print shoeprint image in the database can obtain 2,366 blocks”; FIGS. 2, 5-6).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify the teaching of Li in view of Grivei with the teaching of Chiu by augmenting shoeprint images in the footwear database in order to improve the performance of the training of machine leaning model.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Qin et al (CN-114821681-A), hereinafter Qin teaches a fingerprint augmented method by collecting the normal fingerprint image and the variation fingerprint image of the same fingerprint to construct the training data set, training the circulation countermeasure network according to the training data set and ensuring the generated fingerprint image and the input fingerprint image have the same identity information.
Filgueiras de Araujo et al (US 20200004777 A1), hereinafter Araujo teaches a method using machine-learned image descriptor models for image retrieval applications.
Li et al (US 20240330361 A1), hereinafter Li1 teaches a method for training an image embedding model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAO LIU whose telephone number is (571)272-4539. The examiner can normally be reached Monday-Thursday and Alternate Fridays 8:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIAO LIU/Primary Examiner, Art Unit 2664
