DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/7/26 has been entered. Claims 1-30 were previously and are currently pending.
Response to Arguments
Applicant’s arguments, see remarks, filed 4/7/26, with respect to the prior art rejections of independent claims 1, 16 and 22 have been fully considered and are partially persuasive. With regards to claim 1, and similarly claims 16 and 22, the applicant argues that none of the prior art teach or fairly suggests the limitations of “storing the plurality of vector representations of the reference facial recognition data in an in-RAM database in a Hierarchical Navigable Small World (HNSW) data structure” and “comparing, by the server and using a machine learning module, the facial recognition data to the reference facial recognition data, while the reference facial recognition data is in the in-RAM database”.
Klare (US2016/0132720) discloses storing the plurality of vector representations of the reference facial recognition data in gallery files that are stored in RAM of the search system (paragraph 50), corresponding to “storing the plurality of vector representations of the reference facial recognition data in an in-RAM database”. Then states that a search system compares the query face vectors to the reference face vectors while stored in the gallery files, which are stored in the RAM of the search system (paragraph 51) which corresponds to “comparing, by the server, the facial recognition data to the reference facial recognition data, while the reference facial recognition data is in the in-RAM database”. As previously stated Klare does not disclose that the comparison is specifically performed “using a machine learning module” but that would have been obvious as taught by Wechsler (US2018/0293429).
However, the Examiner does agree that none of the prior art of record teach or fairly suggests the storing is in a Hierarchical Navigable Small World (HNSW) data structure as is now disclosed in claims 1, 16 and 22. However, as discussed in the updated rejection below this would have been obvious in few of the prior art article of “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs” to Malkov et al. (“Malkov”).
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-30 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Independent claim 1, recites:
“storing the plurality of vector representations of the reference facial recognition data in an in-RAM database in a Hierarchical Navigable Small World (HNSW) data structure”; and
“comparing, by the server and using a machine learning module, the facial recognition data to the reference facial recognition data, while the reference facial recognition data is in the in-RAM database”.
Independent claim 16, recites:
“each comparison performed while the plurality of vector representations are stored in an in-RAM database in a Hierarchical Navigable Small World (HNSW) data structure”.
Independent claim 22 recites:
“store the reference facial recognition data and image source information associated with the facial recognition data in an in-RAM database in a Hierarchical Navigable Small World (HNSW) data structure”; and
compare, using a machine learning module, the facial recognition data to the reference facial recognition data, while the reference facial recognition data is in the in- RAM database”.
However, the originally filed specification only mentions the use of an “in-memory (RAM) database” and “Hierarchical Navigable Small World (HNSW)” on page 16. Which states:
“The vector search may require that all reference vectors are store in an in-memory (RAM) database. With compression algorithms like optimized product quantization (OPQ), Hierarchical Navigable Small World (HNSW), the system can search billions of face vectors in under 100 ms.”
The phrase “in-RAM database” is not specifically mentioned. Is this the same thing as thing as “in-memory (RAM)”? Additionally, the specification does not state that the “vector representations of the reference facial recognition data” (claims 1 and 16) or the “reference facial recognition data and image source information associated with the facial recognition data” (claim 22) are stored in a HNSW data structure, within the in-RAM (in-memory (RAM)). A Hierarchical Navigable Small World (HNSW) “data structure” is also not specifically mention anywhere in the specification. Further, the specification discloses that “the vector search may require that all reference vectors are store in an in-memory (RAM)”, however it does not specifically state that they are stored there “while” the comparing/comparison is carried out. In other words, it’s possible that they could be stored there initially and retrieved or transmitted to a different location for the comparison, but their location during comparison is not disclosed in the specification as being in the “in-RAM” or even “in-memory (RAM)”.
Dependent claims 2-15, 17-21 and 23-30 are rejected by the virtue of their dependency upon rejected independent claims 1, 16 and 22.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, defines “storing the reference facial recognition data and image source information associated with the reference facial recognition in a database” and then states “storing the plurality of vector representations of the reference facial recognition data in a an in-RAM database”. So, there appears to be two different databases. Claim 1 later states that “while the reference facial recognition data is in the in-RAM database”, which is considered unclear and therefore indefinite. The “reference facial recognition data” is previously defined as being store in a database, and then the “vector representations of the reference facial recognition data” is defined as being stored in the in-RAM database. The language of claim 1 appears to suggest that just the “vector representations”, and not all the “reference facial recognition data”, is stored in the “in-RAM” database. Therefore reference to what appear to be all the reference facial recognition data as being in the “in-RAM” database seems inconsistent with what is previously defined making the limitation “while the reference facial recognition data is in the in-RAM database” vague and indefinite.
Claims 2-15 are rejected by the virtue of their dependency upon rejected claim 1 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-10, 16-19, 22-26, 29 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over US 2016/0132720 to Klare et al. (“Klare”) in view of US2018/0293429 to Wechsler et al. (“Wechsler”), in view of US 2014/0237610 to Vandervort, in further view of the prior art article of “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs” to Malkov et al. (“Malkov”).
Regarding claim 1, Klare discloses a method comprising:
obtaining, by a server from one or more publicly accessible sites, reference facial recognition data, wherein the reference facial recognition data comprises a plurality of vector representations, each associated with a face or a plurality of faces (Fig. 1, elements 102 and 108; Fig. 2; paragraphs 41-43 and 45-47, where the combination of search (108) and ingestion (102) systems corresponds to the broadest reasonable interpretation of a “server” in which a remote system or process is serving a client/user device through a network, where facial images are obtained via public or private sources (e.g. public URL) and a face recognition algorithm (104) generates reference facial recognition data from the face in the image in the form of feature vector representations for each);
storing the reference facial recognition data and image source information associated with the reference facial recognition data in a database (Fig. 1, element 106; paragraph 42, wherein the reference facial recognition data (i.e. feature vector) and image source information (e.g. URL) are stored together in the gallery files database);
storing the plurality of vector representations of the reference facial recognition data in an in-RAM database (paragraph 50, wherein the gallery files, including reference facial recognition data (i.e. feature vectors) are stored in RAM (i.e. in-RAM) of the search system)
receiving an image from a user device, wherein the image comprises at least one face of a subject (Fig. 1, element 110; Fig. 4; paragraphs 43-44, 48 and 54, wherein a query face image is provided by a user device and received by the search system);
generating facial recognition data comprising a vector representation of the at least one face (Fig. 1, elements 110, 108, 104; Fig. 3; Fig. 5A-5C; paragraphs 48, 49, 67, wherein the query face image is transformed into a feature vector corresponding to the facial recognition data);
comparing, by the server in-RAM database (Fig. 3; paragraphs 50, 51, 69 and 70, wherein the query facial recognition data/vectors are compared to reference facial recognition data/vectors by the search system of the “server” while stored in gallery files that are stored in the RAM of the search system (i.e. in-RAM database));
identifying, based on a comparison of the facial recognition data to the reference facial recognition data, one or more candidates that match the at least one face (Fig. 3; Fig. 4; paragraphs 50, 51, 54, 69 and 70, wherein based on comparison of the vectors, candidates with similar/matching faces are identified);
based on identification of the one or more candidates matching the at least one face, retrieving, from the database, Fig. 1; Fig. 3; Fig. 4; paragraphs 42, 51-54, wherein when a match is determined candidate metadata such as source information (i.e. URL that identifies a source of the image corresponds to the link to an online profile associated with the candidate) is retrieved from the database/gallery); and
sending, to the user device, an image of the one or more candidates and the image source information (Fig. 1; Fig. 3; Fig. 4; paragraphs 42, 51-54, wherein the matched images and metadata such as source information (i.e. URL that identifies a source of the image) are transmitted back to the user’s device for display).
Klare does not disclose expressly that storing the vector representation in a Hierarchical Navigable Small World (HNSW) data structure, comparing facial recognition data (i.e. vectors) is performed using a machine learning module, or retrieving, from a database, based on a predetermined privacy setting of the identified candidate, image source information.
Malkov discloses storing elements in a HNSW data structure for use in comparing a query element with a stored reference element (Abstract and Introduction).
Klare & Malkov are combinable because they are from the same art of information storage, search and retrieval.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique storing elements in a HNSW data structure, as taught by Malkov, into the process comparing query and reference facial recognition data disclosed by Klare.
The suggestion/motivation for doing so would have been to provide significantly increased performance at high recall and in a case of highly clustered data (Malkov, abstract).
Wechsler discloses a process of comparing facial recognition data using a machine learning module to generate the facial recognition data in the form of feature vectors that are used for the comparison (Fig. 4; paragraphs 22-24).
Klare & Wechsler are combinable because they are from the same art of image processing, specifically facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of comparing facial recognition data using a machine learning module to generate the facial recognition data in the form of feature vectors that are used for the comparison, as taught by Wechsler, into the process comparing query and reference facial recognition data disclosed by Klare.
The suggestion/motivation for doing so would have been to provide better recognition performance (Wechsler, paragraph 07).
Additionally, Vandervort discloses a process for retrieving, from a database, based on a predetermined privacy setting of an identified candidate, information associated with the candidate (Figs. 4, 5; paragraphs 10, 11, 48-50, wherein a candidate configures the privacy settings that regulate the information associated with them that can be retrieved).
Klare & Vandervort are combinable because they are from the same art of data retrieval associated with an individual.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of retrieving, from a database, based on a predetermined privacy setting of an identified candidate, information associated with the candidate, as taught by Vandervort, into the process of, based on identification of the one or more candidates matching the at least one face, retrieving, from the database, image source information associated with each of the one or more candidates, wherein the image source information comprises a link to an online webpage associated with the one or more candidates disclosed by Klare.
The suggestion/motivation for doing so would have been to provide improved system/method for managing user privacy across multiple online sites and applications and sharing data smoothly while maintaining security (Vandervort, paragraphs 05, 08, 09).
Therefore, it would have been obvious to combine Klare with Malkov, Wechsler and Vandervort to obtain the invention as specified in claim 1.
Regarding claim 2, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1, wherein the machine learning module comprises AT LEAST ONE OF:
a k-NN algorithm; a neural network (Wechsler, fig. 4 and paragraphs 22-24, wherein the machine learning module used is a convolutional neural network (CNN)); support vector machines (SVMs); logistic regression; naive Bayes; memory-based learning; random forests; bagged trees; decision trees; boosted trees; or boosted stumps.
Regarding claim 3, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 2, wherein the neural network comprises a convolutional neural network (CNN) (Wechsler, fig. 4 and paragraphs 22-24, wherein the machine learning module used is a convolutional neural network (CNN)).
Regarding claim 4, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1, wherein obtaining the reference facial recognition data comprises:
downloading a plurality of images; and generating the reference facial recognition data based on the plurality of downloaded images (Klare, fig. 1, elements 104, 106; fig. 2; paragraphs 42, and 45-47, wherein a plurality of images are downloaded by a web crawler and the reference facial recognition data (i.e. feature vector) is generated therefrom by facial recognition algorithm (104)).
Regarding claim 5, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1, wherein the identifying the one or more candidates that match the at least one face is further based on a scoring algorithm (paragraphs 69 and 70, wherein identifying candidates that match is determined based on a scoring algorithm based on a Euclidean distance similarity score/value).
Regarding claim 6, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 5, wherein the scoring algorithm is based on a distance value (paragraphs 69 and 70, wherein identifying candidates that match is determined based on a Euclidean distance similarity score/value).
Regarding claim 8, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1, wherein the image received from the user device is captured by the user device, a network camera, or imported from a second user device (Klare, paragraphs 44, 48 and 54, wherein the image can be imported from a second user device indicated by its location (i.e. URL)).
Regarding claim 9, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1, wherein the reference facial recognition data comprises one or more facial images downloaded by a web crawler (Klare, fig. 2; paragraphs 41, 42, and 45-47, wherein the reference data is generated from a plurality of facial images downloaded from the internet by a web crawler).
Regarding claim 10, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1, wherein the reference facial recognition data comprises one or more facial images obtained from the Internet, professional websites, law enforcement websites, OR departments of motor vehicles (Klare, fig. 2; paragraphs 41, 42, and 45-47, wherein the reference data is generated from a plurality of facial images obtained from the internet by a web crawler).
Regarding claim 16, Klare discloses a method comprising:
sending, by a user device to a server, an image, wherein the image comprises at least one face of a subject (Fig. 1, elements 108, 110; Fig. 4; paragraphs 43-44, 48 and 54, wherein a query face image is provided by a user device to the search system server (108));
receiving, from the server, image source information associated with each of the one or more candidates, wherein the image source information comprises a link to an online webpage associated with the candidate (Fig. 1; Fig. 3; Fig. 4; paragraphs 42, 51-54, wherein when a match is determined candidate metadata, including source information (i.e. URL that identifies a source of the image corresponds to the link to an online profile associated with the candidate) associated with each matching candidate, is retrieved from the database/gallery by the search system server and sent to the user device (i.e. received by the user device)), wherein the one or more candidates are identified based on Fig. 3; Fig. 4; paragraphs 50, 51, 54, 69 and 70, wherein a vector derived from the query face image is compared to reference facial vectors derived from reference facial image by the search system server, and candidates with similar/matching faces are identified), each associated with a face of a plurality of faces from images obtained from one or more publicly accessible websites (Fig. 1, elements 102, 104, and 108; Fig. 2; paragraphs 41-43 and 45-47, wherein reference facial images are obtained via public or private sources (e.g. public URL) and a face recognition algorithm (104) generates the reference facial vector from the face in the reference image, and then the image, source (i.e. URL) and vector are stored in the gallery file/database), each comparison performed while the plurality of vector representations are stored in an in-RAM database (paragraph 50, wherein the gallery files, including reference facial recognition data (i.e. feature vectors) are stored in RAM (i.e. in-RAM) of the search system. Additionally, see Fig. 3; paragraphs 50, 51, 69 and 70, wherein the query facial recognition data/vectors are compared to reference facial recognition data/vectors by the search system of the “server” while stored in gallery files that are stored in the RAM of the search system (i.e. in-RAM database))
receiving, based on the image source information an image of the one or more candidates, Fig. 1; Fig. 3; Fig. 4; paragraphs 42, 51-54, wherein the matched images are received, each based on the image originally scraped from URL that identifies its source, are transmitted back to the user’s device (i.e. received by the user device) for display).
Klare does not disclose expressly storing the vector representations in a HNSW data structure, that comparing facial recognition data (i.e. vectors) is performed using a machine learning module or receiving, based on the image source information an image of the one or more candidates, based on a predetermined privacy setting of the identified candidate.
Malkov discloses storing elements in a HNSW data structure for use in comparing a query element with a stored reference element (Abstract and Introduction).
Klare & Malkov are combinable because they are from the same art of information storage, search and retrieval.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique storing elements in a HNSW data structure, as taught by Malkov, into the process comparing query and reference facial recognition data disclosed by Klare.
The suggestion/motivation for doing so would have been to provide significantly increased performance at high recall and in a case of highly clustered data (Malkov, abstract).
Wechsler discloses a process of comparing facial recognition data using a machine learning module to generate the facial recognition data in the form of feature vectors that are used for the comparison (Fig. 4; paragraphs 22-24).
Klare & Wechsler are combinable because they are from the same art of image processing, specifically facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of comparing facial recognition data using a machine learning module to generate the facial recognition data in the form of feature vectors that are used for the comparison, as taught by Wechsler, into the process comparing query and reference facial recognition data disclosed by Klare.
The suggestion/motivation for doing so would have been to provide better recognition performance (Wechsler, paragraph 07).
Additionally, Vandervort discloses a process for receiving, based the source of information, other information associated with a candidate, based on a predetermined privacy setting of an identified candidate (Figs. 4, 5; paragraphs 10, 11, 48-50, wherein a candidate configures the privacy settings that regulate the information associated with them that can be retrieved).
Klare & Vandervort are combinable because they are from the same art of data retrieval associated with an individual.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of receiving, based the source of information, other information associated with a candidate, based on a predetermined privacy setting of an identified candidate, as taught by Vandervort, into the process of, receiving, based on the image source information an image of the one or more candidates disclosed by Klare.
The suggestion/motivation for doing so would have been to provide improved system/method for managing user privacy across multiple online sites and applications and sharing data smoothly while maintaining security (Vandervort, paragraphs 05, 08, 09).
Therefore, it would have been obvious to combine Klare with Malkov, Wechsler and Vandervort to obtain the invention as specified in claim 16.
Regarding claim 17, the combination of Klare,, Malkov, Wechsler and Vandervort discloses the method of claim 16, further comprising: preprocessing, prior to sending the image to the server, the image (Wechsler, Fig. 4 and paragraph 22, wherein the input is preprocessed, including normalization of pose and image size, before sending to the machine learn CNN for feature extraction and comparison).
Regarding claim 18, the combination of Klare,, Malkov, Wechsler and Vandervort discloses the method of claim 17, wherein the preprocessing comprises at least one of cropping, resizing, gradation conversion, median filtering, histogram equalization, or size normalized image processing (Wechsler, Fig. 4 and paragraph 22, wherein the input is preprocessed, including normalization of pose and image size, before sending to the machine learn CNN for feature extraction and comparison).
Regarding claim 19, please refer to the rejection of claim 8 above.
Regarding claim 22, Klare discloses a system comprising:
a user device (Fig. 1, element 110, paragraph 44); and
a server (Fig. 1, elements 102 and 108; paragraphs 41-43, where the combination of search (108) and ingestion (102) systems corresponds to the broadest reasonable interpretation of a “server” in which a remote system or process is serving a client/user device through a network),
wherein the user device is configured to:
send, to the server, an image comprising at least one face of a subject (Fig. 1, element 110; Fig. 4; paragraphs 43-44, 48 and 54, wherein a query face image is provided by a user device and received by the search system); and
display an image associated with one or more candidates corresponding to the subject and image source information associated with each of the one or more candidates (Fig. 1; Fig. 3; Fig. 4; paragraphs 42, 51-54, 70, wherein the matched images (i.e. image associated with candidates that match the face of the subject/query image) and metadata such as source information (i.e. URL that identifies a source of the image) are transmitted back to the user’s device for display); and
wherein the server is configured to:
obtain, from one or more publicly accessible sites, reference facial recognition data, wherein the reference facial recognition data comprises a plurality of vector representations, each associated with one face of a plurality of faces (Fig. 1, elements 102 and 108; Fig. 2; paragraphs 41-43 and 45-47, where facial images are obtained via public or private sources (e.g. public URL) by the ingestion system (102) and a face recognition algorithm (104) generates reference facial recognition data from the face in the image in the form of feature vector representations for each);
store the reference facial recognition data and image source information associated with the facial recognition data in an in-RAM database Fig. 1, element 106; paragraphs 42 and 50, wherein the reference facial recognition data (i.e. feature vector) and image source information (e.g. URL) are stored together in the gallery files database and wherein the gallery files, including reference facial recognition data (i.e. feature vectors) are stored in RAM (i.e. in-RAM) of the search system);
generate facial recognition data comprising a vector representation of the at least one face (Fig. 1, elements 110, 108, 104; Fig. 3; Fig. 5A-5C; paragraphs 48, 49, 67, wherein the query face image is transformed into a feature vector, corresponding to the facial recognition data, by the face recognition algorithm (104) of the search system portion of the server);
compare, Fig. 3; paragraphs 50, 51, 69 and 70, wherein the query facial recognition data/vectors are compared to reference facial recognition data/vectors by the search system of the “server” while stored in gallery files that are stored in the RAM of the search system (i.e. in-RAM database));
identify, based on a comparison of the facial recognition data to the reference facial recognition data, one or more candidates that match the at least one face (Fig. 3; Fig. 4; paragraphs 50, 51, 54, 69 and 70, wherein based on comparison of the vectors, candidates with similar/matching faces are identified);
based on identification of the one or more candidates matching the at least one face, retrieve, Fig. 1; Fig. 3; Fig. 4; paragraphs 42, 51-54, wherein when a match is determined candidate metadata such as source information (i.e. URL that identifies a source of the image corresponds to the link to an online profile associated with the candidate) is retrieved from the database/gallery); and
send, to the user device, an image of the one or more candidates and the image source information (Fig. 1; Fig. 3; Fig. 4; paragraphs 42, 51-54, wherein the matched images and metadata such as source information (i.e. URL that identifies a source of the image) are transmitted back to the user’s device for display).
Klare does not disclose expressly storing the reference facial recognition and image source information in a HNSW data structure, that comparing facial recognition data (i.e. vectors) is performed using a machine learning module, or based on identification of the one or more candidates matching the at least one face, retrieve, based on a predetermined privacy setting of the identified candidate, image source information.
Malkov discloses storing elements in a HNSW data structure for use in comparing a query element with a stored reference element (Abstract and Introduction).
Klare & Malkov are combinable because they are from the same art of information storage, search and retrieval.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique storing elements in a HNSW data structure, as taught by Malkov, into the process comparing query and reference facial recognition data disclosed by Klare.
The suggestion/motivation for doing so would have been to provide significantly increased performance at high recall and in a case of highly clustered data (Malkov, abstract).
Wechsler discloses a process of comparing facial recognition data using a machine learning module to generate the facial recognition data in the form of feature vectors that are used for the comparison (Fig. 4; paragraphs 22-24).
Klare & Wechsler are combinable because they are from the same art of image processing, specifically facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of comparing facial recognition data using a machine learning module to generate the facial recognition data in the form of feature vectors that are used for the comparison, as taught by Wechsler, into the process comparing query and reference facial recognition data disclosed by Klare.
The suggestion/motivation for doing so would have been to provide better recognition performance (Wechsler, paragraph 07).
Additionally, Vandervort discloses a process for retrieving, from a database, based on a predetermined privacy setting of an identified candidate, information associated with the candidate (Figs. 4, 5; paragraphs 10, 11, 48-50, wherein a candidate configures the privacy settings that regulate the information associated with them that can be retrieved).
Klare & Vandervort are combinable because they are from the same art of data retrieval associated with an individual.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of retrieving, from a database, based on a predetermined privacy setting of an identified candidate, information associated with the candidate, as taught by Vandervort, into the process of, based on identification of the one or more candidates matching the at least one face, retrieving, from the database, image source information associated with each of the one or more candidates, wherein the image source information comprises a link to an online webpage associated with the one or more candidates disclosed by Klare.
The suggestion/motivation for doing so would have been to provide improved system/method for managing user privacy across multiple online sites and applications and sharing data smoothly while maintaining security (Vandervort, paragraphs 05, 08, 09).
Therefore, it would have been obvious to combine Klare with Malkov, Wechsler and Vandervort to obtain the invention as specified in claim 22.
Regarding claim 23, please refer to the rejection of claims 17 and 18 above.
Regarding claims 24-26, please refer to the rejections of claims 4, 9 and 10, respectively, above.
Regarding claims 29 and 30, please refer to the rejections of claims 5 and 6, respectively, above.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over US 2016/0132720 to Klare et al. (“Klare”) in view of US 2018/0293429 to Wechsler et al. (“Wechsler”) and US 2014/0237610 to Vandervort, in view of the prior art article of “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs” to Malkov et al. (“Malkov”), in further in view of US 2018/0101742 to Burge et al. (“Burge”).
As to claim 7, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1.
The combination of Klare, Malkov, Wechsler and Vandervort does not disclose expressly wherein the wherein the vector representation of the at least one face comprises a 512 point vector or 1024 point vector.
However, Burge discloses comparing by a server device the facial recognition data to reference facial recognition data associated with a plurality of stored facial images of individuals to identify at least one likely candidate matching the captured facial image (Fig. 3, Fig. 5B, elements 524-532, wherein the first binary vector is compared to the plurality of vectors stored in a database to determine a likely match), wherein the facial recognition data comprise a vector representation of the captured facial image of the subject and the reference facial recognition data comprise a vector representation of the stored facial image in the database (Fig. 3; Figs. 5a- 5b, elements 510-522; paragraphs 58, 70-93, wherein the image is transformed into a first binary vector corresponding to facial recognition data that is then compared with the plurality of vector representing stored facial images in the gallery files or databases) and wherein the vector representation of the captured facial image of the subject or the vector representation of the stored facial image in the database comprises a 512 point vector or a 1024 point vector (paragraph 76).
Klare, Malkov, Wechsler, Vandervort & Burge are combinable because they are from the same art of facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the use a vector representation of the captured or stored facial image that comprises a 512 point vector or 1024 point vector, as taught by Burge, into the process of using facial recognition for providing information about a subject disclosed by the combination of Klare, Malkov, Wechsler and Vandervort.
The suggestion/motivation for doing so would have been to provide time efficient comparison (Burge, paragraph 05).
Therefore, it would have been obvious to combine Klare, Malkov, Wechsler and Vandervort with Burge to obtain the invention as specified in claim 7.
Claims 11 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over US 2016/0132720 to Klare et al. (“Klare”) in view of US 2018/0293429 to Wechsler et al. (“Wechsler”) and US 2014/0237610 to Vandervort, in view of the prior art article of “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs” to Malkov et al. (“Malkov”), in further in view of US 2014/0294257 to Tussy.
As to claim 11, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1.
The combination of Klare, Malkov, Wechsler and Vandervort does not disclose expressly wherein the database comprises a plurality of criminal records associated with the reference facial recognition data.
Tussy discloses a process that includes capturing a facial image by a user device, sending the image to a server for comparison with images in a database, wherein the database also comprises criminal records associated with the stored facial images (paragraphs 85, 86, 110 and 117).
Klare, Malkov, Wechsler, Vandervort & Tussy are combinable because they are from the same art of facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of associating a facial image with criminal records in a database, as taught by Tussy, into the process of using facial recognition for providing information about a subject disclosed by the combination of Klare, Malkov, Wechsler and Vandervort.
The suggestion/motivation for doing so would have been to alert a user of a potentially dangerous individual (Tussy, paragraphs 86 and 110).
Therefore, it would have been obvious to combine Klare, Malkov, Wechsler and Vandervort with Tussy to obtain the invention as specified in claim 11.
Claim 27 is rejected for the same reasons and logic disclosed above with regards to claim 11.
Claims 12, 15, 20 and 28 rejected under 35 U.S.C. 103 as being unpatentable over US 2016/0132720 to Klare et al. (“Klare”) in view of US 2018/0293429 to Wechsler et al. (“Wechsler”) and US 2014/0237610 to Vandervort, in view of the prior art article of “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs” to Malkov et al. (“Malkov”), in further view of US 2019/0354750 to Nazemi et al. (“Nazemi”).
As to claim 12, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1.
The combination of Klare, Malkov, Wechsler and Vandervort does not disclose expressly sending a notification to the user device if a first candidate, of the one or more candidates, poses a high risk to the public or is a criminal.
Nazemi discloses a process that includes capturing a facial image by a user device, sending the image to a central server for comparison with images in a database and transmitting back information/notification to the user device if the identified candidate poses a high risk to public (paragraphs 02, 07, 27).
Klare, Malkov, Wechsler, Vandervort & Nazemi are combinable because they are from the same art of facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of transmitting a notification back to a user device if an identified candidate poses a high risk to the public, as taught by Nazemi, into the process of providing information about a subject disclosed by the combination of Klare, Malkov, Wechsler and Vandervort.
The suggestion/motivation for doing so would have been to provide swift automated facial recognition in high traffic queues (Nazemi, paragraph 02).
Therefore, it would have been obvious to combine Klare, Malkov, Wechsler and Vandervort with Nazemi to obtain the invention as specified in claim 12.
Claims 20 and 28 are rejected for the same reasons and logic disclosed above with regards to claim 12 above.
As to claim 15, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1.
The combination of Klare, Malkov, Wechsler and Vandervort does not disclose expressly providing access to the database to a plurality of users.
Nazemi discloses a process that includes providing access to a plurality of officers/user, to a central server and database of reference facial images (fig 1, paragraphs 24-28).
Klare, Malkov, Wechsler, Vandervort & Nazemi are combinable because they are from the same art of facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of providing access to a database to a plurality of users, as taught by Nazemi, into the process of providing information about a subject disclosed by the combination of Klare, Malkov, Wechsler and Vandervort.
The suggestion/motivation for doing so would have been to provide swift automated facial recognition in high traffic queues for determining if an individual should be granted access into the country (Nazemi, paragraph 02).
Therefore, it would have been obvious to combine Klare, Malkov, Wechsler and Vandervort with Nazemi to obtain the invention as specified in claim 15.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over US 2016/0132720 to Klare et al. (“Klare”) in view of US 2018/0293429 to Wechsler et al. (“Wechsler”) and US 2014/0237610 to Vandervort, in view of the prior art article of “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs” to Malkov et al. (“Malkov”), in further view of US 2016/0196467 to Xia.
As to claim 13, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1.
The combination of Klare, Malkov, Wechsler and Vandervort does not disclose expressly the image comprises a three-dimensional facial image of the subject.
Xia discloses a process of 3D facial image recognition (Figs. 1 and 2).
Klare, Malkov, Wechsler, Vandervort & Xia are combinable because they are from the same art of facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of using 3D images of a subjects face for facial identification, as taught by Xia, in to the process for using facial recognition to acquire/provide information about a subject as disclosed by the combination of Klare, Malkov, Wechsler and Vandervort.
The suggestion/motivation for doing so would have been to use a 3D facial image, as opposed to a 2D facial image, that provides the advantage not being seriously affected by illumination robustness, gestures and expressions (Xia, paragraph 05).
Therefore, it would have been obvious to combine Klare, Malkov, Wechsler and Vandervort with Xia to obtain the invention as specified in claim 13.
Claims 14 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over US 2016/0132720 to Klare et al. (“Klare”) in view of US 2018/0293429 to Wechsler et al. (“Wechsler”) and US 2014/0237610 to Vandervort, in view of the prior art article of “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs” to Malkov et al. (“Malkov”), in further in view of US 2012/0054691 to Nurmi.
As to claim 14, the combination of Klare, Malkov, Wechsler and Vandervort discloses the method of claim 1.
The combination of Klare, Malkov, Wechsler and Vandervort does not disclose expressly wherein the image comprises a second face, wherein the method further comprises identifying a relation between the subject and the second subject.
Nurmi discloses a process for performing facial recognition on a facial image that contains facial images of multiple subjects (Nurmi, paragraph 42) and identifying a relation between two or more 15subjects having facial images captured in a single image (Nurmi, Fig. 7, paragraphs 52, 54, 72, 82 and 83).
Klare, Malkov, Wechsler, Vandervort & Nurmi are combinable because they are from the same art of facial image processing.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of performing facial recognition on a facial image that contains facial images of multiple subjects and identifying a relation between two or more subjects in the image, as taught by Nurmi, into the process of using facial recognition for providing information about a subject disclosed by the combination of Klare, Malkov, Wechsler and Vandervort.
The suggestion/motivation for doing so would have been to provide a user-friendly, efficient and reliable manner in which to determine common friend of individual (Nurmi, paragraph 08).
Therefore, it would have been obvious to combine Klare, Malkov, Wechsler and Vandervort with Nurmi to obtain the invention as specified in claim 14.
Claim 21 is rejected for the same reasons and logic disclosed above with regards to claim 14 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON W CARTER whose telephone number is (571)272-7445. The examiner can normally be reached 8am - 5pm (Mon - Fri).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON W CARTER/Primary Examiner, Art Unit 2661
Prosecution Insights