DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
The action is responsive to the Applicant’s Amendment filed on 9/04/2025. Claims 1-20 are pending in the application. Claims 1-6, and 9-20 are amended.
Applicant’s amendments to the Abstract have overcome each and every objection previously set forth in the Non-Final Office Action mailed 6/04/2025.
The 112(b) rejection of claim 1 previously set forth in the Non-Final Office Action mailed 6/04/2025 is hereby withdrawn.
Response to Arguments
Applicant’s arguments with respect to the rejections of claims 1-20 have been fully considered but they are not persuasive. In view of the claim amendments, the rejections are being updated accordingly.
Further, regarding the new limitations recited in claims 1-6, and 9-20, it is submitted that they are properly addressed by the new ground of rejection.
In regards to independent claim 1, Applicant argued that, “the present claims are not under the ‘Mental Processes’ grouping as alleged by the Examiner… the application achieves the technical benefits that solve the technical problem in the prior art”. However, the claimed steps are not integrated into a practical application, such as providing the ranked search results as output to a query.
Applicant also argued that neither Kin nor Ishizaka “discloses or suggests restricting the search range using metadata.” However, Kim teaches restrict a search range of the database based on at least one predetermined criterion in paragraphs [0054]-[0057], citing an example of, “only an object having a particular visible range of colors may correspond to the object of interest… the search distance reduces and image retrieval is faster.” Ishizaka also teaches this limitation in paragraph [0227].
In regards to independent claims 9, 15, and 20, the emphasized limitations that the Applicant argues in claims 9, 15, and 20 are similar to the emphasized limitations of claim 1, which have been addressed above. See the response of claim 1 above for explanation.
Furthermore, it is also submitted that all limitations in pending claims, including those not specifically argued, are properly addressed. The reason is set forth in the rejections. See claim analysis below for detail.
Claim Rejections - 35 USC§ 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The claims recite a database configured to store images captured from a plurality of cameras, and a processor configured to, when a probe image containing a person of interest is inputted, restrict a search range of the database based on at least one predetermined criterion, perform a primary ranking, and re-rank the at least one candidate image based on similarity between the probe image and the at least one candidate image.
The claimed process is similar to a method of mental processes, particularly concepts performed in the human minds (including an observation, evaluation, judgement, opinion), which is one of the groupings of abstract ideas according to Prong One in Step 2A of the 2019 Patent Subject Matter Eligibility Guidance since the steps of restrict a search range, perform a primary ranking, and re-rank the candidate image, which allows processes such as searching information--are directed to a series of thought processes (i.e. mental processes).
Also this judicial exception is not integrated into a practical application because searching is merely indicating allowing processes (e.g. primary ranking and re-ranking processes) to happen, which does not mean the process of re-ranking will actually occur and result in a practical application, such as providing the ranked search results as output to a query.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Additional elements (e.g. sex of person, time, location) are directed to types of information being manipulated. The types of information being manipulated does not impose a meaningful limit on the judicial exception, such that the claims are more than a drafting effort design to monopolize exception, because the claimed steps could be performed in a same manner to achieve the same outcome with other types of information other than the ones being used in the claims.
The additional processes (e.g. calculate differences, train neural network, restrict search range, calculating Euclidean distance and Jaccard distance) are merely directed to intended usages since the processes are not being integrated into a practical application.
Hence, the claims do not include additional elements or the combination of the elements are sufficient to amount to significantly more than the judicial exception and fail to integrate the judicial exception into practical application according to Prong Two in Step 2A of the 2019 Patent Subject Matter Eligibility Guidance because the claimed elements or the combination do not impose any meaningful limits on practicing the abstract idea.
Further, in view of Step 2B of the 2019 Patent Subject Matter Eligibility Guidance, it is determined that the computing elements (such as an image search device) in the claim amount to no more than usage of a generic computing system having generic computing components-- such as a database, a feature extraction unit, and a processor, which fails to provide an inventive concept or significantly more than an abstract idea because the elements do not necessarily improve the functional of a computing system or an improvement to a technical field since generic computing is well known.
Thus, for at least the reasoning above, the pending claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 20210019345 A1) in view of Ishizaka et al. (US 20100026842 A1).
Regarding Claim 1, Kim discloses an image search device ([0043]: Referring to FIG. 2, the image control system 1 includes at least one imaging device 10) comprising:
a database configured to store images captured from a plurality of cameras spaced apart from each other and metadata of the images (Figs. 1-2; [0043]: retrieval database 31. In some embodiments, the image control system 1 may further include a total database 32 to further store images or related information that are not stored in the retrieval database 31); and
a processor ([0044]: For example, the hardware may refer to a computing device capable of processing data, including… a processor) configured to, when a probe image containing a person of interest is inputted, restrict a search range of the database based on at least one predetermined criterion, the at least one predetermined criterion corresponding to the metadata of the images ([0054]-[0057]: A standard for a particular object type may be further applied to the object of interest… The object characteristic may be an intrinsic property of the object such as shape, color and surface. For example, when the standard for the particular object characteristic is further applied, only an object having a particular visible range of colors may correspond to the object of interest… the search distance reduces and image retrieval is faster),
However, Kim does not explicitly teach “perform a primary ranking by extracting at least one candidate image included in the restricted search range, and similar to the probe image based on a distance between a feature vector of the probe image and a feature vector of at least one image, and re-rank the at least one candidate image based on similarity between the probe image and the at least one candidate image”.
On the other hand, in the same field of endeavor, Ishizaka teaches
perform a primary ranking by extracting at least one candidate image included in the restricted search range, and similar to the probe image based on a distance between a feature vector of the probe image and a feature vector of at least one image (Fig. 2; [0217]: For example, this process of face discrimination may be one that extracts feature quantity from each of a registered face image and a normalized face image, which are candidates for comparison… the process of face discrimination may be one that performs discrimination processing using a feature vector as feature quantity; Fig. 6; [0227]: For example, it is possible to define and restrict the maximum value of face data to be stored in the face data section 640 on the basis of the size and position of a face detected in one frame and the predetermined conditions of a face at the high rank of a face score, or the like), and
re-rank the at least one candidate image based on similarity between the probe image and the at least one candidate image (Fig. 2; [0217]: and then performs the face discrimination based on the extracted feature quantity. That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities. When this calculated similarity exceeds a threshold value, it is determined that a face in the normalized face image corresponds to a registered face).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim to incorporate the teachings of Ishizaka to re-rank the candidate image based on similarity between the probe image and the candidate image.
The motivation for doing so would be to determine whether a face detected by the face detector is a registered face stored in the database, as recognized by Ishizaka ([0217] of Ishizaka: The face discrimination unit 222 determines whether a face detected by the face detector 212 is a registered face stored in content management file storage 250).
Regarding Claim 2, the combined teachings of Kim and Ishizaka disclose the image search device of claim 1.
Ishizaka further teaches wherein the metadata comprises
sex of a person contained in an image ([0202]: FIG. 21A illustrates an example of extended face data. The extended face data includes a "sex difference score" indicating a sex difference in a detected face and "angle information" indicating an angle of the detected face in the frame. The face metadata with these pieces of data added thereto is defined as a metadata version of "1.10", and "1.10" is stored in the field of a metadata version 632 of the header section 630),
a feature vector ([0217]: Alternatively, furthermore, the process of face discrimination may be one that performs discrimination processing using a feature vector as feature quantity),
time when the image was captured ([0085]: For example, the property file 400 may manage each of the video or still image content files according to the date and time when it is recorded by the imaging apparatus 100), and
location where the image was captured ([0182]: When a content player device recognizes the metadata version as "1.00", desired data of the face data section 640 is quickly accessed because each data having a fixed length is locate data predetermined location).
Regarding Claim 3, the combined teachings of Kim and Ishizaka disclose the image search device of claim 2.
Ishizaka further teaches wherein the processor pre-calculates differences between feature vectors of a plurality of images and stores the same in the database, before the probe image is inputted ([0100]: For example, as shown in FIG. 5, face images 511 to 514 are stored as the registered face images of the persons 521 to 524 in the thumbnail file 500; [0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities… Alternatively, the process of face discrimination may be one that performs discrimination processing with a weak discriminator that uses a difference between the feature quantities).
Regarding Claim 4, the combined teachings of Kim and Ishizaka disclose the image search device of claim 3.
Kim further teaches wherein, upon receiving a first new image from one of the plurality of cameras, the processor extracts a first feature vector of the first new image and calculates differences between the first feature vector and the feature vectors of the plurality of images stored in the database and stores the same in the database ([0066]: The object of interest detection model extracts a feature from the input image and outputs a feature vector based on the extracted feature. The object of interest detection model may be further configured to score the output feature vector for use to classify the class of the object of interest).
Regarding Claim 5, the combined teachings of Kim and Ishizaka disclose the image search device of claim 2.
Kim further teaches wherein the processor trains a neural network on a plurality of different images, as input data, captured at different locations at different times to extract the sex of the person contained in the image and a feature vector and stores the trained neural network in a memory (Figs. 1-2; [0061]-[0067]: The region detection scheme includes, for example… Neural Network (NN)… For example, the object of interest detection model may be trained to classify a predetermined type of the object of interest… To this end, each training sample may further include type information of the object of interest).
Regarding Claim 6, the combined teachings of Kim and Ishizaka disclose the image search device of claim 2.
Ishizaka further teaches wherein the metadata comprises at least one of the sex ([0202]: FIG. 21A illustrates an example of extended face data. The extended face data includes a "sex difference score" indicating a sex difference in a detected face and "angle information" indicating an angle of the detected face in the frame. The face metadata with these pieces of data added thereto is defined as a metadata version of "1.10", and "1.10" is stored in the field of a metadata version 632 of the header section 630),
time ([0085]: For example, the property file 400 may manage each of the video or still image content files according to the date and time when it is recorded by the imaging apparatus 100), and
location information ([0182]: When a content player device recognizes the metadata version as "1.00", desired data of the face data section 640 is quickly accessed because each data having a fixed length is locate data predetermined location),
wherein the processor restricts the search range of images containing the same person as the person contained in the probe image by performing filtering of image data stored in the database based on the sex of the person contained in the probe image ([0227]: For example, it is possible to define and restrict the maximum value of face data to be stored in the face data section 640… face metadata may be restricted on the basis of the size and position of the detected face or the like),
performing filtering based on the time when the image was captured, performing filtering based on the location where the image was captured ([0230]-[0231]: An example shown in FIG. 25 illustrates a case where condition (1) and condition (2) are set up as those that store face data in a content management file 340).
Regarding Claim 7, the combined teachings of Kim and Ishizaka disclose the image search device of claim 6.
Ishizaka further teaches wherein the processor extracts a candidate image group similar to the probe image by calculating Euclidean distances between feature vectors of images included in the restricted search range and the feature vector of the probe image ([0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities… Alternatively, the process of face discrimination may be one that performs discrimination processing with a weak discriminator that uses a difference between the feature quantities… Alternatively… using a feature vector as feature quantity).
Regarding Claim 8, the combined teachings of Kim and Ishizaka disclose the image search device of claim 1.
Ishizaka further teaches wherein the similarity comprises similarity between images based on Jaccard distance ([0216]-[0217]: The face discrimination unit 222 and stores an algorithm which is used when extracting the feature quantity to the face feature quantity 647 of the face data section 640… That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities… Alternatively, the process of face discrimination may be one that performs discrimination processing with a weak discriminator that uses a difference between the feature quantities… Alternatively… using a feature vector as feature quantity).
Regarding Claim 9, Kim discloses an image search device ([0043]: Referring to FIG. 2, the image control system 1 includes at least one imaging device 10) comprising:
a database that stores a plurality of images captured from a plurality of cameras spaced apart from each other and differences in feature information between the plurality of images (Figs. 1-2; [0043]: retrieval database 31. In some embodiments, the image control system 1 may further include a total database 32 to further store images or related information that are not stored in the retrieval database 31);
a feature extraction unit that extracts the sex of a person contained in the image and a feature vector of the image (Fig. 2; [0066]: The object of interest detection model extracts a feature from the input image and outputs a feature vector based on the extracted feature);
wherein the feature extraction unit includes a neural network that is trained on a plurality of different images, as input data, captured at different locations at different times to extract the sex of a person contained in the image and a feature vector (Figs. 1-2; [0061]-[0067]: The region detection scheme includes, for example… Neural Network… Additionally, the object of interest detection model may be further trained to derive more detailed information associated with the object of interest included in the input image. For example, the object of interest detection model may be trained to classify a predetermined type of the object of interest).
However, Kim does not explicitly teach “a processor that, when a probe image containing a person of interest is inputted, perform a primary ranking by extracting at least one candidate image from the plurality of images stored in the database based on a distance between a feature vector of the probe image and respective vectors of the plurality of images ranks selected images based on similarity with the person of interest, and re-rank the at least one candidate image re-ranks the ranked images based on similarity between the probe image and the at least one candidate image, and wherein the processor calculates and stores, in the database, differences between feature vectors of the plurality of images before the probe image is inputted”.
On the other hand, in the same field of endeavor, Ishizaka teaches
a processor ([0057] FIG. 1 is a block diagram illustrating an exemplary configuration of an imaging device 100) that,
when a probe image containing a person of interest is inputted, perform a primary ranking by extracting at least one candidate image from the plurality of images stored in the database based on a distance between a feature vector of the probe image and respective vectors of the plurality of images (Fig. 2; [0217]: For example, this process of face discrimination may be one that extracts feature quantity from each of a registered face image and a normalized face image, which are candidates for comparison… the process of face discrimination may be one that performs discrimination processing using a feature vector as feature quantity; Fig. 6; [0227]: For example, it is possible to define and restrict the maximum value of face data to be stored in the face data section 640 on the basis of the size and position of a face detected in one frame and the predetermined conditions of a face at the high rank of a face score, or the like), and
re-rank the at least one candidate image based on similarity between the probe image and the at least one candidate image (Fig. 2; [0217]: and then performs the face discrimination based on the extracted feature quantity. That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities. When this calculated similarity exceeds a threshold value, it is determined that a face in the normalized face image corresponds to a registered face), and
wherein the processor calculates and stores, in the database, differences between feature vectors of the plurality of images before the probe image is inputted ([0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities… Alternatively, the process of face discrimination may be one that performs discrimination processing with a weak discriminator that uses a difference between the feature quantities… Alternatively… using a feature vector as feature quantity; Figs. 1-2; [0043]: In some embodiments, the image control system 1 may further include a total database 32 to further store images or related information that are not stored in the retrieval database 31; [0203]: FIG. 22 illustrates metadata stored by one storage device).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim to incorporate the teachings of Ishizaka to re-rank the candidate image based on similarity between the probe image and the candidate image.
The motivation for doing so would be to determine whether a face detected by the face detector is a registered face stored in the database, as recognized by Ishizaka ([0217] of Ishizaka: The face discrimination unit 222 determines whether a face detected by the face detector 212 is a registered face stored in content management file storage 250).
Regarding Claim 10, the combined teachings of Kim and Ishizaka disclose the image search device of claim 9.
Ishizaka further teaches wherein, upon receiving the plurality of images captured from the plurality of cameras, the processor stores, in the database,
time when the image was captured ([0085]: For example, the property file 400 may manage each of the video or still image content files according to the date and time when it is recorded by the imaging apparatus 100),
location where the image was captured ([0182]: When a content player device recognizes the metadata version as "1.00", desired data of the face data section 640 is quickly accessed because each data having a fixed length is locate data predetermined location), and the sex ([0202]: FIG. 21A illustrates an example of extended face data. The extended face data includes a "sex difference score" indicating a sex difference in a detected face and "angle information" indicating an angle of the detected face in the frame. The face metadata with these pieces of data added thereto is defined as a metadata version of "1.10", and "1.10" is stored in the field of a metadata version 632 of the header section 630) and
a feature vector of the image which are extracted through the feature extraction unit ([0217]: Alternatively, furthermore, the process of face discrimination may be one that performs discrimination processing using a feature vector as feature quantity).
Regarding Claim 11, the combined teachings of Kim and Ishizaka disclose the image search device of claim 10.
Ishizaka further teaches wherein, upon receiving a first new image from one of the plurality of cameras,
the processor extracts a first feature vector of the first new image through the feature extraction unit ([0022]: when a detected faces does not correspond to any of existing registered faces by face-discrimination processing, the detected face may be registered as a new registered face every time it is detected) and
calculates differences between the first feature vector and the feature vectors of the plurality of images stored in the database and stores the same in the database ([0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities).
Regarding Claim 12, the combined teachings of Kim and Ishizaka disclose the image search device of claim 9.
Ishizaka further teaches wherein the processor restricts a range of comparison targets to be compared with the probe image, among the images included in the database, based on the sex of the person contained in the probe image, time when the probe image was captured, and location where the probe image was captured ([0227]: For example, it is possible to define and restrict the maximum value of face data to be stored in the face data section 640 on the basis of… the predetermined conditions of a face at the high rank of a face score, or the like).
Regarding Claim 13, the combined teachings of Kim and Ishizaka disclose the image search device of claim 11.
Ishizaka further teaches wherein the processor ranks images based on Euclidean distances between a feature vector of the probe image and feature vectors of the images as falling within a comparison target range ([0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities. When this calculated similarity exceeds a threshold value, it is determined that a face in the normalized face image corresponds to a registered face).
Regarding Claim 14, the combined teachings of Kim and Ishizaka disclose the image search device of claim 9.
Kim further teaches wherein the probe image is inputted by a user through an input means of the image search device or selected from the database ([0056]: Alternatively, the object of interest may rely on a user input. In this case, the user input may include a feature component for object identification, or a particular image into which a feature component of a particular object is incorporated).
Regarding Claim 15, Kim discloses an image search method ([0169]: The image control system 1 performs a method for selecting an image of interest) comprising:
storing in a database images captured from a plurality of cameras spaced apart from each other and metadata of the images (Figs. 1-2; [0043]: retrieval database 31. In some embodiments, the image control system 1 may further include a total database 32 to further store images or related information that are not stored in the retrieval database 31); and
when a probe image containing a person of interest is inputted, restricting a search range based on at least one predetermined criterion corresponding to the metadata of the images ([0054]-[0057]: A standard for a particular object type may be further applied to the object of interest… The object characteristic may be an intrinsic property of the object such as shape, color and surface. For example, when the standard for the particular object characteristic is further applied, only an object having a particular visible range of colors may correspond to the object of interest… the search distance reduces and image retrieval is faster);
However, Kim does not explicitly teach “performing a primary ranking by extracting at least one candidate image included in the restricted search range and similar to the probe image based on a distance between a feature vector of the probe image and a feature vector of at least one image; and re-ranking the at least one candidate image based on similarity between the probe image and the at least one candidate image.
On the other hand, in the same field of endeavor, Ishizaka teaches
performing a primary ranking by extracting at least one candidate image included in the restricted search range and similar to the probe image based on a distance between a feature vector of the probe image and a feature vector of the selected at least one image (Fig. 2; [0217]: For example, this process of face discrimination may be one that extracts feature quantity from each of a registered face image and a normalized face image, which are candidates for comparison… the process of face discrimination may be one that performs discrimination processing using a feature vector as feature quantity; Fig. 6; [0227]: For example, it is possible to define and restrict the maximum value of face data to be stored in the face data section 640 on the basis of the size and position of a face detected in one frame and the predetermined conditions of a face at the high rank of a face score, or the like); and
re-ranking the at least one candidate image based on similarity between the probe image and the at least one candidate image (Fig. 2; [0217]: and then performs the face discrimination based on the extracted feature quantity. That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities. When this calculated similarity exceeds a threshold value, it is determined that a face in the normalized face image corresponds to a registered face).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim to incorporate the teachings of Ishizaka to re-rank the candidate image based on similarity between the probe image and the candidate image.
The motivation for doing so would be to determine whether a face detected by the face detector is a registered face stored in the database, as recognized by Ishizaka ([0217] of Ishizaka: The face discrimination unit 222 determines whether a face detected by the face detector 212 is a registered face stored in content management file storage 250).
Regarding Claim 16, the combined teachings of Kim and Ishizaka disclose the image search method of claim 15.
Ishizaka further teaches wherein the metadata includes sex of a person contained in an image ([0202]: FIG. 21A illustrates an example of extended face data. The extended face data includes a "sex difference score" indicating a sex difference in a detected face and "angle information" indicating an angle of the detected face in the frame. The face metadata with these pieces of data added thereto is defined as a metadata version of "1.10", and "1.10" is stored in the field of a metadata version 632 of the header section 630),
a feature vector ([0217]: Alternatively, furthermore, the process of face discrimination may be one that performs discrimination processing using a feature vector as feature quantity),
time when the image was captured ([0085]: For example, the property file 400 may manage each of the video or still image content files according to the date and time when it is recorded by the imaging apparatus 100), and
location where the image was captured ([0182]: When a content player device recognizes the metadata version as "1.00", desired data of the face data section 640 is quickly accessed because each data having a fixed length is locate data predetermined location).
Regarding Claim 17, the combined teachings of Kim and Ishizaka disclose the image search method of claim 16.
Ishizaka further teaches wherein the storing in the database includes calculating differences between feature vectors of a plurality of images and storing the same in the database, before the probe image is inputted ([0100]: For example, as shown in FIG. 5, face images 511 to 514 are stored as the registered face images of the persons 521 to 524 in the thumbnail file 500; [0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities… Alternatively, the process of face discrimination may be one that performs discrimination processing with a weak discriminator that uses a difference between the feature quantities… Alternatively… using a feature vector as feature quantity).
Regarding Claim 18, the combined teachings of Kim and Ishizaka disclose the image search method of claim 17.
Ishizaka further teaches further comprising: upon receiving a first new image from one of the plurality of cameras ([0225]: when a detected faces does not correspond to any of existing registered faces by face-discrimination processing, the detected face may be registered as a new registered face every time it is detected), extracting a first feature vector of the first new image ([0011]: In addition, the imaging apparatus may further include a feature quantity extractor that extracts feature quantity of the detected face may be further provided); and
calculating differences between the first feature vector and the feature vectors of the plurality of images stored in the database ([0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities… Alternatively, the process of face discrimination may be one that performs discrimination processing with a weak discriminator that uses a difference between the feature quantities… Alternatively… using a feature vector as feature quantity) and
storing the same in the database ([0010]: The storage unit is provided for storing a content management file that stores a file entry for managing contents and a particular face entry for managing a particular face which is a face of a particular person).
Regarding Claim 19, the combined teachings of Kim and Ishizaka disclose the image search method of claim 16.
Kim further teaches wherein the storing of metadata of the images includes:
training a neural network on a plurality of different images, as input data, captured at different locations at different times to extract the sex of the person contained in the image and a feature vector; and obtaining the sex of the person in the image and a feature vector by using the trained neural network (Figs. 1-2; [0061]-[0067]: The region detection scheme includes, for example… Neural Network (NN)… For example, the object of interest detection model may be trained to classify a predetermined type of the object of interest… To this end, each training sample may further include type information of the object of interest).
Regarding Claim 20, Kim discloses an image search method ([0169]: The image control system 1 performs a method for selecting an image of interest) comprising:
storing, in a database, a plurality of images captured from a plurality of cameras spaced apart from each other and pre-calculated differences in feature information between the plurality of images (Figs. 1-2; [0043]: retrieval database 31. In some embodiments, the image control system 1 may further include a total database 32 to further store images or related information that are not stored in the retrieval database 31);
extracting sex of a person contained in an image and a feature vector of the image ([0202]: FIG. 21A illustrates an example of extended face data. The extended face data includes a "sex difference score" indicating a sex difference in a detected face… The face metadata with these pieces of data added thereto is defined as a metadata version of "1.10", and "1.10" is stored in the field of a metadata version 632 of the header section 630) by using a pre-trained artificial neural network (Figs. 1-2; [0061]-[0067]: The region detection scheme includes, for example… Neural Network (NN)… For example, the object of interest detection model may be trained to classify a predetermined type of the object of interest… To this end, each training sample may further include type information of the object of interest); and
wherein the artificial neural network is trained on a plurality of different images, as input data, captured at different locations at different times to extract and store sex of a person contained in an image and a feature vector (Figs. 1-2; [0061]-[0067]: For example, the object of interest detection model may be trained to classify a predetermined type of the object of interest… To this end, each training sample may further include type information of the object of interest).
However, Kim does not explicitly teach “when a probe image containing a person of interest is inputted,- performing a primary ranking by extracting at least one candidate image from the plurality of images stored in the database based on a distance between a feature vector of the prove image and respective feature vectors of the plurality of images re-ranking the at least one candidate image based on Jaccard distance”.
On the other hand, in the same field of endeavor, Ishizaka teaches
when a probe image containing a person of interest is inputted, performing a primary ranking by extracting at least one candidate image from the plurality of images stored in the database based on a distance between a feature vector of the prove image and respective feature vectors of the plurality of images (Fig. 2; [0217]: For example, this process of face discrimination may be one that extracts feature quantity from each of a registered face image and a normalized face image, which are candidates for comparison… the process of face discrimination may be one that performs discrimination processing using a feature vector as feature quantity; Fig. 6; [0227]: For example, it is possible to define and restrict the maximum value of face data to be stored in the face data section 640 on the basis of the size and position of a face detected in one frame and the predetermined conditions of a face at the high rank of a face score, or the like) and
re-ranking the at least one candidate image based on Jaccard distance (Figs. 2, 6; [0216]-[0217]: The face discrimination unit 222 and stores an algorithm which is used when extracting the feature quantity to the face feature quantity 647 of the face data section 640… That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities. When this calculated similarity exceeds a threshold value, it is determined that a face in the normalized face image corresponds to a registered face), and
wherein differences between feature vectors of the plurality of images are calculated and stored in the database, before the probe image is inputted ([0100]: For example, as shown in FIG. 5, face images 511 to 514 are stored as the registered face images of the persons 521 to 524 in the thumbnail file 500; [0217]: That is, a comparison between feature quantity extracted from a registered face image and feature quantity extracted from a normalized face image is made to calculate the similarity between these feature quantities).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim to incorporate the teachings of Ishizaka to re-rank the candidate image based on similarity between the probe image and the candidate image.
The motivation for doing so would be to determine whether a face detected by the face detector is a registered face stored in the database, as recognized by Ishizaka ([0217] of Ishizaka: The face discrimination unit 222 determines whether a face detected by the face detector 212 is a registered face stored in content management file storage 250).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIRLEY D. HICKS whose telephone number is (571)272-3304. The examiner can normally be reached Mon - Fri 7:30 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached on (571) 272-4085. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S D H/Examiner, Art Unit 2168
/CHARLES RONES/Supervisory Patent Examiner, Art Unit 2168
Prosecution Insights