DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. TAIWAN-113110698, filed on 06/12/2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/23/2024, 12/16/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Status
Claim(s) 1 and 6-8 and is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ma et al (U.S. 9,633,268 A1; Ma).
Claim(s) 2-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ma et al (U.S. 9,633,268 A1; Ma), in view of Tan et al (U.S. 20170243058 A1; Tan).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 and 6-8 and is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ma et al (U.S. 9,633,268 A1; Ma).
Regarding claim 1, Ma discloses a person identification method for determining an identity of a person, (Col 1 – lines 61-66: “a method and a device for gait recognition, which is used to solve the problem of the resulting inaccurate recognition due to too small inter-class differences between different persons and too large intra-class differences of a same person in different scenes for the gait feature for judging and recognizing”) implemented by a processor (Col 14 – lines 18-21: “the method described above can be implemented by instructing the related hardware with programs which can be stored in a computer readable storage medium such as ROM/RAM, magnetic disk, optical disk or the like.”) and comprising:
obtaining a gait dataset that is related to the person; (Figs.1 and 5 and Col 2 : lines 2-3: “S11: extracting an initial gait feature of a gait video of a person to be recognized …a gait video of the person to be recognized is obtained ”)
obtaining a group determination (a corresponding optimized gait feature) based on a first gait recognition model, a second gait recognition model (two identical sub neural networks) and the gait dataset (the initial gait feature), where the group determination indicates whether the person belongs to a group that includes a plurality of predetermined members; (Figs. 1 and 5; Col 2 – lines 4-33 : “S12: obtaining a corresponding optimized gait feature according to a trained sub neural network and the initial gait feature … With the initial gait features of known persons obtained in step S21, the initial gait features of any two known persons are selected as a sample pair, and these sample pairs are organized in groups. If the two initial gait features in a sample pair belong to the same person, this sample pair is classified into the first sample pair group; if the two initial gait features in a sample pair belong to two different persons, this sample pair is classified into the second sample pair group… S23: training a neural network comprising two identical sub neural networks which are provided in parallel and share weight according to the first sample pair group and the second sample pair group, such that the degree of similarity between two optimized gait features output by the two sub neural networks for any sample pair in the first sample pair group is greater than the degree of similarity between two optimized gait features output by the two sub neural networks for any sample pair in the second sample pair group.”;) and
when the group determination indicates that the person belongs to the group, obtaining an identity determination based on a third gait recognition model (a recognizing module) and the gait dataset (matching library), where the identity determination indicates which one of the predetermined members the person is. (Figs. 1-5 and Col 2- lines 40-47: “with the highest degree of similarity, judging whether the highest degree of similarity is greater than a preset threshold of the degree of similarity, and if so, determining the information of the known person in the matching library corresponding to the optimized gait feature which has the highest degree of similarity with the optimized gait feature of the person to be recognized as the information of the person to be recognized.”; Col 13 – lines 18-25: “recognizing module 53 is specifically used to judge whether the highest degree of similarity is greater than a preset threshold of the degree of similarity, and if so, to determine the information of the known person in the matching library corresponding to the optimized gait feature which has the highest degree of similarity with the optimized gait feature of the person to be recognized as the information of the person to be recognized.”)
Regarding claim 6, Ma discloses the obtaining of the identity determination includes: obtaining a plurality of gait data segments from the gait dataset, where each of the gait data segments includes a plurality of segment values corresponding respectively to a plurality of gait features; (Figs. 1 and 4; Col 10 -lines 4-15: “for an image sequence of the gait video, extracting a foreground silhouette image sequence using a Graph Cut image segmentation method … the method for extracting an initial gait feature of a person to be recognized described above can be used for obtaining initial gait features of gait videos of other persons.”) for each of the gait data segments, obtaining an identity based on the segment values that are included in the gait data segment and the third gait recognition model, where the identity indicates which one of the predetermined members the person is; (Col 13 – lines 2-25 : “ a training module 54 used to obtain an initial gait feature of a gait video of each known person; to construct a sample pair set, which comprises a first sample pair group and a second sample pair group, wherein each sample pair of the first sample pair group includes two initial gait features of a same person and each sample pair of the second sample pair group includes two initial gait features of different persons … recognizing module 53 is specifically used to judge whether the highest degree of similarity is greater than a preset threshold of the degree of similarity, and if so, to determine the information of the known person in the matching library corresponding to the optimized gait feature which has the highest degree of similarity with the optimized gait feature of the person to be recognized as the information of the person to be recognized.” and obtaining the identity determination based on the identities obtained for the gait data segments. (Col 13 – lines 18-25: “recognizing module 53 is specifically used to judge whether the highest degree of similarity is greater than a preset threshold of the degree of similarity, and if so, to determine the information of the known person in the matching library corresponding to the optimized gait feature which has the highest degree of similarity with the optimized gait feature of the person to be recognized as the information of the person to be recognized.”)
Regarding claim 7, Ma discloses the obtaining of the identity includes, for each of the gait data segments: performing feature scaling on the segment values of the gait data segment, so as to obtain a plurality of scaling values corresponding respectively to the segment values; (Figs. 1 and 4; Col 10 -lines 4-15: “for an image sequence of the gait video, extracting a foreground silhouette image sequence using a Graph Cut image segmentation method, normalizing sizes of images in the foreground silhouette image sequence and using the normalized foreground silhouette image sequence as the initial gait feature… the method for extracting an initial gait feature of a person to be recognized described above can be used for obtaining initial gait features of gait videos of other persons.”) and obtaining the identity based on the scaling values and the third gait recognition model. Col 13 – lines 18-25: “recognizing module 53 is specifically used to judge whether the highest degree of similarity is greater than a preset threshold of the degree of similarity, and if so, to determine the information of the known person in the matching library corresponding to the optimized gait feature which has the highest degree of similarity with the optimized gait feature of the person to be recognized as the information of the person to be recognized.”)
Regarding claim 8, Ma discloses each of the first gait recognition model, the second gait recognition model, and the third gait recognition model was trained based on a plurality of training datasets of gaits corresponding respectively to the predetermined members. (Col 7– line 50 to Col 8 – line 3 : “there are 3000 gait videos of known persons, and according to the information of a known person recorded in each gait video, the gait video has been marked with the information of the known person. … therefore, it is necessary to set the preset value and the preset value is set to 1 to ensure the equal number of the two kinds of sample pairs, which can thereby ensure that accurate and reliable sub neural networks can be obtained by training”; Col 6 – lines 62-65: “The training steps of the sub neural networks used in the recognition steps are illustrated in FIG. 1, which is a flowchart of the gait recognition and illustrates the training steps of the sub neural networks used in the recognition steps.”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ma et al (U.S. 9,633,268 A1; Ma), in view of Tan et al (U.S. 20170243058 A1; Tan).
Regarding claim 2, Ma discloses wherein the obtaining of the group determination includes: obtaining a plurality of gait data segments from the gait dataset, (Figs.1 and 5 and Col 2 : lines 2-3: “S11: extracting an initial gait feature of a gait video of a person to be recognized …a gait video of the person to be recognized is obtained ”)
However, Ma does not disclose where each of the gait data segments includes a plurality of segment values corresponding respectively to a plurality of gait features; obtaining a first identification result based on the gait data segments and the first gait recognition model; obtaining a second identification result based on the gait data segments and the second gait recognition model; obtaining a matching percentage between the first identification result and the second identification result; and obtaining the group determination based on the matching percentage.
Tan discloses obtaining a plurality of gait data segments from the gait dataset, where each of the gait data segments includes a plurality of segment values corresponding respectively to a plurality of gait features; (Paragraph 41: “Step S11: extracting a gait energy image sequence GEI-1, . . . GEI-I, . . . GEI-N from a training gait video sequence involving multiple views. First a conventional foreground segmentation method based on a Gaussian mixture model is used to extract silhouettes of a person from the gait video sequence,”
obtaining a first identification result (feature a) based on the gait data segments (GEI-a) and the first gait recognition model (GEI-a); (Paragraphs 42-44: “Silhouettes of persons are extracted from a sequence of said 1100 gait videos to calculate the gait energy image…. Step S13: sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm. As shown in FIG. 1, the feature extracting module of the matching model based on the convolutional neural network is used to extract corresponding features of gait energy images GEI-a, GEI-b as feature a and feature b.”)
obtaining a second identification result (feature b) based on the gait data segments and the second gait recognition model (GEI-b); (Paragraphs 42-44: “Silhouettes of persons are extracted from a sequence of said 1100 gait videos to calculate the gait energy image. …Step S13: sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm. As shown in FIG. 1, the feature extracting module of the matching model based on the convolutional neural network is used to extract corresponding features of gait energy images GEI-a, GEI-b as feature a and feature b.)
obtaining a matching percentage between the first identification result and the second identification result; (Paragraph 45: “Step S14: comparing features of the two gait energy images extracted in S13 and giving a score of similarity by using the perceptron module of the matching model based on the convolutional neural network, and determining if said two images have the same identity.”) and obtaining the group determination based on the matching percentage. (Paragraph 45: “Step S14: comparing features of the two gait energy images extracted in S13 and giving a score of similarity by using the perceptron module of the matching model based on the convolutional neural network, and determining if said two images have the same identity. For example, when the similarity has a value ranging from 0 to 1, it can be set that when the similarity is greater than 0.5, the gait video sequences corresponding to said pair of features can be predicted to have the same identity; otherwise, they are predicted to have different identities.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Ma by including a gait sequence by means of a gait energy image, and trains a matching model through a deep convolutional neural network that is taught by Tan, to make the invention that a gait recognition method based on deep learning; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improve cross-view gait recognition as well as reducing complexity of calculation of deep learning module. (Tan: Paragraphs 50-51)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 3, Ma, as modified by Tan discloses all the claims invention. Tan further discloses the obtaining of the first identification result includes, for each of the gait data segments, obtaining a first identity (feature a) based on the segment values that are included in the gait data segment and the first gait recognition model, where the first identity indicates which one of the predetermined members the person is; (Paragraphs 41-44: “First a conventional foreground segmentation method based on a Gaussian mixture model is used to extract silhouettes of a person from the gait video sequence … Silhouettes of persons are extracted from a sequence of said 1100 gait videos to calculate the gait energy image. …Step S13: sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm. As shown in FIG. 1, the feature extracting module of the matching model based on the convolutional neural network is used to extract corresponding features of gait energy images GEI-a, GEI-b as feature a and feature b.)
the obtaining of the second identification result includes, for each of the gait data segments, obtaining a second identity (feature b) based on the segment values that are included in the gait data segment and the second gait recognition model, where the second identity indicates which one of the predetermined members the person is; (Paragraphs 41-44: “First a conventional foreground segmentation method based on a Gaussian mixture model is used to extract silhouettes of a person from the gait video sequence, … Silhouettes of persons are extracted from a sequence of said 1100 gait videos to calculate the gait energy image. …Step S13: sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm. As shown in FIG. 1, the feature extracting module of the matching model based on the convolutional neural network is used to extract corresponding features of gait energy images GEI-a, GEI-b as feature a and feature b.)
the obtaining of the matching percentage includes, for each of the gait data segments, obtaining one of a positive determination and a negative determination based on the first identity and the second identity, (Paragraphs 43-44: “ selecting positive samples and negative samples. Pairs of gait energy images having the same identity are selected as positive samples, and pairs of gait energy images having different identities are selected as negative samples. The selection of the gait energy images should be a selection from gait energy images of different views based on the same probability. … Step S13: sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm”) where the positive determination is obtained when both the first identity and the second identity indicate a same one of the predetermined members, (Paragraphs 43-45: “ selecting positive samples and negative samples. Pairs of gait energy images having the same identity are selected as positive samples, and pairs of gait energy images having different identities are selected as negative samples. The selection of the gait energy images should be a selection from gait energy images of different views based on the same probability. … Step S14: comparing features of the two gait energy images extracted in S13 and giving a score of similarity by using the perceptron module of the matching model based on the convolutional neural network, and determining if said two images have the same identity.”) and the negative determination is obtained when the first identity and the second identity indicate different ones of the predetermined members, (Paragraphs 43-45: “ selecting positive samples and negative samples. Pairs of gait energy images having the same identity are selected as positive samples, and pairs of gait energy images having different identities are selected as negative samples. The selection of the gait energy images should be a selection from gait energy images of different views based on the same probability. … Step S14: comparing features of the two gait energy images extracted in S13 and giving a score of similarity by using the perceptron module of the matching model based on the convolutional neural network, and determining if said two images have the same identity.”) and obtaining the matching percentage based on the one of the positive determination and the negative determination obtained for each of the gait data segments; (Paragraphs 43-45: “ selecting positive samples and negative samples. Pairs of gait energy images having the same identity are selected as positive samples, and pairs of gait energy images having different identities are selected as negative samples. The selection of the gait energy images should be a selection from gait energy images of different views based on the same probability. … Step S14: comparing features of the two gait energy images extracted in S13 and giving a score of similarity by using the perceptron module of the matching model based on the convolutional neural network, and determining if said two images have the same identity.”) and
the first identities obtained for the gait data segments cooperatively form the first identification result (feature a), and the second identities obtained for the gait data segments cooperatively form the second identification result (feature b)t. (Paragraphs 41-44: “First a conventional foreground segmentation method based on a Gaussian mixture model is used to extract silhouettes of a person from the gait video sequence, … Silhouettes of persons are extracted from a sequence of said 1100 gait videos to calculate the gait energy image. …Step S13: sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm. As shown in FIG. 1, the feature extracting module of the matching model based on the convolutional neural network is used to extract corresponding features of gait energy images GEI-a, GEI-b as feature a and feature b.)
Regarding claim 4, Ma, as modified by Tan discloses all the claims invention. Tan further discloses the obtaining of the first identity includes, for each of the gait data segments: performing feature scaling on the segment values of the gait data segment, so as to obtain a plurality of scaling values corresponding respectively to the segment values; (Paragraph 41: “extracting a gait energy image sequence GEI-1, . . . GEI-I, . . . GEI-N from a training gait video sequence involving multiple views. First a conventional foreground segmentation method based on a Gaussian mixture model is used to extract silhouettes of a person from the gait video sequence, foreground areas are located and cut according to the gravity centers of the silhouettes and are normalized to the same dimension by scaling, then an average silhouette image of each sequence is acquired, which is the gait energy image … Silhouettes of persons are extracted from a sequence of said 1100 gait videos to calculate the gait energy image.”) and obtaining the first identity based on the scaling values and the first gait recognition model. (Paragraph 44: “ sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm. As shown in FIG. 1, the feature extracting module of the matching model based on the convolutional neural network is used to extract corresponding features of gait energy images GEI-a, GEI-b as feature a and feature b.”)
Regarding claim 5, Ma, as modified by Tan discloses all the claims invention. Tan further discloses the obtaining of the second identity includes, for each of the gait data segments: performing feature scaling on the segment values of the gait data segment, so as to obtain a plurality of scaling values corresponding respectively to the segment values; (Paragraphs 41-42: “extracting a gait energy image sequence GEI-1, . . . GEI-I, . . . GEI-N from a training gait video sequence involving multiple views. First a conventional foreground segmentation method based on a Gaussian mixture model is used to extract silhouettes of a person from the gait video sequence, foreground areas are located and cut according to the gravity centers of the silhouettes and are normalized to the same dimension by scaling, then an average silhouette image of each sequence is acquired, which is the gait energy image. … Silhouettes of persons are extracted from a sequence of said 1100 gait videos to calculate the gait energy image.”) and obtaining the second identity based on the scaling values and the second gait recognition model. (Paragraph 44: “ sending each pair of gait energy images forming the positive and negative samples in S12 to the matching model based on the convolutional neural network, and extracting their corresponding features by means of a forward propagation algorithm. As shown in FIG. 1, the feature extracting module of the matching model based on the convolutional neural network is used to extract corresponding features of gait energy images GEI-a, GEI-b as feature a and feature b.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cole et al (U.S. 20150282766 A1), Method and Apparatus to Infer Object and Agent Properties, Activity Capacities, Behaviors and Intents from Contact and Pressure Images”, teaches about An apparatus for determining a non-apparent attribute of an object having a sensor portion with which the object makes contact and to which the object applies pressure. The apparatus has a computer in communication with the sensor portion that receives signals from the sensor portion corresponding to the contact and pressure applied to the sensor portion, and determines from the signals the non-apparent attribute. The apparatus has an output in communication with the computer that identifies the non-apparent attribute determined by the computer.
Tafazzoli et al (U.S. 20170243354 A1), “Automatic Frontal-View Gait Segmentation for Abnormal Gait Quantification”, teaches about a computer-implemented method for gait analysis of a subject includes obtaining visual data from an image capture device positioned in front of or behind the subject, the visual data comprising at least two image frames of the subject over a period of time walking toward or away from the image capture device, the at least two image frames capturing at least a portion of the gait of the subject, detecting within the at least two images body parts as two-dimensional landmarks using a pose estimation algorithm on each of the at least two frames, generating a joint model depicting the location of the at least one joint in each of the at least two frames, using the joint model to segment a gait cycle for the at least one joint, and comparing the gait cycle to a threshold value to detect abnormal gait.
Liu et al (U.S. 20220148335 A1), “Disentangled Representations For Gait Recognition”, teaches about A computer-implemented method is presented for identifying a person includes: receiving a set of images for a given person walking over a period of time; extracting canonical features of the given person from the set of images using a first neural network, where the canonical features describe body shape of the given person; extracting gait features of the given person from the set of images using the first neural network and a second neural network, where the gait features describe gait of the given person; and identifying, by the image processor, the given person by comparing the canonical features of the given person and the gait features of the given person to the plurality of feature sets.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Duy A Tran whose telephone number is (571)272-4887. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL R MISTRY can be reached at (313)-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY TRAN/ Examiner, Art Unit 2674
/ONEAL R MISTRY/ Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights