Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 9 is objected to because of the following informalities: The claim reads 1-distributed stochastic neighbour embedding, which is an obvious typographical error, because the previous version of the claims reads t-distributed stochastic neighbour embedding, no amendments are indicated, and because there is no support in the specification for 1-distributed. For the purpose of examination, the claim will be examined as reading t-distributed stochastic neighbour embedding.
Claim 11 is objected to because of the following informalities: The claim reads one or more selected data instance which appears to be a typographical error which should read one or more selected data instances.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4-7, 10, and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tian et al., “A Face Annotation Framework with Partial Clustering and Interactive Labeling,”
Regarding Claim 1, Tian teaches a method of modelling an unlabeled or partially labeled target dataset with a machine learning model for classification or regression (Tian, Abstract, “a novel interactive face annotation framework combining unsupervised and interactive learning” where face annotation denotes classification and the face images are initially partially labeled, see pg. 4, 1st column, 4th paragraph, “Suppose there are K labeled groups of identities … and an unlabeled face set
G
0
which define the beginning state”) comprising: processing the target dataset by the machine learning model (Tian, pg. 4, 1st column, 3rd paragraph, “In each step, the system uses the information from the labeled faces to automatically infer an optimal subset of unlabeled faces for user annotation”); preparing a subgroup of the target dataset for presentation to a user for labelling or label verification; receiving label verification or user re-labelling or user labelling of the subgroup (Tian, pg. 4, 1st column, 3rd paragraph, “to automatically infer an optimal subset of unlabeled faces for user annotation. The annotation step will be iteratively used until all faces are labeled. Using this strategy, the overall user interactions can be reduced” & pg. 2, 2nd column, 2nd paragraph, “a user can do batch labeling with only one click”); and re-processing the updated target dataset by the machine learning model (Tian, pg. 4, 1st column, 3rd paragraph, “The annotation step will be iteratively used until all faces are labeled … by finding an optimal subset of all unlabeled faces in each annotation step” where finding the next optimal subset for labeling is processing the updated target dataset by the model).
Regarding Claim 4, Tian teaches a method according to Claim 1 (and thus the rejection of Claim 1 is incorporated). Tian further teaches wherein the preparing comprises determining a plurality of representative data instances and preparing a cluster plot of only those representative data instances for presenting that cluster plot (Tian, pg. 4, 1st column, 2nd paragraph, “The partial clustering algorithm automatically groups similar faces into several evident clusters” & pg. 2, 2nd column, 2nd paragraph, “a user can do batch labeling with only one click” after viewing the cluster, e.g. a plot of only the images in that cluster).
Regarding Claim 5, Tian teaches a method according to Claim 4 (and thus the rejection of Claim 4 is incorporated). Tian further teaches wherein the plurality of representative data instances is determined in feature space or input space (Tian presents the images to the user for annotation, thus the instances are determined in input space).
Regarding Claim 6, Tian teaches a method according to Claim 4 (and thus the rejection of Claim 4 is incorporated). Tian further teaches wherein the plurality of representative data instances is determined in input space (Tian presents the images to the user for annotation, thus the instances are determined in input space).
Regarding Claim 7, Tian teaches a method according to Claim 4 (and thus the rejection of Claim 4 is incorporated). Tian further teaches wherein the plurality of representative data instances is determined by sampling (Tian, pg. 5, 1st column, 4th paragraph, “we first pick one unlabeled face
x
d
as the seed of
Q
” where picking one face to start is sampling).
Regarding Claim 10, Tian teaches a method according to Claim 1 (and thus the rejection of Claim 1 is incorporated). Tian further teaches wherein the preparing comprises … identifying similar data instances to one or more selected data instances (Tian, pg. 5, 1st column, 4th paragraph, “we first pick one unlabeled face
x
d
as the seed of
Q
, and then do a local search over its neighbors, each time searching for [instances with maximum
a
i
d
which is similarity, by pg. 4, 2nd column, 3rd paragraph, “
a
i
j
is the similarity measure between face
i
and
j
”] and put it into
Q
until Eq. 13 starts to decrease”) by a Bayesian sets method (where “Q” is a set and the similarity is Bayesian, see pg. 2, 2nd column, 6th paragraph, “Using the Bayesian rule, face similarity can be calculated” i.e. Eq. 1) for presenting those similar data instances (as part of the clustering algorithm to determine which images to present).
Regarding Claim 11, Tian teaches a method according to Claim 1 (and thus the rejection of Claim 1 is incorporated). Tian further teaches wherein the preparing comprises identifying similar data instances to one or more selected data instances (Tian, pg. 5, 1st column, 4th paragraph, “we first pick one unlabeled face
x
d
as the seed of
Q
, and then do a local search over its neighbors, each time searching for [instances with maximum
a
i
d
which is similarity, by pg. 4, 2nd column, 3rd paragraph, “
a
i
j
is the similarity measure between face
i
and
j
”] and put it into
Q
until Eq. 13 starts to decrease”) by a Bayesian sets method (where “Q” is a set and the similarity is Bayesian, see pg. 2, 2nd column, 6th paragraph, “Using the Bayesian rule, face similarity can be calculated” i.e. Eq. 1) for presenting those similar data instances (as part of the clustering algorithm to determine which images to present).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Tian et al., “A Face Annotation Framework with Partial Clustering and Interactive Labeling.”
Regarding Claim 3, Tian teaches a method according to Claim 1 (and thus the rejection of Claim 1 is incorporated). Tian further teaches determining a targeted subgroup of the target dataset for targeted presentation to a user for labelling … of that targeted subgroup (Tian, pg. 4, 1st column, 3rd paragraph, “to automatically infer an optimal subset of unlabeled faces for user annotation”). The invention of Tian itself does not teach label verification, but Tian (in a background section) discusses this feature (Tian, pg. 2, 1st column, 2nd paragraph, “Suh et al. proposed a framework to allow cluster annotation … the errors in the clustering need to be corrected one by one by the user”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to allow label verification in the invention of Tian. The motivation to do so is to allow the correction of any errors in the clustering.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Tian, in view of Wang et al., “Unsupervised Category Discovery via Looped Deep Pseudo-Task Optimization Using a Large Scale Radiology Image Database.”
Regarding Claim 2, Tian teaches a method according to Claim 1 (and thus the rejection of Claim 1 is incorporated). Tian does not teach wherein the machine learning algorithm is a convolutional neural network, a support vector machine, a random forest or a neural network. Rather, Tian’s machine learning algorithm clusters image features (Tian, pg. 2, 2nd column, 4th paragraph), but is silent on how the image features are obtained. However, Wang, in the analogous art of image clustering, teaches that CNNs are good ways to extract features for image clustering (Wang, Fig. 1, “Deep CNN features extracting and encoding” before “Clustering CNN features”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate CNN feature extraction of Wang into the image clustering system of Tian. The motivation to do so is that “CNN models … offer more effective and deep image features to facility more meaningful clustering” (Wang, pg. 2, 1st column, last paragraph).
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Tian, in view of van der Maaten et al., “Visualizing Data using t-SNE.”
Regarding Claim 8, Tian teaches a method according to Claim 4 (and thus the rejection of Claim 4 is incorporated). Tian does not teach, but van der Maaten does teach wherein the preparing comprises a dimensionality reduction of the plurality of representative data instances to 2 or 3 dimensions, optionally wherein the dimensionality reduction is by t-distributed stochastic neighbor embedding (van der Maaten, title, “Visualizing Data using t-SNE” & Abstract, “by giving each datapoint a location in a two or three-dimensional map) & pg. 2590, Fig, 2(a)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention a plot such as that of van der Maaten, obtained via t-SNE, in the visualization/label selection method of Tian. The motivation to do so is to allow labelling users to visualize outliers and to visualize the clusters produced by Tian (van der Maaten, Abstract, “t-SNE is better than existing techniques at creating a single map that reveals structure at many different scales” of the clustered data).
Regarding Claim 9, the Tian/van der Maaten combination of Claim 8 teaches a method according to Claim 8 (and thus the rejection of Claim 8 is incorporated). The combination has already been demonstrated to teach wherein the dimensionality reduction is by t-distributed stochastic neighbor embedding (van der Maaten, title, “Visualizing Data using t-SNE”).
Claims 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Tian, in view of Haller, US Patent 10,380,696.
Regarding Claim 12, Tian teaches a method of producing a computational model … with a machine learning model (Tian, Abstract, “a novel interactive face annotation framework combining unsupervised and interactive learning”) comprising: receiving a plurality of unlabeled … images (Tian, pg. 4, 1st column, 4th paragraph, “Suppose there are K labeled groups of identities … and an unlabeled face set
G
0
which define the beginning state”) processing the … images by the machine learning model (Tian, pg. 4, 1st column, 3rd paragraph, “In each step, the system uses the information from the labeled faces to automatically infer an optimal subset of unlabeled faces for user annotation”); preparing a subgroup of the … images for presentation to a user for labelling or label verification; receiving label verification or user re-labelling or user labelling of the subgroup (Tian, pg. 4, 1st column, 3rd paragraph, “to automatically infer an optimal subset of unlabeled faces for user annotation. The annotation step will be iteratively used until all faces are labeled. Using this strategy, the overall user interactions can be reduced” & pg. 2, 2nd column, 2nd paragraph, “a user can do batch labeling with only one click”); and re-processing the plurality of … images by the machine learning model (Tian, pg. 4, 1st column, 3rd paragraph, “The annotation step will be iteratively used until all faces are labeled … by finding an optimal subset of all unlabeled faces in each annotation step” where finding the next optimal subset for labeling is processing the updated set of images by the model).
Tian teaches annotation and clustering for facial images, not vehicle images, but Haller teaches clustering of vehicle images (Haller, column 11, lines 50-62, “for performing a cluster analysis on the claim data prior to performing the data analysis … so that a similarity between data point (e.g. particular images …) within a cluster is maximized … resulting in sets of insurance claims clusters having similar image attribute vectors”). It would have been obvious to one of ordinary skill in the art before the effective filing date of claimed invention to use the annotation and clustering method of Tian on vehicle images instead of faces. The motivation to do so is to annotate vehicle images for machine learning insurance claim systems such as that of Haller.
Regarding Claim 13, the Tian/Haller combination of Claim 12 teaches a method according to Claim 12 (and thus the rejection of Claim 12 is incorporated). Tian further teaches determining a targeted subgroup of the … images for targeted presentation to a user for labelling … of that targeted subgroup (Tian, pg. 4, 1st column, 3rd paragraph, “to automatically infer an optimal subset of unlabeled faces for user annotation,” where vehicle images are taught via Haller in the combination). The invention of Tian itself does not teach label verification, but Tian (in a background section) discusses this feature (Tian, pg. 2, 1st column, 2nd paragraph, “Suh et al. proposed a framework to allow cluster annotation … the errors in the clustering need to be corrected one by one by the user”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to allow label verification in the invention of Tian. The motivation to do so is to allow the correction of any errors in the clustering.
Regarding Claim 14, the Tian/Haller combination of Claim 12 teaches a method according to Claim 12 (and thus the rejection of Claim 12 is incorporated). Tian further teaches wherein the preparing comprises … identifying similar data instances to one or more selected data instances (Tian, pg. 5, 1st column, 4th paragraph, “we first pick one unlabeled face
x
d
as the seed of
Q
, and then do a local search over its neighbors, each time searching for [instances with maximum
a
i
d
which is similarity, by pg. 4, 2nd column, 3rd paragraph, “
a
i
j
is the similarity measure between face
i
and
j
”] and put it into
Q
until Eq. 13 starts to decrease”) by a Bayesian sets method (where “Q” is a set and the similarity is Bayesian, see pg. 2, 2nd column, 6th paragraph, “Using the Bayesian rule, face similarity can be calculated” i.e. Eq. 1) for presenting those similar data instances (as part of the clustering algorithm to determine which images to present).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Tian, in view of Haller, and further in view of Kanhere, US PG Pub 2010/0322476.
Regarding Claim 15, the Tian/Haller combination of Claim 12 teaches a method according to Claim 12 (and thus the rejection of Claim 12 is incorporated). Tian/Haller does not teach any combination of vehicle and non-vehicle images, but Kanhere teaches receiving a plurality of non-vehicle images with the plurality of unlabeled vehicle images (Kanhere, [0142], “a set of training images comprising vehicles and non-vehicles”) and Kanhere further teaches the desirability of separating vehicle and non-vehicle images, e.g. removing the non-vehicle images to produce a plurality of unlabeled vehicle images (Kanhere, [0142], “to separate vehicle images from non-vehicle images”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the clustering and labelling method of Tian (including the image verification taught in Tian, pg. 2, 1st column, 2nd paragraph) to cluster the images into vehicle and non-vehicle categories in order to get only vehicle images, thus teaching processing the vehicle images with the non-vehicle images by the machine learning model; preparing the non-vehicle images for presentation to a user for verification, receiving verification of the non-vehicle images (see the rejection of Claim 12, applied to vehicle and non-vehicle images). The motivation to do so is to have a clean set of vehicle-only images for use in the Tian/Haller insurance claim machine learning combination, i.e. data cleaning and preparation.
Claim 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Tian, in view of Haller, and further in view of Collins, US Patent 9,824,453.
Regarding Claim 16, the Tian/Haller combination of Claim 12 teaches a method according to Claim 12 (and thus the rejection of Claim 12 is incorporated). Tian/Haller does not teach wherein the subgroup of vehicles all show a specific vehicle part, but Collins teaches this limitation (Collins, column 62, lines 27-35, “The computing device may generate clusters of images … The device may identify, for example, 4 clusters of vehicles. The computing device may classify … vehicles as having an added spoiler” i.e. vehicles in a subgroup/cluster will all show a specific vehicle part). It would have been obvious to one of ordinary skill in the art, in the Tian/Haller combination, to generate clusters showing a specific vehicle part. The motivation to do so is to properly identify images with which to process insurance claims in the Tian/Haller combination.
Regarding Claim 17, the Tian/Haller/Collins combination of Claim 16 teaches a method according to Claim 12, wherein the subgroup of vehicle images all show a specific vehicle part (and thus the rejection of Claim 16 is incorporated). The combination has not yet been shown to teach a specific vehicle part in a damaged condition, but Haller teaches a) that it is desirable to cluster vehicle images on common attributes (Haller, column 11, lines 50-62, ”resulting in sets of insurance claims clusters having similar image attribute vectors”) and b) that damaged vehicle parts are image attributes which we want the system to detect (Haller, column 2, lines 37-42, “a plurality of images of a damaged vehicle … a set of image attributes that is indicative of a content of at least some of the plurality of images of the damaged vehicle”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to determine clusters of damaged vehicle parts in the Tian/Haller/Collins invention. The motivation to do so is to aid in the insurance claim processing of the Tian/Haller combination.
Regarding Claim 18, the Tian/Haller/Collins combination of Claim 17 teaches a method according to Claim 12, wherein the subgroup of vehicle images all show a specific vehicle part in a damaged condition (and thus the rejection of Claim 17 is incorporated). The combination has not yet been shown to teach a specific vehicle part in a damaged condition capable of repair, but Haller teaches such parts (Haller, column 2, lines 58-61, “based on the set of image attributes indicative of the damaged vehicle, respective indications of one or more replacement parts needed to repair the damaged vehicle”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to determine clusters of repairable damaged vehicle parts in the Tian/Haller/Collins invention. The motivation to do so is to aid in the insurance claim processing of the Tian/Haller combination.
Regarding Claim 19, the Tian/Haller/Collins combination of Claim 17 teaches a method according to Claim 12, wherein the subgroup of vehicle images all show a specific vehicle part in a damaged condition (and thus the rejection of Claim 17 is incorporated). The combination has not yet been shown to teach a specific vehicle part in a damaged condition suitable for replacement, but Haller teaches such parts (Haller, column 2, lines 58-61, “based on the set of image attributes indicative of the damaged vehicle, respective indications of one or more replacement parts needed to repair the damaged vehicle”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to determine clusters of replaceable damaged vehicle parts in the Tian/Haller/Collins invention. The motivation to do so is to aid in the insurance claim processing of the Tian/Haller combination.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Chen, US Patent 11,288,789 also teaches cluster analysis for damage vehicle insurance analysis.
Aubry et al., “Understanding deep features with computer-generated imagery” teaches dimension reduction of images for understanding clusters.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN M SMITH whose telephone number is (469)295-9104. The examiner can normally be reached Monday - Friday, 8:00am - 4pm Pacific.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN M SMITH/Primary Examiner, Art Unit 2122
Prosecution Insights