Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, 7, 9, 11-12, 15-23 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Rodriguez-Serrano, hereafter referred to as Rodriguez (Rodriguez-Serrano, J.A., Sandhawalia, H., Bala, R., Perronnin, F., Saunders, C. (2012). Data-Driven Vehicle Identification by Image Matching. In: Fusiello, A., Murino, V., Cucchiara, R. (eds) Computer Vision – ECCV 2012. Workshops and Demonstrations. ECCV 2012. Lecture Notes in Computer Science, vol 7584. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33868-7_53).
Regarding claim 1, Rodriguez discloses a vehicle identity recognition device ([Introduction, first paragraph] vehicle identification from images such as those included in cameras) comprising:
a data input interface configured to receive a first image of a first vehicle and a second image of a second vehicle ([pg. 538, second paragraph] given an image of an unknown vehicle (i.e. first image), we propose to describe the license plate sub-image and match it against a database of annotated license images to find a near-duplicate image (i.e. second image)); and
at least one processor configured to control ([Introduction, first paragraph] the system for vehicle identification from images may be implemented on devices such as mobile phones (i.e. which include processors); see also [pg. 542, 4 Experiments]):
a region of interest extractor to extract a first region of interest from the first image ([pg. 537, Fig. 1] plate detection from vehicle image; see also [pg. 541, first paragraph] plate localization algorithm to yield tight plate regions) and
extract a second region of interest from the second image corresponding to the first region of interest (Fig. 3-4, [pg. 540, Synthesizing an Ideal License Plate Image.] synthesizing license plate images by extracting distorted versions of an original license plate image (see also [pg. 540, Photo-Realistic Transformation.])),
the first region of interest and the second region of interest being partial regions of a vehicle including a vehicle license plate ([pg. 537, Fig. 1] plate detection from vehicle image; see also [pg. 541, first paragraph] plate localization algorithm to yield tight plate regions, wherein both the original and synthesized images are license plate images);
a machine learner to perform machine learning by inputting the first region of interest as training data ([pg. 541, 3.2 Similarity Learning] the training procedure includes a step including components r and s, real plate image and simulated plate images (i.e. first ROI images));
an image matcher to determine whether the first region of interest on which the machine learning is performed matches the second region of interest ([pg. 541, 3.2 Similarity Learning] minimizing the loss makes pairs with the same identity have a higher similarity than non-matching pairs (i.e. of r (original) and s (synthesized) plate images));
a license plate identifier to identify whether a license plate of the first vehicle is identical to a license plate of the second vehicle; and a controller to recognize the second vehicle as an identical vehicle based on the first region of interest matching the second region of interest and the license plate of the first vehicle being identical to the license plate of the second vehicle ([pg. 537, first paragraph] given a plate image region, search for a near-duplicate image in an annotated database, and if a duplicate is found then the identity of the near-duplicate is transferred to the input region image (i.e. the vehicle in the input image is determined to have the identity of the vehicle in the duplicate image); see also [pg. 539, Matching.]).
Regarding claim 2, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the at least one processor is further configured to control: a weight applier to apply different weights to each of a vehicle license plate area and a vehicle license plate external region for at least one of the first region of interest and the second region of interest ([pg. 541, 3.2 Similarity Learning], as in equation (2), the learning process involves applying a weight (1-k) to a sample with the same identity as the real sample and a weight (k) to a sample with a different identity (i.e. external to the license plate)).
Regarding claim 3, Rodriguez discloses the vehicle identity recognition device of claim 2. Rodriguez further discloses wherein a first weight is applied to the vehicle license plate area and a second weight, higher than the first weight, is applied to the vehicle license plate external region ([pg. 541, 3.2 Similarity Learning], as in equation (2), the learning process involves applying a weight (1-k) to a sample with the same identity as the real sample and a weight (k) to a sample with a different identity (i.e. external to the license plate)).
Regarding claim 4, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the at least one processor is further configured to control: an image scrambler to apply an artificial image change to at least one of the first region of interest and the second region of interest, and wherein the artificial image change comprises one or more of an image brightness change, a contrast change, a blur change, and image tampering ([pg. 540, Photo-realistic Transformation] a number of transformations and distortions are applied to produce realistic images which simulate camera capture, Fig. 3(b) illustrates some operations considered (e.g. in Fig. 3B, distortions of poor contrast, rotation, shadow, blurry, and rust are applied to versions of the license plate image)).
Regarding claim 5, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the at least one processor is further configured to control: a feature point extractor to extract feature points for each of the extracted first region of interest and the extracted second region of interest ([pg. 538, Plate Features] for the license plate image descriptor we extract Fisher vectors (i.e. by extracting feature points form the image regions)), the machine learner to perform the machine learning based on the extracted feature points as input, and the image matcher to determine whether the regions of interest are matched based on the extracted feature points ([pg. 541, 3.2 Similarity Learning] measuring of similarity between Fisher vectors as a measure of matching between synthetic and real plate images by the learning model to determine if there is a match).
Regarding claim 7, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the at least one processor is further configured to control the license plate identifier to: convert a license plate included in the first region of interest and a license plate included in the second region of interest into a frontal image, and determine whether the license plate of the second vehicle matches the license plate of the first vehicle ([pg. 537, last paragraph], Fig. 1 shows a vehicle image taken at an angle wherein the plate is then detected and displayed from a front angle, the image is then matched to a synthetic second region to determine the output match).
Regarding claim 9, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the at least one processor is further configured to control the image matcher to: convert the first region of interest on which the machine learning is performed and the second region of interest into images of a same angle, and determine whether the converted images match ([pg. 537, last paragraph], Fig. 1 shows a vehicle image taken at an angle wherein the plate is then detected and displayed from a front angle, the image is then matched to a synthetic second region which is at the same front angle to determine the output match).
Regarding claim 11, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the machine learning is at least one of supervised learning and unsupervised learning ([pg. 541, 3.2 Similarity Learning] supervised semantic indexing approach for similarly learning model).
Regarding claim 12, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the at least one processor is further configured to control the machine learner to perform machine learning by inputting the first region of interest as training data for each of three color components ([pg. 540, Photo-Realistic Transformation] the images are transformed to RGB (three-channel color) images; [pg. 541, 3.2 Similarity Learning] the synthetic color images are then used to train the similarity learning model).
Regarding claim 15, Rodriguez discloses everything claimed as applied above (see rejection of claim 1), further including vehicle identity recognition method performed by a vehicle identity recognition device ([Introduction, first paragraph] vehicle identification from images such as those included in cameras) including at least one processor and a memory that stores instructions executable by the at least one processor, the vehicle identity recognition method performed by the instructions under the control of the at least one processor ([Introduction, first paragraph] the system for vehicle identification from images may be implemented on devices such as mobile phones (i.e. which include processors, memory, etc.); see also [pg. 542, 4 Experiments]).
Regarding claim 16, Rodriguez discloses the vehicle identity recognition method of claim 15 as applied above. Rodriguez further discloses everything claimed as applied above (see rejection of claim 2).
Regarding claim 17, Rodriguez discloses the vehicle identity recognition method of claim 16 as applied above. Rodriguez further discloses everything claimed as applied above (see rejection of claim 3).
Regarding claim 18, Rodriguez discloses the vehicle identity recognition method of claim 15 as applied above. Rodriguez further discloses everything claimed as applied above (see rejection of claim 4).
Regarding claim 19, Rodriguez discloses the vehicle identity recognition method of claim 15 as applied above. Rodriguez further discloses extracting feature points from each of the first region of interest and the second region of interest ([pg. 538, Plate Features] for the license plate image descriptor we extract Fisher vectors (i.e. by extracting feature points form the image regions)).
Regarding claim 20, Rodriguez discloses everything claimed as applied above (see rejection of claim 1), further including a vehicle identity recognition device ([Introduction, first paragraph] vehicle identification from images such as those included in cameras) comprising: a data input interface configured to receive a first image of a first vehicle and a second image of a second vehicle ([pg. 542, 4 Experiments] on-board cameras and mobile phones can be used to apply the method including capturing and inputting images); and at least one processor configured to control ([Introduction, first paragraph] the system for vehicle identification from images may be implemented on devices such as mobile phones (i.e. which include processors); see also [pg. 542, 4 Experiments]).
Regarding claim 21, Rodriguez discloses the vehicle identity recognition device of claim 1 as applied above. Rodriguez further discloses wherein the at least one processor is further configured to control an entry or an exit of the second vehicle based on whether the second vehicle is recognized as the identical vehicle ([pg. 544, 5 Conclusions] entry/exit control is an application that can benefit from the propose method (i.e. of matching vehicle license plate image to another license plate image); see also [pg. 539, 3 License Plate Image Simulation and Matching]).
Regarding claim 22, Rodriguez discloses the vehicle identity recognition device of claim 15 as applied above. Rodriguez further discloses everything claimed as applied above (see rejection of claim 21).
Regarding claim 23, Rodriguez discloses the vehicle identity recognition device of claim 20 as applied above. Rodriguez further discloses everything claimed as applied above (see rejection of claim 21).
Allowable Subject Matter
Claims 6, 8, 10, 13-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 6, Rodriguez discloses the vehicle identity device of claim 1 as applied above. Rodriguez fails to disclose wherein the license plate of the first vehicle is a value input by a user, and wherein the at least one processor is further configured to control the license plate identifier to: recognize the license plate of the second vehicle by OCR, and determine whether the license plate of the second vehicle matches the license plate of the first vehicle.
Regarding claim 8, Rodriguez discloses the vehicle identity device of claim 7 as applied above. Rodriguez fails to disclose wherein the at least one processor is further configured to control the license plate identifier to: recognize at least one of a font, an aspect ratio, and a blank ratio of the license plate included in the first region of interest and the license plate included in the second region of interest.
Regarding claim 10, Rodriguez discloses the vehicle identity device of claim 1 as applied above. Rodriguez fails to disclose wherein the at least one processor is further configured to control: based on a matching probability between the first region of interest on which the machine learning is performed and the second region of interest being greater than or equal to a reference value, the image matcher to determine that the first region of interest matches the second region of interest.
Regarding claim 13, Rodriguez discloses the vehicle identity device of claim 1 as applied above. Rodriguez fails to disclose wherein the data input interface is further configured to repeatedly receive the first image of the first vehicle a predetermined number of times.
Regarding claim 14, Rodriguez discloses the vehicle identity device of claim 1 as applied above. Rodriguez fails to disclose wherein the at least one processor is further configured to control: based on a time at which the first vehicle is recognized by a specific camera and a second time at which the second vehicle is recognized by the specific camera being within a predetermined threshold, the controller to recognize the first vehicle and the second vehicle as different vehicles.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kang (KR 20170084463 A) discloses a vehicle authentication device based on character recognition of vehicle license plates such as for parking management systems.
Tatematsu (US 20220375235 A1) discloses management of a vehicle by extracting feature information and a predetermined database, further including analyzing license plate number images.
Popov (US 20190251369 A1) discloses a license plate detection and recognition system including training data to prepare ground truth data and generating bounding boxes of candidate images.
Wang (CN 112597995 A) discloses a license plate detection model training method including capturing images in different scenarios and imaging conditions.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLINE DEPALMA whose telephone number is (571)270-0769. The examiner can normally be reached Mon-Thurs 9:00am-4pm Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at 571-272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLINE E. DEPALMA/Examiner, Art Unit 2675
/SJ Park/Primary Examiner, Art Unit 2675
Prosecution Insights