DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number TAIWAN 112143572 dated 11/13/2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 10/04/2024 and 06/27/2025 have been considered and placed in the application file.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation using the
plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claim 10 and Claim 20 recite “at least one of” then listing “at least one of the plurality of first processing stages, at least one of the plurality of second processing stages and at least one of the plurality of third processing stages”. Since “at least one of” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim Objections
Claim 9 and 19 is objected to because of the following informalities:
Line 3 of Claim 9 and 19 currently reads “a time difference between lengths of time” Examiner could not find a definition in the specification for “a time difference” or “length of time”. Since there is no definition of the length of time being discussed in this limitation the metes and bounds of the scope of the claim is not clearly defined. Th examiner suggests adding a definition for the length of time claimed to properly define the scope of the claim. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1- 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claims 1 and 11 and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1- 20 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101. The Examiner will analyze Claim 1, similar rationale is applied to the analogous independent claim. The rationale, under MPEP § 2106, for this finding is explained below:
The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claim is related to a process since the claim is directed to a method.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 is directed to the abstract idea of using a mental process to separate, categorize, and sort the contextual elements of sequential frames taken from a video. The claim recites machine learning models claimed in a broad matter that does not amount to significantly more. The claims recite mental processes that include determining contextual elements of a video frame based on the elements present in the claim. Under Berkheimer v. HP, The Federal Circuit determined that these claims were directed to mental processes of parsing and comparing images, because the steps were recited at a high level of generality and merely used computers, specifically a processor, as a tool to perform the processes. Berkheimer v. HP, Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018), where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016);
If the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The Examiner interprets that the Claim 1 limitation does not provide additional elements or combination of additional elements to a practical application since the claims are not adding insignificant extra-solution activity to the judicial exception. The abstract idea is not integrated into a practical application because the additional elements fail to provide a technical improvement. Data gathering information that describes the contextual elements of the image do not add a practical application and could reasonably be done by a person selecting what contextual elements to focus on in the image still and separating the license plate from the vehicle in an image. Selecting frames with the contextual attributes and classifying the frames could be done in a human mind. Performing the processes at the same time is further data manipulation and does not integrate the invention into practical application. The instant specification ¶0004 describes the practical application being to utilize parallel processing to maximize computer resources, this applications fail to provide a technical improvement. Specifically, the analysis method does not integrate a judicial exception into practical application. See Genetic Techs. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016) (eligibility "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself."). For a claim reciting a judicial exception to be eligible, the additional elements (if any) in the claim must "transform the nature of the claim" into a patent-eligible application of the judicial exception, Alice Corp., 573 U.S. at 217, 110 USPQ2d at 1981, either at Prong Two or in Step 2B. If there are no additional elements in the claim, then it cannot be eligible. In such a case, after making the appropriate rejection, it is a best practice for the examiner to recommend an amendment, if possible, that would resolve eligibility of the claim.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the Claims state analyzing images for well-known characteristics with a high level of generality. The claims lack an inventive concept because the elements, considered individually and as an order combination are an abstract idea, and can easily be performed in the human mind based on data analysis, the data in this case being the context contained in frames from video images, where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).
Claims 2-10 and 12-20 depend on the independent claim/s include all the limitations of the independent claim. The Examiner finds that Claims 2 and 12 do not state significantly more since the claim adds limitation of processing the image into different stages, which are not clearly defined as to what occurs in each stage, is further data manipulation activity, which is further data manipulation activity which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is insignificant extra solution activity of additional data gathering 2106.05(g). The Examiner finds that Claims 3 and 13 do not state significantly more since the claim adds limitation of processing the image into different stages, which are not clearly defined as to what occurs in each stage, is further data manipulation activity, which is further data manipulation activity which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is insignificant extra solution activity of additional data gathering 2106.05(g). The Examiner finds that Claims 4 and 14 do not state significantly more since the claim adds limitation of processing the image into different stages, which are not clearly defined as to what occurs in each stage, is further data manipulation activity, which is further data manipulation activity which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is insignificant extra solution activity of additional data gathering 2106.05(g). The Examiner finds that Claim 5 and 15 do not state significantly more since the claim adds the limitation of determining if there is a second vehicle, and processing the data associated with the second vehicle is further data gathering activity; which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is insignificant extra solution activity of additional data gathering 2106.05(g). The Examiner finds that Claim 6, and 16 do not state significantly more since the claim adds the limitation of operating the pipelines synchronously is insignificant extra solution activity. It is insignificant extra solution activity 2106.05(g). The Examiner finds that Claims 7 and 17 do not state significantly more since the claim adds limitation of processing the image into different stages, which are not clearly defined as to what occurs in each stage, is further data manipulation activity, which is further data manipulation activity which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is insignificant extra solution activity of additional data gathering 2106.05(g). The Examiner finds that Claims 8 and 18 do not state significantly more since the claim adds element of location and plate number variables, which is further data gathering activity, which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is insignificant extra solution activity of additional data gathering 2106.05(g). The Examiner finds that Claims 9 and 19 do not state significantly more since the claim adds merging of data, which is further data manipulations activity, which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is insignificant extra solution activity of additional data manipulation 2106.05(g). The Examiner finds that Claim 10 and 20 do not state significantly more since the claim adds the limitation of operating the pipelines synchronously is insignificant extra solution activity. Therefore, it is insignificant extra solution activity 2106.05(g).
Thus, Claims 2-10 and 12-20 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more. Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
For the analogous independent claim 11 to the independent claim 1, the analogous limitations can be analyzed in the same way as above for the claim 1 hence rejected under 101. Moreover, the claim of 11 further recites at the statutory category of “system” respectively which is a limitation that the examiner interprets that the claim is related to a manufacture since the claim is directed to a system to implement the method of claim 1, and is consistent with the abstract ideas, the claims recite mental processes that include interpreting video frames and making decisions. Under Berkheimer v. HP, The Federal Circuit determined that these claims were directed to mental processes of parsing and comparing images, because the steps were recited at a high level of generality and merely used computers and computer components, including processor coupled to memory, as a tool to perform the processes. Berkheimer v. HP, Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018). This limitation of a residual component just further implements the abstract ideas to be performed by generic computer or software/hardware components of additional elements of the different types of analyzation methods. Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as unpatentable over Zhou et al. (CN107729818A (using translation from IP.com and images from google translate) hereafter referred to as Zhou) in view of Normington et al (US Patent Publication 2022/0309809 Al hereafter referred to as Normington) in further view of Al-Batat, Reda, et al. ("An end-to-end automated license plate recognition system using YOLO based vehicle and license plate detection with vehicle classification." Sensors 22.23 (2022): 9477 hereafter referred to as Al-Batat).
Regarding Claim 1, Zhou teaches an identification method for a license plate (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features), the identification method comprising:
sequentially obtaining a plurality of images (Zhou Pg 8 ¶10 discloses the vehicle images are captured by surveillance video, which the examiner is interpreting as equivalent to sequential, since the video has time stamps), wherein each of the plurality of images comprises one or more vehicles and at least one license plate (Zhou Fig 4 and Fig 5, and discloses the images taken containing the vehicle and license plate);
decomposing each of the plurality of images into a vehicle image and at least one license plate image (Zhou Fig 1 discloses the vehicle query pictures being split into two different pipelines, one for license plate identification and ones for vehicle identification),
through a plurality of first processing stages (Zhou Fig 1 discloses the vehicle query picture going through preprocessing as the first stage);
inputting the at least one vehicle into a vehicle metadata identification model to obtain vehicle metadata through a plurality of second processing stages (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers);
inputting the at least one license plate image into a license plate identification model to identify at least one piece of license plate information through a plurality of third processing stages (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo); and
merging (Zhou Fig 1, and Pg 8 ¶10 discloses both the license plate recognition and vehicle identification being combined to generate a single feature vector) the at least one piece of license plate information (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) and the vehicle metadata (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) to generate a license plate identification result (Zhou Fig 1 discloses visualization results as an outcome of combining the license plate and the vehicle identification).
wherein the plurality of first processing stages and the plurality of second processing stages form a first pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline), the plurality of third processing stages form a second pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a license plate identification to create the license plate recognition pipeline), and the first pipeline architecture and the second pipeline architecture are executed simultaneously (Zhou Fig 1 discloses the two pipelines being processed at the same time to be eventually merged).
PNG
media_image1.png
936
1048
media_image1.png
Greyscale
Zhou does not explicitly disclose configuring at least one processor to perform the following steps.
Normington is in the same field of image analysis of video content for license plate identification. Further, Normington teaches configuring at least one processor (Normington Fig 2, 212, ¶0133 discloses a processor to perform steps and the steps of a method or algorithm disclosed herein may be embodied in processor-executable instructions that may reside on a non-transitory computer-readable or processor-readable storage medium) to perform the following steps.
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhou by incorporating the structure of a processor and memory to integrate the method into a system as taught by Normington; to make an invention that can automatically take the images and determine the vehicle identification based on two different aspects of the vehicle image; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for improvement in accurately identifying characters on a license plate due to many license plates having a variety of designs or pictures included to indicate what country or state the plate is from, to support a special cause, or to allow a motorist to select a plate that they like. These designs or pictures can make it more difficult to detect characters on the plate when the pictures overlap the characters or even when the pictures are located on a perimeter of the plate as disclosed by Normington in ¶0004 and ¶0008.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
The combination of Zhou and Normington does not explicitly teach inputting the vehicle image into a vehicle detection model to detect at least one vehicle.
Al-batat is in the same field of image analysis of video content for license plate identification. Further, Al-batat teaches inputting the vehicle image into a vehicle detection model to detect at least one vehicle (Al-batat Fig 1 and Section 3.3 discloses a vehicle detection model that determines the vehicles present in the image).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhou in view of Normington by incorporating the vehicle detection module to determine if the process needs to be run concurrently of multiple vehicles as taught by Al-batat; to make an invention that can improve automatically detecting the number of vehicles in the image and run the process on multiple vehicles at once to reduce processing time; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for a system that does not require any pre-defined rules for the identification as disclosed by Al-batat in the introduction.
Regarding Claim 2, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 1, wherein the plurality of first processing stages (Zhou Fig 1 discloses the vehicle query picture going through preprocessing as the first stage) comprise:
a frame pre-processing stage (Zhou Fig 1 discloses a preprocessing stage);
a vehicle detection inference stage (Al-batat Fig 1 and Section 3.3 discloses a vehicle detection model that determines the vehicles present in the image); and
a first post-processing stage (Zhou Fig 1 discloses a descriptive feature fusion stage after the feature extraction stage). See Claim 1 for rationale, its parent claim.
Regarding Claim 3, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 1, wherein a third pipeline architecture is formed by the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline, wherein the metadata extraction is being interpreted as the third pipeline architecture), and the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline) comprise:
a first vehicle image pre-processing stage (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles);
a color classifier inference stage (Al-batat Fig 1 and Introduction discloses the next step after the cropping to be a vehicle classifier that could include color according to section 4.2 ); and
a second post-processing stage (Zhou Fig 1 discloses generating feature vectors which the examiner is interpreting to be the second post processing stage since it follows the descriptive feature fusion). See Claim 1 for rationale, its parent claim.
Regarding Claim 4, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 3, wherein a fourth pipeline architecture (Zhou Fig 1 discloses a fourth pipeline that performs license plate recognition) is formed by the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline), and the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline) comprise:
a second vehicle image pre-processing stage (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles, Table 12 discloses performing the image processing on multiple vehicles);
a manufacturer classifier inference stage (Al-batat Fig 1 and Introduction discloses the next step after the cropping to be a vehicle classifier that could include vehicle make according to section 4.2 ); and
a third post-processing stage (Zhou Fig 1 discloses generating feature vectors which the examiner is interpreting to be the third post processing stage since it follows the generate feature vectors). See Claim 1 for rationale, its parent claim.
Regarding Claim 5, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 4, wherein the step of inputting the at least one vehicle into the vehicle metadata identification model to obtain the vehicle metadata through the plurality of second processing stages (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) comprises:
inputting, in response to detecting that a quantity of the at least one vehicle is plural (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles, Table 12 discloses performing the image processing on multiple vehicles), different ones of the vehicle into the third pipeline architecture (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline, wherein the metadata extraction is being interpreted as the third pipeline architecture) and the fourth pipeline architecture (Zhou Fig 1 discloses a fourth pipeline that performs license plate recognition), respectively. See Claim 1 for rationale, its parent claim.
Regarding Claim 6, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 5, wherein the third pipeline architecture (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline, wherein the metadata extraction is being interpreted as the third pipeline architecture) and the fourth pipeline architecture (Zhou Fig 1 discloses a fourth pipeline that performs license plate recognition) are executed synchronously (Zhou Fig 1 discloses the metadata extraction and the license plate recognition occurring at the same level, therefore the examiner is interpreting these are being performed synchronously). See Claim 1 for rationale, its parent claim.
Regarding Claim 7, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 1, wherein the plurality of third processing stages (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) comprise:
a license plate image pre-processing stage (Zhou Fig 1 discloses license plate identification as the first part of the processing stages);
a license plate classifier inference stage (Zhou Fig 1 and Pg 2 ¶04, and Pg 4 ¶08 discloses a license plate recognition where the content of the license plate is determined); and
a fourth post-processing stage (Zhou Fig 1 discloses generating a plate logo vector as the post processing of the license plate recognition). See Claim 1 for rationale, its parent claim.
Regarding Claim 8, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 1, wherein the license plate identification model (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) comprises a location identification model (Zhou Pg 5 ¶1 discloses using the network to identify the location) and a license plate number identification model (Zhou Pg 5 ¶01 discloses using a neural network to identify the segmented characters), and the at least one piece of license plate information (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) comprises location information (Zhou Pg 5 ¶1 discloses using the network to identify the location) and license plate number information (Zhou Pg 5 ¶01 discloses using a neural network to identify the segmented characters). See Claim 1 for rationale, its parent claim.
Regarding Claim 9, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 1, wherein the merging (Zhou Fig 1, and Pg 8 ¶10 discloses both the license plate recognition and vehicle identification being combined to generate a single feature vector) of the at least one piece of license plate information (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) and the vehicle metadata (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) is synchronized (Zhou Fig 1 discloses the metadata extraction and the license plate recognition occurring at the same level, therefore the examiner is interpreting these are being performed synchronously) based on a time difference between lengths of time required for respectively executing (Al-batat Table 12 discloses the time difference between each pipeline when performing the image processing on multiple vehicles) the first pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline), and the second pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a license plate identification to create the license plate recognition pipeline). See Claim 1 for rationale, its parent claim.
Regarding Claim 10, Zhou in view of Normington in further view of Al-batat teaches the identification method (Zhou Pg 1 ¶03, Pg 2 ¶01, and Pg 2 ¶04 discloses an identification method including the license plate identification features) according to claim 1, wherein at least one of the plurality of first processing stages (Zhou Fig 1 discloses the vehicle query picture going through preprocessing as the first stage), at least one of the plurality of second processing stages (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) and at least one of the plurality of third processing stages (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) are executed simultaneously (Zhou Pg 2 ¶12 discloses the models being executed simultaneously) for different images of the plurality of images (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles, Table 12 discloses performing the image processing on multiple vehicles). See Claim 1 for rationale, its parent claim.
Regarding Claim 11, Zhou teaches sequentially obtaining a plurality of images (Zhou Pg 8 ¶10 discloses the vehicle images are captured by surveillance video, which the examiner is interpreting as equivalent to sequential, since the video has time stamps), wherein each of the plurality of images comprises one or more vehicles and at least one license plate (Zhou Fig 4 and Fig 5, and discloses the images taken containing the vehicle and license plate);
decomposing each of the plurality of images into a vehicle image and at least one license plate image (Zhou Fig 1 discloses the vehicle query pictures being split into two different pipelines, one for license plate identification and ones for vehicle identification)
through a plurality of first processing stages (Zhou Fig 1 discloses the vehicle query picture going through preprocessing as the first stage);
inputting the at least one vehicle into a vehicle metadata identification model to obtain vehicle metadata through a plurality of second processing stages (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers);
inputting the at least one license plate image into a license plate identification model to identify at least one piece of license plate information through a plurality of third processing stages (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo); and
merging (Zhou Fig 1, and Pg 8 ¶10 discloses both the license plate recognition and vehicle identification being combined to generate a single feature vector) the at least one piece of license plate information (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) and the vehicle metadata (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) to generate a license plate identification result (Zhou Fig 1 discloses visualization results as an outcome of combining the license plate and the vehicle identification);
wherein the plurality of first processing stages and the plurality of second processing stages form a first pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline), the plurality of third processing stages form a second pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a license plate identification to create the license plate recognition pipeline) , and the first pipeline architecture and the second pipeline architecture are executed simultaneously (Zhou Fig 1 discloses the two pipelines being processed at the same time to be eventually merged).
PNG
media_image1.png
936
1048
media_image1.png
Greyscale
Zhou does not explicitly disclose an identification system for a license plate, the identification system comprising:
at least one processor;
and a memory;
wherein the at least one processor is configured to perform the following steps.
Normington is in the same field of image analysis of video content for license plate identification. Further, Normington teaches an identification system for a license plate (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems), the identification system comprising:
at least one processor (Normington Fig 2, 212, ¶0133 discloses a processor to perform steps and the steps of a method or algorithm disclosed herein may be embodied in processor-executable instructions that may reside on a non-transitory computer-readable or processor-readable storage medium);
and a memory (Normington Fig 15 and ¶0055 discloses a memory);
wherein the at least one processor is configured to perform the following steps (Normington Fig 2, 212, ¶0133 discloses a processor to perform steps and the steps of a method or algorithm disclosed herein may be embodied in processor-executable instructions that may reside on a non-transitory computer-readable or processor-readable storage medium).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhou by incorporating the structure of a processor and memory to integrate the method into a system as taught by Normington; to make an invention that can automatically take the images and determine the vehicle identification based on two different aspects of the vehicle image; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for improvement in accurately identifying characters on a license plate due to many license plates having a variety of designs or pictures included to indicate what country or state the plate is from, to support a special cause, or to allow a motorist to select a plate that they like. These designs or pictures can make it more difficult to detect characters on the plate when the pictures overlap the characters or even when the pictures are located on a perimeter of the plate as disclosed by Normington in ¶0004 and ¶0008.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
The combination of Zhou and Normington does not explicitly teach inputting the vehicle image into a vehicle detection model to detect at least one vehicle.
Al-batat is in the same field of image analysis of video content for license plate identification. Further, Al-batat teaches inputting the vehicle image into a vehicle detection model to detect at least one vehicle (Al-batat Fig 1 and Section 3.3 discloses a vehicle detection model that determines the vehicles present in the image).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhou in view of Normington by incorporating the vehicle detection module to determine if the process needs to be run concurrently on multiple vehicles as taught by Al-batat; to make an invention that can automatically detect the number of vehicles in the image and run the process on multiple vehicles at once to reduce processing time; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for a system that does not require any pre-defined rules for the identification as disclosed by Al-batat in the introduction.
Regarding Claim 12, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 11, wherein the plurality of first processing stages (Zhou Fig 1 discloses the vehicle query picture going through preprocessing as the first stage) comprise:
a frame pre-processing stage (Zhou Fig 1 discloses a preprocessing stage);
a vehicle detection inference stage (Al-batat Fig 1 and Section 3.3 discloses a vehicle detection model that determines the vehicles present in the image); and
a first post-processing stage (Zhou Fig 1 discloses a descriptive feature fusion stage after the feature extraction stage). See Claim 11 for rationale, its parent claim.
Regarding Claim 13, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 11, wherein a third pipeline architecture is formed by the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline, wherein the metadata extraction is being interpreted as the third pipeline architecture), and the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline) comprise:
a first vehicle image pre-processing stage (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles);
a color classifier inference stage (Al-batat Fig 1 and Introduction discloses the next step after the cropping to be a vehicle classifier that could include color according to section 4.2 ); and
a second post-processing stage (Zhou Fig 1 discloses generating feature vectors which the examiner is interpreting to be the second post processing stage since it follows the descriptive feature fusion). See Claim 11 for rationale, its parent claim.
Regarding Claim 14, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 13, wherein a fourth pipeline (Zhou Fig 1 discloses a fourth pipeline that performs license plate recognition) architecture is formed by the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline), and the plurality of second processing stages (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline) comprise:
a second vehicle image pre-processing stage (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles, Table 12 discloses performing this on multiple vehicles);
a manufacturer classifier inference stage (Al-batat Fig 1 and Introduction discloses the next step after the cropping to be a vehicle classifier that could include vehicle make according to section 4.2 ); and
a third post-processing stage (Zhou Fig 1 discloses generating feature vectors which the examiner is interpreting to be the third post processing stage since it follows the generate feature vectors). See Claim 11 for rationale, its parent claim.
Regarding Claim 15, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 14, wherein the step of inputting the at least one vehicle into the vehicle metadata identification model to obtain the vehicle metadata through the plurality of second processing stages (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) comprises:
inputting, in response to detecting that a quantity of the at least one vehicle is plural (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles, Table 12 discloses performing the image processing on multiple vehicles), different ones of the vehicle into the third pipeline architecture (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline, wherein the metadata extraction is being interpreted as the third pipeline architecture) and the fourth pipeline architecture (Zhou Fig 1 discloses a fourth pipeline that performs license plate recognition), respectively. See Claim 11 for rationale, its parent claim.
Regarding Claim 16, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 15, wherein the third pipeline architecture (Zhou Fig 1 discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline, wherein the metadata extraction is being interpreted as the third pipeline architecture) and the fourth pipeline architecture (Zhou Fig 1 discloses a fourth pipeline that performs license plate recognition) are executed synchronously (Zhou Fig 1 discloses the metadata extraction and the license plate recognition occurring at the same level, therefore the examiner is interpreting these are being performed synchronously). See Claim 11 for rationale, its parent claim.
Regarding Claim 17, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 11, wherein the plurality of third processing stages (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) comprise:
a license plate image pre-processing stage (Zhou Fig 1 discloses license plate identification as the first part of the processing stages);
a license plate classifier inference stage (Zhou Fig 1 and Pg 2 ¶04, and Pg 4 ¶08 discloses a license plate recognition where the content of the license plate is determined); and
a fourth post-processing stage (Zhou Fig 1 discloses generating a plate logo vector as the post processing of the license plate recognition). See Claim 11 for rationale, its parent claim.
Regarding Claim 18, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 11, wherein the license plate identification model (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) comprises a location identification model (Zhou Pg 5 ¶1 discloses using the network to identify the location) and a license plate number identification model (Zhou Pg 5 ¶01 discloses using a neural network to identify the segmented characters), and the at least one piece of license plate information (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) comprises location information (Zhou Pg 5 ¶1 discloses using the network to identify the location) and license plate number information (Zhou Pg 5 ¶01 discloses using a neural network to identify the segmented characters). See Claim 11 for rationale, its parent claim.
Regarding Claim 19, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 11, wherein the merging (Zhou Fig 1, and Pg 8 ¶10 discloses both the license plate recognition and vehicle identification being combined to generate a single feature vector) of the at least one piece of license plate information (Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) and the vehicle metadata (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) is synchronized (Zhou Fig 1 discloses the metadata extraction and the license plate recognition occurring at the same level, therefore the examiner is interpreting these are being performed synchronously based on a time difference between lengths of time required for respectively executing (Al-batat Table 12 discloses the time difference between each pipeline when performing the image processing on multiple vehicles) the first pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a first preprocessing stage and a second global feature extraction stage to create the vehicle identification pipeline), and the second pipeline architecture (Zhou Fig 1 (see annotation of Figure 1 below) discloses the vehicle query images going through a license plate identification to create the license plate recognition pipeline). See Claim 11 for rationale, its parent claim.
Regarding Claim 20, Zhou in view of Normington in further view of Al-batat teaches the identification system (Normington ¶0002, ¶0034-¶0035, and Fig 1 discloses automatic license plate recognition (ALPR) systems) according to claim 11, wherein at least one of the plurality of first processing stages (Zhou Fig 1 discloses the vehicle query picture going through preprocessing as the first stage), at least one of the plurality of second processing stages (Zhou Pg 1, ¶08 and Fig 1 discloses the vehicle pictures being input into a neural network model that extracts features of the vehicles to detect the properties of the vehicle in the processing, that can include the type of vehicle, color, doors, and number of passengers) and at least one of the plurality of third processing stages(Zhou Fig 1, Pg 2 ¶04, and Pg 3 ¶13 discloses the license plate images going through license plate identification and recognition to generate a license plate logo) are executed simultaneously (Zhou Pg 2 ¶12 discloses the models being executed simultaneously) for different images of the plurality of images (Al-batat Fig 1 and Introduction discloses a vehicle detection model that determines the vehicles present in the image as the first stage of processing and cropping the image to separate the vehicles, Table 12 discloses performing the image processing on multiple vehicles). See Claim 11 for rationale, its parent claim.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
WO Patent Pub WO-2022027873-A1 to Yan et al. discloses a vehicle re-identification method and device based on multi-modal information fusion, which can solve the problem that the existing reidentification method only uses the vehicle appearance or the license plate number to re-identify the vehicle, and the applied identification features.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights