DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This action is in response to the Application filed on 09/25/2024.
Claims 1-19 are pending.
Priority
The present Application claims priority to EP-23201829.1 with a priority date of 10/05/2023, which is acknowledged.
Information Disclosure Statement
The Information Disclosure Statement (IDS) filed on 09/25/2024 has been fully considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“Computing device”, present in claims 12-19, with corresponding structure found in at least paragraph 0034 of the originally-filed specification.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
If Applicant wishes to avoid the above 112(f) interpretation, the examiner suggests amending claims 12-19 to rename the “computing device” to simply a “computer”.
The examiner notes that the “camera device” of claims 12-18 does not appear to invoke a 112(f) interpretation, as the term “camera” conveys clear structure, unlike the broader term of “computing device”.
35 USC § 101 Analysis
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The examiner has determined that all of claims 1-19 are eligible under 35 U.S.C. 101. The 101 analysis is provided below for a clear record.
Analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added):
A method for automatic license plate recognition, comprising:
recording a first set of images each including one of N1 different license plate numbers;
extracting the license plate number in each image of the first set;
generating an artificial neural network with one separate output node for each of the N1 different license plate numbers in the first set;
training the artificial neural network on the images and the extracted license plate numbers of the first set at least until for each image of the first set the output node for the license plate number included in that image outputs the highest value of all output nodes;
recording a sample image including a license plate number that is also included in the images the artificial neural network has been trained on;
and feeding the sample image into the artificial neural network and recognising the license plate number of the sample image as the license plate number of that output node which outputs the highest value.
Step 1: Does the claim belong to one of the statutory categories? Claim 1 is directed to a process, which is a statutory category of invention (YES).
Step 2A Prong One: Does the claim recite a judicial exception? Steps c and g include mental processes, such as observations, evaluations, judgements, or opinions, that can be practically performed in the human mind, or by a human using pen and paper. Step c requires extracting a license plate number from each image of a set of images; a human could perform this by mentally recognizing the number in each image, or alternatively by writing or entering the number in a general-purpose computer system. Step g requires feeding the sample image into the neural network, which is not a mental process. However, the step further requires “recognising” the number in the sample image as the number corresponding to the node outputting the highest value. A human can practically perform this by observing the number of each output node, and recognizing/identifying the plate number corresponding to said node (YES).
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application? Steps a-b and d-g recite additional elements. Steps d-g integrate step c into a practical application, as the extracted plate numbers are required to train the specific claimed neural network mode. Although the recognition in step g is not further applied in the claim, it is still performed using the specific neural network structure recited in steps d-e, and thus the claim reflects the improvements to the functioning of a computer discussed throughout the specification (YES). Claim 1 is eligible.
Similar analysis is applicable to independent claims 12 and 19. Claims 12 and 19 are eligible.
Claims 2-3 and 13-14 recite a further process of extending the specific neural network structure of the independent claims. Claims 2-3 and 13-14 are eligible.
Claims 4-11 and 15-18 recite additional elements with no new judicial exceptions. Claims 4-11 and 15-18 are eligible.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 7, 11-13, 15, and 18-19 are rejected as being unpatentable over Thomas et al. (U.S. Publ. US-2024/0185566-A1) in view of Baker (U.S. Publ. US-2023/0289611-A1).
Regarding claim 1, Thomas discloses a method for automatic license plate recognition (see figures 5-6), comprising:
recording a first set of images each including one of N1 different license plate numbers; extracting the license plate number in each image of the first set (see paragraphs 0028-0029, where a training dataset depicting vehicles with license plate numbers is obtained, and plate numbers therefrom extracted and labeled for training);
generating an artificial
training the artificial (see paragraphs 0027-0029, where the second machine learning model is trained on the images and corresponding labeled license plate numbers; alternatively, paragraph 0028 specifies that the second model can be trained to predict the geographical jurisdiction of the plate, then a third model trained to detect the plate number therein);
recording a sample image including a license plate number (see figure 6, step 602 and paragraph 0046, where a series of images are recorded of a vehicle entering and exiting a gate);
and feeding the sample image into the artificial (see figure 6, steps 604-608 and paragraph 0046, where the images are divided into a first dataset depicting the vehicle entering the gate and a second dataset depicting the vehicle exiting the gate; the models are then used to identify the plate number in each dataset for subsequent comparison).
Thomas fails to disclose the limitations indicated via
Pertaining to the same field of endeavor, Baker discloses generating an artificial neural network with one separate output node for each of the N1 different license plate numbers in the first set (see figure 13 and paragraphs 0367-0370, where a classifier neural network is built to classify data to a set of clusters/classifications; paragraph 0368 specifies that the determination of the cluster/class types can be deterministic, where each generic data example is assigned to one and only one cluster; paragraph 0370 specifies that the network can have an output node for each cluster, thus the network can have one separate output node for each data example, which can be applied to unique license plates, as Baker's disclosure relates to general-purpose machine learning models as indicated in paragraph 0003);
training the artificial neural network on the images and the extracted license plate numbers of the first set at least until for each image of the first set the output node for the license plate number included in that image outputs the highest value of all output nodes (see figure 13, step 1327 and paragraphs 0369-0370 and 0373-0374, where the model is then trained to accurately recognize the cluster assignment values from the data; paragraph 0372 specifies that each data example is classified according to output node with the highest activation value; thus, the training would result in the appropriate output node always having the highest value for the corresponding data example or plate number);
recording a sample image including a license plate number that is also included in the images the artificial neural network has been trained on; and feeding the sample image into the artificial neural network and recognising the license plate number of the sample image as the license plate number of that output node which outputs the highest value (see figure 13, step 1328 and paragraph 0371, where after training, data can be fed to the network to perform the classification/clustering, which can include old and/or new data such as plate numbers).
Thomas and Baker are considered analogous art, as they are both directed to neural networks for object classification. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Baker into Thomas by applying Baker’s clustering and training methods to Thomas’s plate detection model because doing so allows for generating a model with a desired amount of object classification types (see Baker paragraphs 0375-0377) that avoids overfitting by extending the amount of output nodes to accommodate each class (see Baker paragraphs 0379-0382).
Regarding claim 2, Thomas discloses deleting the images of the first set (see paragraph 0020 and 0043-0044, where, as the system operates to capture images of different vehicles over time, previous images would naturally be deleted from devices such as a cache or memory; note that the claim does not specify when or where from the images are deleted);
and recording a second set of images each including one of N2 different license plate numbers; extracting the license plate number of each image of the second set (see paragraphs 0020-0021, where new sets of vehicle images at the gate are naturally captured and the process repeated over time during operation).
Thomas fails to disclose the remaining limitations of claim 2.
Pertaining to the same field of endeavor, Baker discloses extending the artificial neural network by one further output node for each of the different license plate numbers that are included in the second set and not in the first set (paragraph 0377 specifies that the clustering method can be repeated in passes to modify the clusters via splitting or merging; this paragraph also specifies that the architecture can be respectively changed during the loops of steps 1327-1329 of figure 13; thus, repeated passes would introduce new clusters and corresponding output nodes for new plates);
and training the extended artificial neural network on the images and the extracted license plate numbers of the second set at least until for each image of the second set the output node for the license plate number included in that image outputs the highest value of all output nodes (see figure 13, step 1327 and paragraphs 0369-0370, 0373-0374, where the model would again be trained in the same way to accurately recognize the cluster assignment values from the data);
wherein said steps of feeding and recognising are carried out with the extended artificial neural network (paragraph 0367 specifies that the model generated during the training process of figure 13 is naturally applied to other applications, such as those in Baker's disclosure, or to the discussed license plate application).
Thomas and Baker are considered analogous art, as they are both directed to neural networks for object classification. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Baker into Thomas by applying Baker’s clustering and training methods to Thomas’s plate detection model because doing so allows for generating a model with a desired amount of object classification types (see Baker paragraphs 0375-0377) that avoids overfitting by extending the amount of output nodes to accommodate each class (see Baker paragraphs 0379-0382).
Regarding claim 4, Thomas fails to disclose the limitations of claim 4.
Pertaining to the same field of endeavor, Baker discloses wherein a mapping between the output nodes and the corresponding license plate numbers is stored in a mapping table (see paragraphs 0368-0370, where each data example or plate can be deterministically assigned to one cluster, each cluster corresponding to one output node).
Thomas and Baker are considered analogous art, as they are both directed to neural networks for object classification. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Baker into Thomas by applying Baker’s clustering and training methods to Thomas’s plate detection model because doing so allows for generating a model with a desired amount of object classification types (see Baker paragraphs 0375-0377) that avoids overfitting by extending the amount of output nodes to accommodate each class (see Baker paragraphs 0379-0382).
Regarding claim 7, Thomas in view of Baker discloses wherein the recorded images are pre-processed by at least one of resizing, converting to grayscale, blur filtering, rotating, cropping to an outer boundary of the license plate, and/or image sharpening (see Thomas paragraph 0023, where the first model can first output a bounding box of the license plate, which is considered analogous to cropping the license plate for the second model's subsequent recognition).
Regarding claim 11, Thomas fails to disclose the limitations of claim 11.
Pertaining to the same field of endeavor, Baker discloses wherein the artificial neural network is a convolutional neural network (paragraphs 0100 and 0271 specify that the neural networks described throughout the disclosure can be convolutional neural networks).
Thomas and Baker are considered analogous art, as they are both directed to neural networks for object classification. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Baker into Thomas by using convolutional neural networks because doing so allows for two-dimensional image classification (see Baker paragraph 0271).
Regarding claim 12, Thomas discloses a system for automatic license plate recognition (see figures 4-5), comprising:
a camera device configured to record a first set of images each including one of N1 different license plate numbers (see figure 5, cameras 112 and paragraph 0045);
and a computing device configured to (see figure 4, computer system 400 and paragraphs 0041-0044).
The remainder of claim 12 recites steps identical to those of claim 1. Therefore, Thomas in view of Baker discloses claim 12 as applied to claim 1 above.
Regarding claim 13, Thomas in view of Baker discloses claim 13 as applied to claim 2 above.
Regarding claim 15, Thomas in view of Baker discloses claim 15 as applied to claim 4 above.
Regarding claim 18, Thomas in view of Baker discloses claim 18 as applied to claim 7 above.
Regarding claim 19, Thomas discloses a computing device configured to (see figure 4, computer system 400 and paragraphs 0041-0044).
The remainder of claim 19 recites steps identical to those of claim 1. Therefore, Thomas in view of Baker discloses claim 19 as applied to claim 1 above.
Claims 3 and 14 are rejected as being unpatentable over Thomas et al. (U.S. Publ. US-2024/0185566-A1) in view of Baker (U.S. Publ. US-2023/0289611-A1), and further in view of Leo et al. ("Incremental Deep Neural Network Learning using Classification Confidence Thresholding", IEEE Transactions on Neural Networks and Learning Systems paper, 21 June 2021).
Regarding claim 3, Thomas in view of Baker discloses wherein the images of the second set are fed into the artificial neural network (see Thomas paragraphs 0020-0021, where new sets of vehicle images at the gate are naturally captured and the process repeated over time during operation).
Thomas in view of Baker fails to disclose the remaining limitations of claim 3.
Pertaining to the same field of endeavor, Leo discloses the different license plate numbers that are included in the second set and not in the first set are determined as the different license plate numbers included in those images of the second set for which all of the output nodes output a respective value below a predetermined threshold value (see pages 5-6, Section G. "Training the Network", and Algorithm 2, where an object belonging to a new class is identified based on a low classification confidence of a softmax layer that represents all output nodes; see especially lines 7-13 of algorithm 2, where if the highest confidence output, "p", is not higher than the average of all other outputs times a threshold factor "c", then the object is determined to belong to a new class).
Thomas and Leo are considered analogous art, as they are both directed to neural networks for object classification. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Leo into Thomas and Baker by using Leo’s method of identifying new object classes because doing so allows for adding new output nodes to represent new object classes (see Leo page 2, section A, "Overview").
Regarding claim 14, Thomas in view of Baker and Leo discloses claim 14 as applied to claim 3 above.
Claims 5-6 and 16-17 are rejected as being unpatentable over Thomas et al. (U.S. Publ. US-2024/0185566-A1) in view of Baker (U.S. Publ. US-2023/0289611-A1), and further in view of Wilson et al. (U.S. Publ. US-2023/0049167-A1).
Regarding claim 5, Thomas in view of Baker fails to disclose the limitations of claim 5.
Pertaining to the same field of endeavor, Wilson discloses wherein said step of extracting the license plate number comprises OCR reading the recorded images of the first set of images character by character (see figure 3, step 306 and paragraph 0075, where OCR can be used to identify and annotate ground-truth text from document images).
Thomas and Wilson are considered analogous art, as they are both directed to neural networks for feature recognition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Wilson into Thomas and Baker by using OCR to extract training data because doing so allows for supervised training of machine learning models for vision tasks such as text recognition (see Wilson paragraph 0075).
Regarding claim 6, Thomas in view of Baker fails to disclose the limitations of claim 6.
Pertaining to the same field of endeavor, Wilson discloses wherein said step of extracting the license plate number comprises OCR reading the recorded images of the second set of images character by character (see figure 3, step 306 and paragraph 0075, where OCR can be used to identify and annotate ground-truth text from document images).
Thomas and Wilson are considered analogous art, as they are both directed to neural networks for feature recognition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Wilson into Thomas and Baker by using OCR to extract training data because doing so allows for supervised training of machine learning models for vision tasks such as text recognition (see Wilson paragraph 0075).
Regarding claim 16, Thomas in view of Baker and Wilson discloses claim 16 as applied to claim 5 above.
Regarding claim 17, Thomas in view of Baker and Wilson discloses claim 17 as applied to claim 6 above.
Claims 8-10 are rejected as being unpatentable over Thomas et al. (U.S. Publ. US-2024/0185566-A1) in view of Baker (U.S. Publ. US-2023/0289611-A1), and further in view of Peng et al. (U.S. Publ. US-2023/0177821-A1).
Regarding claim 8, Thomas in view of Baker fails to disclose the limitations of claim 8.
Pertaining to the same field of endeavor, Peng discloses wherein, in said step of training, each image of the first set is fed into the artificial neural network P times, P being in a range from 2 to 50 (see paragraph 0122, where a text recognition neural network model can be trained for an arbitrary amount of epochs, or until the model converges to a desired accuracy; thus, the amount/range of training epochs is interpreted as a result-effective variable that determines or effects the model accuracy to an arbitrary, desired degree; see MPEP 2144.05.II.B).
Thomas and Peng are considered analogous art, as they are both directed to neural networks for feature recognition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Peng into Thomas and Baker by using a training epoch amount in the range of 2 to 50 because tuning the amount of training epochs to obtain a desired model accuracy is a common and well-known practice in the field of deep learning.
Regarding claim 9, Thomas in view of Baker fails to disclose the limitations of claim 9.
Pertaining to the same field of endeavor, Peng discloses wherein, in said step of training, each image of the first set is fed into the artificial neural network P times, P being in a range from 5 to 20 (see paragraph 0122; similar analysis applies from claim 8 above).
Thomas and Peng are considered analogous art, as they are both directed to neural networks for feature recognition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Peng into Thomas and Baker by using a training epoch amount in the range of 5 to 20 because tuning the amount of training epochs to obtain a desired model accuracy is a common and well-known practice in the field of deep learning.
Regarding claim 10, Thomas in view of Baker fails to disclose the limitations of claim 10.
Pertaining to the same field of endeavor, Peng discloses wherein, in said step of training, each image of the first set is fed into the artificial neural network P times, P being in a range from 7 to 13 (see paragraph 0122; similar analysis applies from claim 8 above).
Thomas and Peng are considered analogous art, as they are both directed to neural networks for feature recognition. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Peng into Thomas and Baker by using a training epoch amount in the range of 7 to 13 because tuning the amount of training epochs to obtain a desired model accuracy is a common and well-known practice in the field of deep learning.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS JOHN HELCO whose telephone number is (703)756-5539. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella, can be reached at telephone number 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/NICHOLAS JOHN HELCO/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights