DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on February is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is considered by examiner.
Examiner note regarding Drawings
It is the examiner’s opinion any form of photograph image shown in Figure 5 is considered necessary and “are the only practicable medium for illustrating the claimed invention” per 37 CFR 1.84(b)(1) because the invention pertains to “using [a] diffusion model to generate synthetic images” (specification ¶ [0001]). Figure 5 is representative of the invention and demonstrates “example license plate images that represent different layout designs under different lighting” (¶ [0112]), which may not be captured with sufficient detail in a line drawing to demonstrate the applicant’s invention. Therefore, no drawing objection is raised.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, 10, 11, 17, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kohler et al (US 2025/00785538) in view of Gupta et al (US 2023/0343114).
Regarding Claim 1, Kohler et al teach a method for improving vehicle identification accuracy in a vehicle management system (method using system 100 for automated license plate recognition, including learning models stored and executed on edge devices 102 and server 104; Fig 1A, 3-5A, 7 and ¶ [0048], [0058]), comprising:
parsing, by a first language model (text-adapted (first language) vision transformer 310 within the edge device 102; Fig 3-5A and ¶ [0191]-[0192], [0220]-[0221]), a guidance prompt to extract elements of a license plate (the text-adapted vision transformer 310 may make an inaccurate prediction of a license plate aesthetic or character prediction 505 (inaccurate prediction serving as the guidance prompt); Fig 3-5A and ¶ [0218]-[0221]),
wherein the guidance prompt specifies an instruction for generating a synthetic license plate image (the inaccurate prediction, represented as low-accuracy plate features 505, serves as the guidance prompt for generating the non-real license plate features; Fig 3-5A and ¶ [0221]-[0222]);
applying a pre-trained layout generation model (transformer encoder 406 of vision transformer 310; Fig 3-5A and ¶ [0200]) to the elements of the license plate to output a layout of the license plate (the vision transformer 310 uses positional vector 404 data of image patches 400 in a transformer encoder 406 to determine character and position (layout) data of the license plate; Fig 3-5A and ¶ [0199]-[0203]),
wherein the pre-trained layout generation model is trained using real license plate images (the vision transformer 310, including encoder 406, is pretrained with real license plate-text pairs 326 of real-life license plate images 124; Fig 3-5A and ¶ [0181]-[0183], [0213]);
generating, by a second language model (random plate number (second language) generator 328; Fig 3-5A and ¶ [0185]), a set of condition embeddings based on the layout of the license plate (random plate number generator 328 generates non-real license plate number based on constraints associated with the low-accuracy plate features 505 (set of conditions) of the license plate features 501 (layout embeddings represented as positional vectors 404 of image patches 400, such as plate aesthetics or configuration (¶ [0152]-[0155]); Fig 3-5A and ¶ [0219]-[0224]);
generating a synthetic license plate image by applying a generative model (latent diffusion model 318; Fig 3, 5A, 7 and ¶ [0185], [0225]) conditioned on the set of condition embeddings (the latent diffusion model 318 generates a non-real (synthetic) license plate based on a number and plate features representing erroneous or inaccurate license plate recognition/predictions made by vision transformer 310 from previous iterations; Fig 3-5A, 7 and ¶ [0185]-[0187], [0225]-[0227]); and
training a license plate identification model using the synthetic license plate image (the vision transformer 310 is continually trained and may include training based on the non-real (synthetic) license plate image datasets; Fig 3-5A and ¶ [0227], [0232]-[0233]).
Kohler et al does not explicitly teach the real license plate images are labeled with bounding boxes for each of the elements. Examiner notes Kohler et al teaches the use of bounding boxes for identifying objects in the field of view using deep learning model(s) (shown in Fig 2C based on object detection and classification ¶ [0058], [0143]-[0144]).
Gupta et al is analogous art pertinent to the technological problem addressed in the current application and teach real license plate images are labeled with bounding boxes for each of the elements (a license plate 12 is identified in the cropped region 16 of a real image, with individual rectangle bounding boxes 18 for the license plate characters, used to predict the license plate by the character recognition algorithm, which is similarly trained with real images; Fig 1, 3, 4 and ¶ [0026]-[0030], [0040], [0053]-[0057]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Kohler et al with Gupta et al including each of the license plates images are labeled with bounding boxes for each of the elements. By using individual bounding boxes for each character, the characters may be analyzed over multiple frames for positioning to determine one-to-one correspondence and determine a confidence value for each character and position to correctly identify the character individually and string the characters together for a frame with the greatest confidence, thereby resulting in improved character detection accuracy, as recognized by Gupta et al (¶ [0004], [0029]-[0030]).
Regarding Claim 7, Kohler et al in view of Gupta et al teach the method of claim 1 (as described above), wherein the set of condition embeddings (Kohler et al, layout embeddings represented as positional vectors 404 of image patches 400, such as plate aesthetics or configuration (¶ [0152]-[0155]); Fig 3-5A and ¶ [0219]-[0224]) includes: a layout embedding corresponding to the layout of the license plate (positional vectors 404 of image patches 400 each represent an embedding corresponding to the license plate layout; Fig 4 and ¶ [0152]-[0152]-[0155]), and a mask embedding corresponding to one or more regions of the license plate that are to be masked and not to be modified by the generative model (the license plate recognition engine 304 identifies the license plate patch image data 400 based on the vector 404 to determine the regions (outside the license plate are therefore interpreted as masked embeddings) to not use to generate the synthetic license plate; ¶ [0152]-[0156], [0185]).
Regarding Claim 10, Kohler et al in view of Gupta et al teach the method of claim 1 (as described above), further comprising: generating random values for the elements of the license plate (Kohler et al, randomly selecting random characters and configurations for low-accuracy features 505may be selected and linked with an integer between 1 and n (number of characters), used to generate a random value to select random characters in the randomly selected license plate configuration; Fig 5B and ¶ [0228]-[0230]); and generating the guidance prompt based on the random values of the elements of the license plate (Kohler et al, the guidance prompt is based on the inaccurate predictions, represented as low-accuracy plate features 505, for generating non-real (synthetic) plate features, determined based on recognition accuracy; Fig 5A and ¶ [0231]-[0233]).
Regarding Claim 11, Kohler et al teach a non-transitory computer-readable medium (memory of control unit 112 in system 100; Fig 1A and ¶ [0048]-[0050], [0058]) comprising memory with instructions encoded (server 104 memory stores instructions; Fig 1A and ¶ [0052]) thereon, the instructions comprising instructions to cause one or more processors to perform steps (instructions for control unit 112 processor to execute method for automated license plate recognition, including learning models stored and executed on edge devices 102 and server 104; Fig 1A, 3-5A, 7 and ¶ [0052], [0058]) comprising: steps identical to claim 1 (as described above).
Regarding Claim 17, Kohler et al in view of Gupta et al teach the non-transitory computer-readable medium of claim 11 (as described above), including further limitations identical to claim 7 (as described above).
Regarding Claim 20, Kohler et al teach a system (system 100; Fig 1A and ¶ [0048]-[0050], [0058]) comprising: memory with instructions encoded thereon (server 104 memory with instructions; Fig 1A and ¶ [0052]); and one or more processors (processors of control unit 112; Fig 1A and ¶ [0058]) that, when executing the instructions, are caused to perform operations (instructions for control unit 112 processor to execute method for automated license plate recognition, including learning models stored and executed on edge devices 102 and server 104; Fig 1A, 3-5A, 7 and ¶ [0052], [0058]) comprising: steps identical to claim 1 (as described above).
Claims 2-6, 12-16 are rejected under 35 U.S.C. 103 as being unpatentable over Kohler et al (US 2025/00785538) in view of Gupta et al (US 2023/0343114) and Popov et al (US 2023/0153698).
Regarding Claim 2, Kohler et al in view of Gupta et al teach the method of claim 1 (as described above), including the pre-trained layout generation model (transformer encoder 406 of vision transformer 310; Fig 3-5A and ¶ [0200]).
Kohler et al in view of Gupta et al does teach the model is configured to: identify a license plate template based on the guidance prompt; and for each element of the license plate, generate a position of the element on the license plate based on the license plate template; and generate a size of the element on the license plate based on the license plate template.
Popov et al is analogous art pertinent to the technological problem addressed in the current application and teach a model (license plate build unit 308; Fig 3 and ¶ ¶ [0045], [0055]-[0056]) configured to: identify a license plate template based on the guidance prompt (the plate build unit 308 identifies a template applicable to a plate under examination; Fig 3 and ¶ [0056]); and for each element of the license plate, generate a position of the element on the license plate based on the license plate template (the specified location for a given character is identified based on the template; ¶ [0055]-[0056]); and generate a size of the element on the license plate based on the license plate template (associated data corresponding to the position is determined, including license plate size and character size (based on character scanning and extraction output, including size ¶ [0036]-[0037], [0045]); ¶ [0053]-[0056]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Kohler et al in view of Gupta et al with Popov et al including the model configured to: identify a license plate template based on the guidance prompt; and for each element of the license plate, generate a position of the element on the license plate based on the license plate template; and generate a size of the element on the license plate based on the license plate template. By using a template of the license plate, a comparison may be made to identify the characters and classify the license plate quickly as well as identify unrecognized characters, which may be used for further training the machine learning model, as recognized by Popov et al (¶ [0007], [0020], [0037]-[0038]).
Regarding Claim 3, Kohler et al in view of Gupta et al and Popov et al teach the method of claim 2 (as described above), wherein the identified license plate template includes one or more of following elements: a jurisdiction associated with the license plate, a month of registration, a year of registration, and a slogan (Popov et al, template information used by the plate build unit 308 may include plate-specific jurisdiction; Fig 3 and ¶ [0055]-[0056]); the guidance prompt includes one or more elements corresponding to the one or more elements of the identified license plate template (Popov et al, a template may indicate a specified location should include a letter rather than number character, prompting potential removal of data corresponding to such position; Fig 3 and ¶ [0056]-[0057]).
Regarding Claim 4, Kohler et al in view of Gupta et al and Popov et al teach the method of claim 3 (as described above), wherein identifying the license plate template is based on the jurisdiction specified in the guidance prompt, and the identified template is associated with the jurisdiction (Popov et al, a plate under examination, compared to an applicable template, may be identified based on specific characters in given locations (plate-specific jurisdiction), to identify the recognized jurisdiction; Fig 3 and ¶ [0055]-[0056]).
Regarding Claim 5, Kohler et al in view of Gupta et al and Popov et al teach the method of claim 2 (as described above), wherein the license plate template includes a two-dimensional coordinate corresponding to a position of an element of the license plate template (Popov et al, the character positions in the license plate template corresponds to a matrix value (matrix representing the 2D coordinate position); ¶ [0086]) and generating the position of the element on the license plate includes generating a two-dimensional coordinate corresponding to the position of the element on the license plate based on the two-dimensional coordinate corresponding to the element of the license plate template (Popov et al, the first received image corresponding to the license plate region applies rules regarding the layout/content, including identifying characters, which are compared to multiple character positions in the template based on the matrix (2D coordinate data), including accounting for scale and placement; ¶ [0084]-[0089]).
Regarding Claim 6, Kohler et al in view of Gupta et al and Popov et al teach the method of claim 2 (as described above), the license plate template includes a width and a height corresponding to an element of the license plate template (Popov et al, the license plate template accounts for scale of the license plate and characters (resolution scale such that the machine learning classifier detects shape and size variation); ¶ [0089], [0095], [0099], [0102]), and generating the size of the element on the license plate includes generating a width and a height corresponding to the size of the element on the license plate based on the width and height corresponding to the element of the license plate template (Popov et al, the first received image corresponding to the license plate region applies rules regarding the layout/content, including identifying characters, which are compared to multiple character positions in the template based on the matrix (2D coordinate data), including accounting for scale (shape and size variation) and placement; ¶ [0084]-[0089], [0095], [0099], [0102]).
Regarding Claim 12, Kohler et al in view of Gupta et al teach the non-transitory computer-readable medium of claim 11 (as described above), including further limitations identical to claim 2 (as described above).
Regarding Claim 13, Kohler et al in view of Gupta et al teach the non-transitory computer-readable medium of claim 12 (as described above), including further limitations identical to claim 3 (as described above).
Regarding Claim 14, Kohler et al in view of Gupta et al teach the non-transitory computer-readable medium of claim 13 (as described above), including further limitations identical to claim 4 (as described above).
Regarding Claim 15, Kohler et al in view of Gupta et al teach the non-transitory computer-readable medium of claim 12 (as described above), including further limitations identical to claim 5 (as described above).
Regarding Claim 16, Kohler et al in view of Gupta et al teach the non-transitory computer-readable medium of claim 12 (as described above), including further limitations identical to claim 6 (as described above).
Allowable Subject Matter
Claims 8-9, 18-19 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding Claim 8, 18, the following limitations were not identified in combination with all of the limitations in which it depends to be taught by the prior art (claim 8 recited below and claimed in parallel for the base claim and all intervening claims for claim 18):
8. The method of claim 7, further comprising applying dual attention to the guidance prompt based on the mask embedding to generate: a text embedding corresponding to words or sub-words in the guidance prompt, and a character embedding corresponding to characters in the guidance prompt.
Regarding Claim 9, the following limitations were not identified in combination with all of the limitations in which it depends to be taught by the prior art:
9. The method of claim 1, wherein the generative model is a diffusion model trained to: receive a real license plate image as input; encode the real license plate image into a vector; generating a noisy vector by applying forward diffusion to the vector to incrementally add noise, the noisy vector associated with the real license plate image with added noise; generating a denoised vector by applying reverse diffusion to the noisy vector to incrementally remove noise, the denoised vector associated with the synthetic image; and decode the denoised vector to create the synthetic license plate image.
Regarding Claim 19, the following limitations were not identified in combination with all of the limitations in which it depends to be taught by the prior art:
19. The non-transitory computer-readable medium of claim 11, the generative model is a diffusion model trained to: receive a real license plate image as input; encode the real license plate image into a vector; apply forward diffusion to the vector to incrementally add noise, generating a noisy vector associated with the real license plate image with added noise; apply reverse diffusion to the noisy vector to produce a denoised vector representing a synthetic license plate image, conditioned on the condition embeddings; and decode the denoised vector to create the synthetic license plate image.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al (License plate recognition in low quality image by using Latent Diffusion YOLOv7) teach a diffusion model for removing noise from a synthetic license plate image when the image was generated from data obtained during harsh weather conditions resulting in blur or low recognition.
Ribeiro et al (Brazillian Mercosur License Plate Detection: a Deep Learning Approach Relying on Synthetic Imagery) teach an automated license plate recognition method and system including a CNN trained with synthetically generated license plate imagery to train the model with the synthetic images generated to insert noise to mimic common shadows and visual obstructions.
Zhang et al (A Robust Attention Framework for License Plate Recognition in the Wild) teach a cycleGAN model for synthetic license plate image generation used for training data including generation of asymmetrical synthetic license plate images and using 2D attention heat maps when decoding character data of real license plate images on a trained neural network model.
Hwang et al (US 2026/0127857, application US 18/940,792), from the same co-inventors and assignee, teach a method and system for license plate vehicle identification using machine learning techniques including a diffusion model to create the synthetic license plate but does not claim all of the features of the current application including the generation of synthetic license plate data based on extraction of real license plate images labeled with bounding boxes to determine embeddings of the license plate layout, as claimed in the current application.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATHLEEN M BROUGHTON whose telephone number is (571)270-7380. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN M BROUGHTON/Primary Examiner, Art Unit 2661
Prosecution Insights