Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 15, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 3 and 13 – 15 are rejected under 35 U.S.C 103 as being unpatentable over Fukuda et al. Patent Publication No. WO-2024261931-A1 (hereinafter Fukuda) in view of Seo US Patent Application Publication No. US-20210232863-A1 (hereinafter Seo).
Regarding claim 1, Fukuda discloses an information processing system, comprising (Fukuda in [0018] discloses, “an image inspection device 200 includes a learning processing part 203”): a division unit configured to divide a captured image (Fukuda in [0033] discloses, “the region division processing unit (302)”); a correction equation creation unit configured to create a correction equation for each divided region of the divided captured image based on a feature amount expressing an image of the divided region (Fukuda in [0050] discloses about correction equation, “a correction amount reflection processing unit (702) uses the estimated kernel to execute convolution processing for each image or for each partial region, thereby correcting the diffusion of the pixel value”. Furthermore, Fukuda in [0010] discloses about feature amount, “The image quality feature amount extraction unit divides the first image into partial regions having a unit larger than a predetermined abnormal size ...The image processing device is characterized by having a feature amount extraction processing unit for extracting a feature amount vector to be processed from other partial regions of the first image”); an inference image creation unit configured to create an inference image expressed by the feature amount for each divided region according to the correction equation for each divided region (Fukuda in [0020] discloses about inference image creation, “An inference processing unit (208) converts the image quality of the second image (207) to approximate the image quality of the first image (206) by using a first image (206) which is an inspection image for which abnormality is to be discovered, ... and an image quality correction model (204) created by a learning processing unit (203), and corrects the corrected second image (209) Create”. Additionally, Fukuda in [0050] and [0010] discloses about correction equation and feature amount).
Fukuda doesn’t disclose about the following limitation as further recited in the claim.
Seo discloses a learning unit configured to execute machine learning on a learning model in which the inference image is used as training data, and the captured image is used as input data (Seo in [0073] discloses, “the corrected image corresponding to the original image are obtained, the electronic device 100 may input the obtained original image and the correction image to the neural network model to train the neural network model”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Seo into the system of Fukuda because it would allow the system can create a more accurate inspection model by generating more training images from small number of original images.
Summary of Citations (Seo)
Paragraph [0073]; “the corrected image corresponding to the original image are obtained, the electronic device 100 may input the obtained original image and the correction image to the neural network model to train the neural network model”.
Summary of Citations (Fukuda)
Paragraph [0010]; “The image quality feature amount extraction unit divides the first image into partial regions having a unit larger than a predetermined abnormal size ...The image processing device is characterized by having a feature amount extraction processing unit for extracting a feature amount vector to be processed from other partial regions of the first image”.
Paragraph [0018]; “an image inspection device 200 includes a learning processing part 203”.
Paragraph [0020]; “An inference processing unit (208) converts the image quality of the second image (207) to approximate the image quality of the first image (206) by using a first image (206) which is an inspection image for which abnormality is to be discovered, ... and an image quality correction model (204) created by a learning processing unit (203), and corrects the corrected second image (209) Create”.
Paragraph [0033]; “the region division processing unit (302)”.
Paragraph [0050]; “a correction amount reflection processing unit (702) uses the estimated kernel to execute convolution processing for each image or for each partial region, thereby correcting the diffusion of the pixel value”.
Regarding claim 2, Fukuda discloses the information processing system according to claim 1, further comprising: an inference unit (Fukuda in [0018] discloses, “an inference processing part 208”) configured to infer an inspection image obtained by image capturing by using the learning model on which the execution of the machine learning is done; and a determination unit configured to determine the inspection image based on the inference by the inference unit (Fukuda in [0021 – 002] discloses, “A comparison processing unit (210) uses the corrected second image (209) and the corrected first image (206) to calculate an image difference (211) ... From the difference (211) of the image, an abnormality determination unit (212) identifies a region having an abnormality in the first image (206) and/or the second image (207) through analysis processing including threshold processing for discovering an abnormality such as a defect” Furthermore, Fukuda in [0019] discloses about a learning model, “A learning processing unit (203) learns a machine learning model for converting the image quality of the first image (201) for learning and/or the second image (202) for learning, using the first image (201) for learning, a second image (202) for learning corresponding thereto, and a predetermined abnormality size (205) indicating the size of an abnormality such as a predetermined defect”).
Summary of Citations (Fukuda)
Paragraph [0018]; “in Fig. 2, an image inspection device 200 includes a learning processing part 203, an inference processing part 208, a comparison processing part 210, and an abnormality determination part 212”.
Paragraph [0019]; “A learning processing unit (203) learns a machine learning model for converting the image quality of the first image (201) for learning and/or the second image (202) for learning, using the first image (201) for learning, a second image (202) for learning corresponding thereto, and a predetermined abnormality size (205) indicating the size of an abnormality such as a predetermined defect”.
Paragraph [0021 – 0022]; “A comparison processing unit (210) uses the corrected second image (209) and the corrected first image (206) to calculate an image difference (211) ... From the difference (211) of the image, an abnormality determination unit (212) identifies a region having an abnormality in the first image (206) and/or the second image (207) through analysis processing including threshold processing for discovering an abnormality such as a defect”.
Regarding claim 3, Fukuda in the combination discloses the information processing system according to claim 1, wherein the learning model outputs output data as a result of recognition of the input data by the machine learning (Fukuda in [0029] discloses, “The conversion is performed by using a neural network outputting a corrected second image (209) contained in an image quality correction model (204) obtained by a learning processing unit (203)”).
Summary of Citations (Fukuda)
Paragraph [0029]; “The conversion is performed by using a neural network outputting a corrected second image (209) contained in an image quality correction model (204) obtained by a learning processing unit (203)”.
Regarding claim 13, Fukuda in the combination discloses the information processing system according to claim 1, wherein the captured image is an image for inspection of semiconductor processing or for inspection of a printing element substrate that can eject a liquid (Fukuda in [0002] discloses, “A technique to compare an inspection image with a reference image has been adopted for evaluation of semiconductor circuit defects”).
Summary of Citations (Fukuda)
Paragraph [0002]; “A technique to compare an inspection image with a reference image has been adopted for evaluation of semiconductor circuit defects”.
Regarding claim 14, claim 14 is claim 1 except for an input reception unit configured to receive input of a captured image, thus the rejection of claim 1 is incorporated herein. With respect to the addition limitation, reference Fukuda in [0033] discloses, “When learning processing is started as shown in Fig. 5, in a step S501, a first image 201 for learning, a second image 202 for learning, and a predetermined abnormal size 205 are inputted to a learning processing part 203”.
Summary of Citations (Fukuda)
Paragraph [0033]; “When learning processing is started as shown in Fig. 5, in a step S501, a first image 201 for learning, a second image 202 for learning, and a predetermined abnormal size 205 are inputted to a learning processing part 203”.
Regarding claim 15, method claim 15 corresponds to apparatus claim 1. Therefore, the
rejection analysis of claim 1 is applicable to claim 15.
Claims 4 and 6 are rejected under 35 U.S.C 103 as being unpatentable over Fukuda in view of Seo and further in view of Shimada Patent Application Publication No. WO-2023047826-A1 (hereinafter Shimada).
Regarding claim 4, Fukuda in the combination discloses the information processing system according to claim 1.
Fukuda and Seo in the combination doesn’t disclose about the following limitation as further recited in the claim.
Shimada discloses the captured image is captured by using infrared light (Shimada in [0031] discloses, “The optical filter switching unit 50 (optical filter switching mechanism) is a device for switching to different optical filters between capturing with visible light and capturing with near-infrared light”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Shimada into the system of Fukuda in view of Seo because it would allow the system to more accurately determine the difference in the inspection subject that might be difficult to distinguish in an ordinary visible light.
Summary of Citations (Shimada)
Paragraph [0031]; “The optical filter switching unit 50 (optical filter switching mechanism) is a device for switching to different optical filters between capturing with visible light and capturing with near-infrared light”.
Regarding claim 6, Fukuda in the combination discloses the information processing system according to claim 1.
Fukuda and Seo in the combination doesn’t disclose about the following limitation as further recited in the claim.
Shimada discloses the captured image is captured by using visible light (Shimada in [0031] discloses, “The optical filter switching unit 50 (optical filter switching mechanism) is a device for switching to different optical filters between capturing with visible light and capturing with near-infrared light”).
Summary of Citations (Shimada)
Paragraph [0031]; “The optical filter switching unit 50 (optical filter switching mechanism) is a device for switching to different optical filters between capturing with visible light and capturing with near-infrared light”.
Claim 5 is rejected under 35 U.S.C 103 as being unpatentable over Fukuda in view of Seo and further in view of Kang Patent Application Publication No. KR-20210016106-A (hereinafter Kang).
Regarding claim 5, Fukuda in the combination discloses the information processing system according to claim 1, wherein the correction equation (Fukuda in [0050] discloses about correction equation, “a correction amount reflection processing unit (702) uses the estimated kernel to execute convolution processing for each image or for each partial region, thereby correcting the diffusion of the pixel value”).
Fukuda and Seo doesn’t disclose about the following limitation as further recited in the claim.
Kang discloses created from three or more captured images in which a thickness, a surface roughness, and a refractive index of a subject are varied (Kang in [0017] discloses, “The measurement device 100 may irradiate the measurement object 20 with light to measure changes in intensity, phase, etc., and measure various physical properties such as asymmetry, thickness, refractive index, surface roughness, etc. of the measurement object as measurement image data”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Kang into the system of Fukuda in view of Seo because it would allow the system to learn how the inspection image changes when the subject thickness, surface roughness, and a refractive index changes, so, it can generate more realistic training images.
Summary of Citations (Kang)
Paragraph [0017]; “The measurement device 100 may irradiate the measurement object 20 with light to measure changes in intensity, phase, etc., and measure various physical properties such as asymmetry, thickness, refractive index, surface roughness, etc. of the measurement object as measurement image data”.
Claim 7 is rejected under 35 U.S.C 103 as being unpatentable over Fukuda in view of Seo and further in view of Galor Gluskin Patent Application Publication No. US-20230239553-A1 (hereinafter Micha).
Regarding claim 7, Fukuda in the combination discloses the information processing system according to claim 1.
Fukuda and Seo doesn’t disclose about the following limitation as further recited in the claim.
Micha discloses the correction equation is created from image data obtained by RGB correction (Micha in [0085] discloses, “the color correction engine 404 can compute a statistical measure (e.g., average values) for each color channel (e.g., a G channel .... an R channel average ... across all of the pixels in the image captured by the reference image sensor (e.g., pixels of the second input image 408 of FIG. 4). Similarly, the color correction engine 404 can compute a statistical measure (e.g., average values) for each color channel ... across all of the pixels in the image captured by the main image sensor (e.g., first input image 406 of FIG. 4). In such implementations, the color correction engine 404 can determine a correction ratio for each color (e.g., RGB)”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Micha into the system of Fukuda in view of Seo because it would allow the system to preserve useful color information.
Summary of Citations (Micha)
Paragraph [0085]; “the color correction engine 404 can compute a statistical measure (e.g., average values) for each color channel (e.g., a G channel average G.sub.ref, ave, an R channel average R.sub.ref, ave, and a B channel average, B.sub.ref, ave) across all of the pixels in the image captured by the reference image sensor (e.g., pixels of the second input image 408 of FIG. 4). Similarly, the color correction engine 404 can compute a statistical measure (e.g., average values) for each color channel (e.g., a G channel average G.sub.main, ave, an R channel average R.sub.main, ave, and a B channel average B.sub.main, ave) across all of the pixels in the image captured by the main image sensor (e.g., first input image 406 of FIG. 4). In such implementations, the color correction engine 404 can determine a correction ratio for each color (e.g., RGB)”.
Claim 8 is rejected under 35 U.S.C 103 as being unpatentable over Fukuda in view of Seo and further in view of Ren Patent Application Publication No. CN-106023235-A (hereinafter Ren).
Regarding claim 8, Fukuda in the combination discloses the information processing system according to claim 1.
Fukuda and Seo doesn’t disclose about the following limitation as further recited in the claim.
Ren discloses the correction equation is created from image data obtained by binarization correction (Ren in [0018] discloses, “geometric distortion correction is carried out on the crop seed image according to the calibration polynomial, the crop seed image after geometric correction is subjected to image enhancement, binaryzation, denoising and filling processing, black represents a background area, white represents crop seeds”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Ren into the system of Fukuda in view of Seo because it would allow the system have clearer separation of patterns and boundaries.
Summary of Citations (Ren)
Paragraph [0018]; “geometric distortion correction is carried out on the crop seed image according to the calibration polynomial, the crop seed image after geometric correction is subjected to image enhancement, binaryzation, denoising and filling processing, black represents a background area, white represents crop seeds”.
Claim 9 is rejected under 35 U.S.C 103 as being unpatentable over Fukuda in view of Seo and Micha and further in view of Lee US Patent Application Publication No. US-20250014169-A1 (hereinafter Lee).
Regarding claim 9, Fukuda in the combination discloses the information processing system according to claim 7.
Fukuda, Seo and Micha doesn’t disclose about the following limitation as further recited in the claim.
Lee discloses the correction equation is calculated by primary correction (Lee in [0038] discloses, “A primary correction is performed by applying the selected primary correction constant SCT to the design image DSI to thereby generate a primary corrected design image DSI-C1 (see FIG. 6) (operation S340)”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Lee into the system of Fukuda in view of Seo and Micha because it would allow the system to efficiently generate multiple corrected training image without requiring complex process.
Summary of Citations (Lee)
Paragraph [0038]; “A primary correction is performed by applying the selected primary correction constant SCT to the design image DSI to thereby generate a primary corrected design image DSI-C1 (see FIG. 6) (operation S340)”.
Claim 10 is rejected under 35 U.S.C 103 as being unpatentable over Fukuda in view of Seo and Micha and further in view of Ren.
Regarding claim 10, Fukuda in the combination discloses the information processing system according to claim 7.
Fukuda, Seo and Micha doesn’t disclose about the following limitation as further recited in the claim.
Ren discloses the correction equation is calculated by polynomial approximation correction (Ren in [0018] discloses, “geometric correction is carried out on the crop seed image, the correction process is that a geometric calibration polynomial is established in Matlab software by using a black-and-white grid intersection point as a geometric correction reference point, geometric distortion correction is carried out on the crop seed image according to the calibration polynomial”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Lee into the system of Fukuda in view of Seo and Micha because it would allow the system to perform a curved correction formula that represents complex changes in subject’s color, brightness and reflectivity more accurately.
Summary of Citations (Ren)
Paragraph [0018]; “geometric correction is carried out on the crop seed image, the correction process is that a geometric calibration polynomial is established in Matlab software by using a black-and-white grid intersection point as a geometric correction reference point, geometric distortion correction is carried out on the crop seed image according to the calibration polynomial, the crop seed image after geometric correction is subjected to image enhancement, binaryzation, denoising and filling processing, black represents a background area, white represents crop seeds”.
Claims 11 and 12 are rejected under 35 U.S.C 103 as being unpatentable over Fukuda in view of Seo and further in view of Jeong Patent Application Publication No. KR-20230112520-A (hereinafter Jeong).
Regarding claim 11, Fukuda in the combination discloses the information processing system according to claim 1.
Fukuda and Seo doesn’t disclose about the following limitation as further recited in the claim.
Jeong discloses the division unit performs the division for each pattern of a subject (Jeong in [0033] discloses, “The segmentation model implemented in gray scale may be a binary model. The segmentation model may, for example, generate a segmented pattern image with background on a pixel-by-pixel basis. The conversion module 210 may generate the second image 22 by dividing the background and the pattern image from the first image 21”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Jeong into the system of Fukuda in view of Seo because it would allow the system to create separate correction equation and corrected inference image for each pattern.
Summary of Citations (Jeong)
Paragraph [0033]; “The segmentation model implemented in gray scale may be a binary model. The segmentation model may, for example, generate a segmented pattern image with background on a pixel-by-pixel basis. The conversion module 210 may generate the second image 22 by dividing the background and the pattern image from the first image 21”.
Regarding claim 12, Fukuda in the combination discloses the information processing system according to claim 1.
Fukuda and Seo doesn’t disclose about the following limitation as further recited in the claim.
Jeong discloses the division unit performs the division into a pattern of a subject and a background (Jeong in [0033] discloses, “The segmentation model implemented in gray scale may be a binary model. The segmentation model may, for example, generate a segmented pattern image with background on a pixel-by-pixel basis. The conversion module 210 may generate the second image 22 by dividing the background and the pattern image from the first image 21”).
Summary of Citations (Jeong)
Paragraph [0033]; “The segmentation model implemented in gray scale may be a binary model. The segmentation model may, for example, generate a segmented pattern image with background on a pixel-by-pixel basis. The conversion module 210 may generate the second image 22 by dividing the background and the pattern image from the first image 21”.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to ZAID MUHAMMAD SALEH whose telephone number is (703)756-1684.
The examiner can normally be reached M-F 8 am - 5 pm ET. Examiner interviews are available
via telephone, in-person, and video conferencing using a USPTO supplied web-based
collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO
Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.If attempts to
reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be
reached on (571)272-7332. The fax phone number for the organization where this application or
proceeding is assigned is 571-273-8300. Information regarding the status of published or
unpublished applications may be obtained from Patent Center. Unpublished application
information in Patent Center is available to registered users. To file and manage patent
submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center
and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For
additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative, call 800-786
9199 (IN USA OR CANADA) or 571-272-1000.
/ZAID MUHAMMAD SALEH/
Examiner, Art Unit 2668
07/18/2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights