18Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy filed on 16 March, 2024.
Response to Amendment
The amendment filed 20 April, 2026 has been entered.
The amendment of claims 1, 4 – 11, and 14 – 20 has been acknowledged.
The cancellation of claims 2, 3, 12, and 13 has been acknowledged.
Response to Arguments
Applicant’s arguments, see page 10, filed 20 April, 2026 with respect to the objection of claims 1, 9, 11, 19, and 20 have been fully considered and are persuasive. The objection of claims 1, 9, 11, 19, and 20 have been withdrawn.
Applicant’s arguments, see page 11, filed 20 April, 2026 with respect to the rejection of claims 4 – 8 and 14 – 18 under 35 U.S.C. § 112(b) have been fully considered and are persuasive. The rejections of claims 4 – 8 and 14 – 18 under 35 U.S.C. § 112(b) have been withdrawn. However, under further examination a new rejection under 35 U.S.C. 112(b) is made for claims 4, 6, 14, 16.
Applicant’s arguments, see page 11, filed 20 April, 2026 with respect to the rejection of claims 1 – 3, 5 – 7, 9, 11 – 13, 15 – 17, 19, and 20 under 35 U.S.C. § 101 have been fully considered and are persuasive. The rejection of claims 1 – 3, 5 – 7, 9, 11 – 13, 15 – 17, 19, and 20 under 35 U.S.C. § 101 has been withdrawn.
Applicant’s arguments, see page 12, filed 20 April, 2026 with respect to the rejection of claims 1 – 3, 9, 11 – 13, 19, and 20 under 35 U.S.C. § 102 have been fully considered and are persuasive. The rejection of claims 1 – 3, 9, 11 – 13, 19, and 20 under 35 U.S.C. § 102 has been withdrawn.
Claim Objections
Claim 1 and 11 are objected to because of the following informalities:
Lines 7 – 8 states “where K is an integer larger than or equal to one”, and lines 16 – 17 states “where L is an integer larger than or equal to one and smaller than K”. In the event K is equal to 1, L is not able to be equal to 0, nor is it able to be equal to K which is 1. This should be amended to correct this issue.
Claim 6 is objected to because of the following informalities:
Line 5 states “the type including photograph”, the examiner believes this should say “the type including the photograph” for clarity.
Line 6 states “among L references values”, this should be amended to “among L reference values”.
Claim 7 and 17 is objected to because of the following informalities:
Line 4 states “including at least illustration or photograph”, the examiner believes this should say “including at least an illustration or photograph” for clarity.
Appropriate correction is required.
Claim 9 and 19 are objected to because of the following informalities:
Lines 6 – 7 states “where T is an integer larger than or equal to one”, and lines 15 - 16 states “where U is an integer larger than or equal to one and smaller than T”. In the event T is equal to 1, U is not able to be equal to 0, nor is it able to be equal to T which is 1. This should be amended to correct this issue.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 4, 7, 11, 14, 17, and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the histogram" in line 22 - 23. There is insufficient antecedent basis for this limitation in the claim. Line 22 – 23 states “by using the histogram”, however, the previous claim limitation states “generating a histogram for each color component…”. The applicant’s specification filed 30 January, 2024, ¶ 0061 states “In this embodiment, three histograms of the RGB color components are generated.”. “the histogram” does not clearly recite which of the plurality of histograms is intended to be referred to for calculating the peak value of each color component and instead only refers to a single histogram.
Claims 11 and 20 are rejected for the same rationale.
Claim 1 recites the limitation “generating a histogram for each color component” in line 20, however, “each color component” is not explicitly clear in what it refers to. Is this each color component of the entire first captured image? Each color component of the object region? ¶ 0061 of the applicant’s specification filed 30 January, 2024 states “ In S210, the processor 210 generates a histogram for each color component by using the object region image data of the target object region.”. The examiner believes the claim should be amended to utilize this information and make clear the histograms of each color component are explicitly for the object region image data of the target object region.
Claims 11 and 20 are rejected for the same rationale.
Claims 4 and 14 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 4 and 14, the term “a value indicating the target object image” and “a value indicating a reference object image” lack description in the specification. As written in the claim, under broadest reasonable interpretation, “a value indicating the target object image” and “a value indicating a reference object image” the value describes the images themselves, such as a collection of images comprising image 1, image 2, image 3, …, image X. ¶ 0077 – 0083 appear to describe the corresponding processing of these claims, however, ¶ 0077 states “the processor 310 calculates a difference Vd between the image of the target object region (referred to as the target object image) and the reference image corresponding to the target object region… The difference Vd may be various values indicating the difference between the target object image and the reference image… The processor 310 calculates, for each pixel, three absolute values of the three differences of the three RGB component values between the target object image and the reference image.”. This clearly describes a process of calculating values for each pixel within a target object image, and comparing those to the corresponding pixels of the reference image. This process is distinctly different than the claim limitation of “where it is determined that the visual is normal when a difference between a value indicating the target object image and a value indicating a reference object image is smaller than a reference value”. One who is skilled in the art could not reasonable review this claim limitation and come to the conclusion that it pertains to the pixel based mathematical operations as described in the specification.
Regarding claims 7 and 17, lines 4 and 6 states “a type including at least illustration or photograph”. Previously in claim 6 and 16, there is already “a type including a photograph” which is identified by having the largest reference value. It is unclear if the type in claim 7 is intended to be the same as the type of claim 6 as both types are noted to include a photograph. These types do not appear to be distinct from one another beyond the type of claim 7 possibly including an illustration, however, because the type of claim 7 is at least illustration or photograph, this type could merely comprises a photograph which is understood to be the type declared in claim 6. It should be made explicitly clear in the claim if these types are meant to be the same or different.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 9, 11, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Arroyo et al (U.S. Patent Publication No. 2022/0114821 A1, hereinafter “Arroyo”) in view of Hayasaki et al (U.S. Patent Publication No. 2015/0326752 A1, hereinafter “Hayasaki”).
Regarding claim 1, Arroyo teaches a non-transitory computer-readable storage medium storing a set of program instructions for a computer that generates data for inspecting visual of an inspection target, the set of program instructions, when executed by a controller of the computer (¶ 0046: The program(s) may be embodied in software stored on one or more non-transitory computer readable storage media… associated with processor circuitry located in one or more hardware devices, but the entire program and/or parts thereof could alternatively be executed by one or more hardware devices other than the processor circuitry and/or embodied in firmware or dedicated hardware), causing the computer to perform:
based on first captured image data indicating a first captured image of a first inspection target (¶ 0025: As shown in FIG. 1, the system 100 may include a first image 102A, a second image 102B, and/or and third image 102C for categorization.), detecting K object regions corresponding to K objects by using a trained object detection model, where K is an integer larger than or equal to one (Figure 3; ¶ 0023: For example, an R-CNN may be employed to determine regions of text likely to be a description of an associated product in a banner image.; ¶ 0032: Generally speaking, examples disclosed herein detect regions related to textual descriptions of each product in a banner image, then recognize associated banner text as distinguished from any other text that might reside on the product images themselves, and further classifies the recognized text into different product categories, as described in further detail below), the first inspection target including the K objects and having no visual abnormalities (¶ 0026: As shown by the system 100, an example categorization circuitry 110 may be used to perform the categorization of the images 102A-102C including respective portions of text 106A-106C.);
generating first correspondence data indicating K correspondences corresponding to respective ones of the K object regions (¶ 0054: The example region detection model training circuitry 202 generates bounding boxes around detected regions (block 908)), each of the K correspondences indicating a correspondence between object region information and condition information (¶ 0039: In particular, the example category identification circuitry 224 classifies the text into one or more categories.), the object region information being information specifying an object region in the first captured image of the first inspection target (¶ 0054: The example region detection model training circuitry 202 generates bounding boxes around detected regions (block 908); Examiner’s note: A bounding box is known to acquire and store the coordinate information of the bounding box.), the condition information indicating an inspection condition associated with a type of the object region, the inspection condition being among L inspection conditions, where L is an integer larger than or equal to one and smaller than or equal to K (¶ 0040: The trained model is used to perform inference of the categories related to each product description, and the inference output gives a vector with probabilities for each available category of interest.; ¶ 0054: The example text classification circuitry 114 applies the trained text classification model (block 916) for the purpose of classifying text into one or more categories (block 918).); Examiner’s note: Regarding the L inspection conditions, as this invention applies one inspection condition to each object, it is understood that the number of inspection conditions which are applied to the plurality of objects cannot be greater than the number of objects. Therefore, any art which teaches labelling a detected object with a single inspection condition would read on the limitation of “L is an integer larger than or equal to one and smaller than or equal to K”. For each object, there must be at least one inspection condition, however multiple objects could be labelled with the same inspection condition, therefore L must be less than or equal to K, but greater than or equal to 1.), wherein the generating the first correspondence data includes determining the type of the object region, (¶ 0027: FIG. 2 illustrated the example text classification circuitry 114 as shown in FIG. 1. In some examples, the text classification circuitry 114 includes an example region detection model training circuitry 202 for detecting one or more regions of an image 102A-102C related to text or textual descriptions of each product… For example, in the examples disclosed herein, GT (Ground Truth) information is employed about generated bounding boxes and associated classes of information to teach one or more neural networks to localize and classify objects of interest. In particular, once bounding boxes are localized in a particular image, the R-CNN classifies the bounding boxes based on the GT information provided.), the determining the type of the object region including:
storing the first correspondence data in a memory (¶ 0054: The example region detection model training circuitry 202 generates bounding boxes around detected regions (block 908)…; ¶ 0039: In particular, the example category identification circuitry 224 classifies the text into one or more categories.).
Arroyo does not explicitly teach generating a histogram for each color component by using image data of the object region; calculating a peak value of each color component of the object region by using the histogram; determining whether the peak value of each color component of the object region is smaller than a threshold; and in response to determining that the peak value is smaller than the threshold, determining that the object region is a particular image type.
However, Hayasaki does teach generating a histogram for each color component by using image data of the object region (Figure 16; ¶ 0134: With respect to each of the partial images into which the dividing process section 122 has divided output target image data, the histogram creating section 123 creates a histogram representing a distribution of the number of pixels for density values (pixel values) with respect to each of color channels constituting the partial image.);
calculating a peak value of each color component of the object region by using the histogram (¶ 0137: The adjustment value calculating section 124 calculates a temporary adjustment value based on a peak closest to the maximum value of the pixel values in the histogram of each color channel which histogram has been created for each partial image; ¶ 0138: Specifically, the adjustment value calculating section 124 specifies, in a histogram, a pixel value maxX and the number of pixels max V at the peak closest to the maximum value (255 here) of the pixel values (peak appearing first in a direction from the maximum value to the minimum value of the pixel values));
determining whether the peak value of each color component of the object region is smaller than a threshold (¶ 0144: For at least one color channel, a difference between maxX and XX is not less than a predetermined threshold thX (e.g. thX=30).; Examiner’s note: The difference between maxX and XX is still the value of the peak pixel maxX after an adjustment value is applied. This does not change the peak value maxX from continuing to be the largest value pixel and therefore is the peak value compared to a threshold.); and
in response to determining that the peak value is smaller than the threshold, determining that the object region is a particular image type (¶ ¶ 0145: The partial image meeting the condition A indicates that, in the histogram, a peak closest to the maximum value of the pixel values is broad. This indicates that the partial image has no or little amount of a page background region, as in cases of a photographic region or a drawing.)
Arroyo and Hayasaki are considered to be analogous art as both pertain to image region anaylsis. Therefore, it would have been obvious to one of ordinary skill in the art to combine the systems for categorizing image text (as taught by Arroyo) and the image processing apparatus (as taught by Hayasaki) before the effective filing date of the claimed invention. The motivation for this combination of references would be the method of Hayasaki does not need to cut out image data from the target part. This prevents the reduction of the exactness of correcting color unevenness in the image. (See ¶ 0005).
This motivation for the combination of Arroyo and Hayasaki is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 9, the Arroyo and Hayasaki combination teaches the non-transitory computer-readable storage medium according to claim 1.
Additionally, Arroyo teaches wherein the set of program instructions, when executed by the controller, causing the computer to perform:
based on second captured image data indicating a second captured image of a second inspection target (¶ 0025: As shown in FIG. 1, the system 100 may include a first image 102A, a second image 102B, and/or and third image 102C for categorization. In some examples, the images 102A, 102B, and/or 102C are banner images, whereas in other examples the images 102A, 102B, and/or 102C are a different type of image. In some examples, each image 102A-102C includes one or more product representations ( e.g., an image, a graphic depicting the product, etc.), respectively.; Examiner’s note: Arroyo teaches a process of inputting multiple images to perform the text detection and classification, therefore the limitations of claim 9 are the same as the independent claim, only described on subsequent images as disclosed in Arroyo ¶ 0025), detecting T object regions corresponding to T objects by using the trained object detection model, where T is an integer larger than or equal to one (¶ 0023: For example, an R-CNN may be employed to determine regions of text likely to be a description of an associated product in a banner image.), the second inspection target including the T objects and having no visual abnormalities (¶ 0026: As shown by the system 100, an example categorization circuitry 110 may be used to perform the categorization of the images 102A-102C including respective portions of text 106A-106C.);
generating second correspondence data indicating T correspondences corresponding to respective ones of the T object regions (¶ 0054: The example region detection model training circuitry 202 generates bounding boxes around detected regions (block 908)), each of the T correspondences indicating a correspondence between object region information and condition information (¶ 0039: In particular, the example category identification circuitry 224 classifies the text into one or more categories.), the object region information being information specifying an object region in the second captured image of the second inspection target (¶ 0054: The example region detection model training circuitry 202 generates bounding boxes around detected regions (block 908); Examiner’s note: A bounding box is known to acquire and store the coordinate information of the bounding box.), the condition information indicating a reference value associated with a type of the object region, the reference value being amount U reference values, where U is an integer larger than or equal to one and smaller than T. (¶ 0040: The trained model is used to perform inference of the categories related to each product description, and the inference output gives a vector with probabilities for each available category of interest.; ¶ 0054: The example text classification circuitry 114 applies the trained text classification model (block 916) for the purpose of classifying text into one or more categories (block 918).)); and
storing the second correspondence data in the memory (¶ 0054: The example region detection model training circuitry 202 generates bounding boxes around detected regions (block 908)…; ¶ 0039: In particular, the example category identification circuitry 224 classifies the text into one or more categories.).
Regarding claim 11, claim 11 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Arroyo’s further teaching on:
A controller (¶ 0046: The program(s) may be embodied in software stored on one or more non-transitory computer readable storage media… associated with processor circuitry located in one or more hardware devices, but the entire program and/or parts thereof could alternatively be executed by one or more hardware devices other than the processor circuitry and/or embodied in firmware or dedicated hardware); and
A memory storing instructions, the instructions, when executed by a controller, causing the generation apparatus to perform (¶ 0046: The program(s) may be embodied in software stored on one or more non-transitory computer readable storage media… associated with processor circuitry located in one or more hardware devices, but the entire program and/or parts thereof could alternatively be executed by one or more hardware devices other than the processor circuitry and/or embodied in firmware or dedicated hardware):
Regarding claim 19, claim 19 has been analyzed with regard to respective claim 9 and is rejected for the same reasons of obviousness as used above.
Regarding claim 20, claim 20 has been analyzed with regard to respective claim 1 and is rejected for the same reasons of obviousness as used above.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Arroyo et al (U.S. Patent Publication No. 2022/0114821 A1, hereinafter “Arroyo”) in view of Hayasaki et al (U.S. Patent Publication No. 2015/0326752 A1, hereinafter “Hayasaki”) and further in view of Okazaki et al U.S. Patent Publication No. 2024/0161271 A1, hereinafter “Okazaki”).
Regarding claim 4, the Arroyo and Hayasaki combination teaches the non-transitory computer-readable storage medium according to claim 1.
Arroyo does not explicitly teach wherein the integer L is larger than or equal to two; wherein each of the L inspection conditions is a condition for determining that a difference indicates that visual of an object represented by a target object image is normal, the difference being a difference between the target object image and a reference object, the target object image being an image of a region indicated by the object region information in a captured image for inspection, the reference object image being an image of an object that is preliminary associated with the object region information and that has no abnormality; and wherein the L inspection conditions includes a plurality of inspection conditions indicating different criteria for determining that the difference indicates that the visual is normal.
However, Okazaki teaches wherein the integer L is larger than or equal to two (¶ 0068: The abnormality score map is a diagram in which the score of the abnormality degree corresponding to the magnitude of difference between the image 210 and the reconstructed image 230 is indicated by, for example, color, brightness, density, or the like in units of pixels.);
wherein each of the L inspection conditions is a condition for determining that a difference indicates that visual of an object represented by a target object image is normal, where it is determined that the visual is normal when a difference between a value indicated the target object image and a value indicating a reference object image is smaller than a reference value (¶ 0068: The calculation section 113 may calculate, as an abnormality score map indicating the abnormality degree, the difference in the inspection target region 212 between the image 210 and the reconstructed image 230… The score of the abnormality degree may be a proportion ( e.g., 0.3 or the like) of the difference for each pixel when the maximum value of the difference is set to 1.), the target object image being an image of a region indicated by the object region information in a captured image for inspection (¶ 0068: The difference in the inspection target region 212 between the image 210 and the reconstructed image 230 may be a pixel-based difference), the reference object image being an image of an object that is preliminary associated with the object region information and that has no abnormality (Figure 7; ¶ 0054: The identification section 112 can identify the inspection target region 212 in the image 210 as follows. The identification section 112 uses a template image 240 of the inspection target region 212 of the input normal product. Then, the identification section 112 can identify the inspection target region 212 by template matching between the template image 240 and the image 210. The template image 240 constitutes a predetermined reference image.); and
wherein the L inspection conditions includes a plurality of inspection conditions indicating different reference values (¶ 0068: The calculation section 113 may calculate, as an abnormality score map indicating the abnormality degree, the difference in the inspection target region 212 between the image 210 and the reconstructed image 230.The abnormality score map is a diagram in which the score of the abnormality degree corresponding to the magnitude of the difference between the image 210 and the reconstructed image 230 is indicated by, for example, color, brightness, density, or the like in units of pixels. In the abnormality score map, a portion where the abnormality degree of the target 220 is high can be emphasized. The score of the abnormality degree may be the magnitude itself of the difference between the image 210 and the recon structed image 230 (e.g., an absolute value difference between pixel values).; ¶ 0144: Condition A; ¶ 0146: Condition B; ¶ 0148: Condition C.).
Arroyo and Okazaki are considered to be analogous art as both pertain to image region detection and processing. Therefore, it would have been obvious to one of ordinary skill in the art to combine the method of categorizing image text (as taught by Arroyo) and the information processing apparatus (as taught by Okazaki) before the effective filing date of the claimed invention. The motivation for this combination of references would be the method of Okazaki only compares differences for the image region which is detected within images, thus, if areas of the image outside the detected region are included in the image or the appearance of the object varies because of the nature of the object, erroneous detection of abnormality is reduced. (See ¶ 0082).
This motivation for the combination of Arroyo, Hayasaki, and Okazaki is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Allowable Subject Matter
Claims 5, 6, 8, 10, 15, 16, and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW JONES whose telephone number is (703)756-4573. The examiner can normally be reached Monday - Friday 8:00-5:00 EST, off Every Other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW B. JONES/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights