DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This action is in response to the Application filed on 12/07/2024.
Claims 1-16 are pending.
Priority
This Application claims priority to Korean Application KR-10-2021-0180195 with filing date 12/16/2021, PCT Application PCT/KR2022/018411 with filing date 11/21/2022, and U.S. Application 18/149,762 with filing date 01/04/2023, each of which are acknowledged.
Claim Objections
Claims 1-16 are objected to.
Regarding claims 1-16, the claims refer many times to five options for characteristics of the products, namely, “at least one of a color, a saturation, a brightness, a transparency, and a reflectance of the product” (emphasis added). The use of the word “and” as a conjunctive appears to require the selection of transparency and reflectance together, which does not appear to be Applicant’s intent. The examiner suggests changing the “and” to “or” in each recitation of this group throughout the claims.
As both independent claims 1 and 12 recite this group, this objection applies to all of claims 1-16. The following list specifies exactly where in each claim the list is recited, and thus where “and” should be changed to “or”:
Claim 1, lines 3, 7, 11,
Claim 6, lines 3, 5,
Claim 12, lines 3, 5, 11.
Regarding claim 8, a similar objection applies to the recited group of characteristics, i.e. “at least one of the color, the saturation, and the brightness (emphasis added). The examiner suggests changing “and” to “or” in both lines 2 and 3 of claim 8.
Regarding claim 11, in line 2, “detecting unit” should read “detection unit” (emphasis added).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“Sensor unit”, present in claims 1-12, with corresponding structure, material, or acts found in at least Figure 1, sensor unit 110, Figure 2, sensor unit 210, and page 4, lines 20-29 of the originally-filed specification;
“Detection unit”, present in claims 1-12, with corresponding structure, material, or acts found in at least Figure 1, detection unit 120, Figure 2, detection unit 230, convolution layers 231, fully connected layers 233, page 4, line 30 to page 5, line 3, and page 5, lines 25-26 of the originally-filed specification.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Statutory Double Patenting
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
A rejection based on double patenting of the “same invention” type finds its support in the language of 35 U.S.C. 101 which states that “whoever invents or discovers any new and useful process... may obtain a patent therefor...” (Emphasis added). Thus, the term “same invention,” in this context, means an invention drawn to identical subject matter. See Miller v. Eagle Mfg. Co., 151 U.S. 186 (1894); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Ockert, 245 F.2d 467, 114 USPQ 330 (CCPA 1957).
A statutory type (35 U.S.C. 101) double patenting rejection can be overcome by canceling or amending the claims that are directed to the same invention so they are no longer coextensive in scope. The filing of a terminal disclaimer cannot overcome a double patenting rejection based upon 35 U.S.C. 101.
Claim 1 is provisionally rejected under 35 U.S.C. 101 as claiming the same invention as that of claim 15 of copending Application No. 18/149,762 (reference application).
Regarding claim 1, the present claim 1 is identical to the reference claim 15.
This is a provisional statutory double patenting rejection since the claims directed to the same invention have not in fact been patented.
Nonstatutory Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-16 are rejected on the ground of nonstatutory double patenting as being unpatentable over the claims of U.S. Patent No. US-12087421-B2 (the ‘120 Patent), as well as the claims of U.S. Patent No. US-12670576-B2 (the ‘126 Patent).
Although the claims at issue are not identical, they are not patentably distinct from each other because the reference claim 9 of the ‘120 Patent is narrower than and fully anticipates the present claim 1, and the reference claim 6 of the ‘126 Patent is narrower than and fully anticipates the present claim 1. The remaining present claims are also rejected because the present claim 12 is merely a method claim instead of an apparatus claim, and the present dependent claim set otherwise mirrors those of the ‘120 and ‘126 Patents.
Claims 1-16 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over the claims of copending Applications No. 18/972,876 (the ‘876 Application), 18/972,880 (the ‘880 Application), and 18/972,881 (the ‘881 Application).
Although the claims at issue are not identical, they are not patentably distinct from each other because the reference claim 6 in each of the ‘876, ‘880, and ‘881 Applications are narrower than and fully anticipate the present claim 1. The remaining present claims are also rejected because the present claim 12 is merely a method claim instead of an apparatus claim, and the present dependent claim set otherwise mirrors those of the ‘876, ‘880, and ‘881 Applications.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claims 1-16 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over the claims of copending Application No. 18/972,878 (the '878 Application) in view of Rowe (U.S. Publ. US-2022/0084659-A1).
Regarding claim 1, the reference claim 1 discloses all the limitations of the present claim 1, except for wherein the preprocessor performs auto cropping to extract a shape of the product from the image data to extract at least one feature of a brightness and a shadow of the product based on a shape of the automatically cropped product.
Pertaining to the same field of endeavor, Rowe discloses wherein the preprocessor performs auto cropping to extract a shape of the product from the image data (see figure 6, input image 610, rough crop 612 and paragraphs 0082-0090, where various methods, including automated methods, can be used to crop the input image to localize the shape of a product depicted therein) to extract at least one feature of a brightness and a shadow of the product based on a shape of the automatically cropped product (see figure 6, lighting check 622 and paragraphs 0091-0094, where lighting, glare, and shadows of the product can be recognized during the lighting check).
The ‘878 Application and Rowe are considered analogous art, as they are both directed to image analysis of manmade products. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Rowe into the ‘878 Application by cropping to determine the product shape, brightness, and shadows because doing so ensures image quality for recognizing parts of the product (see Rowe paragraph 0092).
Regarding claim 12, the same rejection as claim 1 above applies, except that the present claim 12 is merely a method claim instead of an apparatus claim.
Regarding claims 2-11 and 13-16, the present dependent claim set otherwise mirrors that of the ‘878 Application.
This is a provisional nonstatutory double patenting rejection.
Claims 1-16 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over the claims of copending Application No. 18/972,879 (the ‘879 Application) in view of Rowe (U.S. Publ. US-2022/0084659-A1).
Regarding claim 1, the reference claim 1 in the ‘879 Application discloses all the limitations of the present claim 1, except for wherein the preprocessor performs auto cropping to extract a shape of the product from the image data to extract at least one feature of a brightness and a shadow of the product based on a shape of the automatically cropped product.
Pertaining to the same field of endeavor, Rowe discloses wherein the preprocessor performs auto cropping to extract a shape of the product from the image data (see figure 6, input image 610, rough crop 612 and paragraphs 0082-0090, where various methods, including automated methods, can be used to crop the input image to localize the shape of a product depicted therein) to extract at least one feature of a brightness and a shadow of the product based on a shape of the automatically cropped product (see figure 6, lighting check 622 and paragraphs 0091-0094, where lighting, glare, and shadows of the product can be recognized during the lighting check).
The ‘879 Application and Rowe are considered analogous art, as they are both directed to image analysis of manmade products. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Rowe into the ‘879 Application by cropping to determine the product shape, brightness, and shadows because doing so ensures image quality for recognizing parts of the product (see Rowe paragraph 0092).
Regarding claim 12, the same rejection as claim 1 above applies, except that the present claim 12 is merely a method claim instead of an apparatus claim.
Regarding claims 2-11 and 13-16, the present dependent claim set otherwise mirrors that of the ‘878 Application.
This is a provisional nonstatutory double patenting rejection.
Claim Rejections – 35 USC § 101
Claims 10-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added):
An AI-based product surface inspecting apparatus, comprising:
a sensor unit which photographs a product to generate image data
and measures at least one of a color, a saturation, a brightness, a transparency, and a reflectance of the product;
a detection unit which detects a defect on the product by inputting the image data to a convolutional neural network (CNN) trained to detect a defect on a product surface;
and a preprocessor which converts the image data based on at least one of the color, the saturation, the brightness, the transparency, and the reflectance of the product to input the converted data to the detection unit,
wherein a number of convolution layers of the convolutional neural network is determined based on a defect detection difficulty determined according to at least one of the color, the saturation, the brightness, the transparency, and the reflectance of the product and a defect type,
and, wherein the preprocessor performs auto cropping to extract a shape of the product from the image data
to extract at least one feature of a brightness and a shadow of the product based on a shape of the automatically cropped product.
Step 1: Does the claim belong to one of the statutory categories? Claim 1 is directed to a machine, which is a statutory category of invention (YES).
Step 2A Prong One: Does the claim recite a judicial exception? Parts c, e-f, and h are regarded as mental processes that can be practically performed in the human mind. Part c requires measuring at least one of the recited characteristics of the product, which can be achieved by any mental observation of either the image or the real-world product. Part e recites generally converting the image data; the claim does not specify what aspect of the image is being converted or how, and thus a mental conversion of the data in any way would read on this limitation. Part f recites determining a number of convolutional layers in the CNN; a mental determination would read on this, as long as it considers in some way the difficulty of detection, at least one of the five characteristics, and the type of defect, which can also be mentally considered. Step h recites “extracting” at least one feature of a brightness and shadow of the product; a mental recognition would read on this, as long as it considers in some way the shape of the cropped product (YES).
Part d does recite detecting a defect on the product, which can be performed mentally, but the claim limits this to be performed by a specific convolutional neural network, which cannot be performed mentally.
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application? Parts a-b, d, and g are regarded as additional elements. Part a is a non-limiting preamble. Part b recites mere data gathering. Part g recites performing auto cropping to extract the shape of the product, which does not integrate the mental processes into a practical application. Part d, however, recites detecting a defect on the product by inputting the image to a CNN specifically; this step incorporates the results of the mental processes, because this CNN has a number of layers that is determined in step f, which is partly based on the characteristics measured in steps c/h, and the data input to the CNN is converted in step e (YES). Claim 1 is eligible.
Similar analysis is applicable to claim 12. Claim 12 is eligible.
Claims 2-5 and 13-16 recite additional elements with no new judicial exceptions. Claims 2-5 and 12-16 are eligible.
Claims 6-7 recite further details of determining the number of convolution layers that can still be performed mentally. However, step d of parent claim 1 still integrates these into a practical application. Claims 6-7 are eligible.
Claim 8 recites further details of the conversion that can still be performed mentally. However, step d of parent claim 1 still integrates these into a practical application. Claim 8 is eligible.
Claim 9 recites that the conversion includes determining a sharpening frequency, which can be performed mentally, and applying the filter, which is directed to mathematical calculations. However, step d of parent claim 1 still integrates these into a practical application. Claim 9 is eligible.
Claim 10 recites that the detection unit detects a position, size, and type of the defect, and determining that the defect is not a defect based on the position, size, and type occurring at a predetermined consistency level, which can both be performed mentally. The claim does not specify that the detection unit uses the CNN to do this, unlike part d of parent claim 1. Thus, claim 10 is not eligible.
Claim 11 recites displaying a bounding box based on the determined position and size of the defect, which amounts to mere instructions to perform a mental process using a computer. The claim further recites mere data output. Claim 11 is not eligible.
Allowable Subject Matter
The examiner notes that claims 1-12 are not reasonably disclosed or suggested by the prior art of record under 35 U.S.C. 102 or 103, but still have other remaining rejections above.
Regarding claim 1, although general CNN-based defect detection is well-known in the art, the art fails to disclose or reasonably suggest wherein a number of convolution layers of the convolutional neural network is determined based on a defect detection difficulty determined according to at least one of the color, the saturation, the brightness, the transparency, and the reflectance of the product and a defect type, as claimed.
Nikolaos et al ("Dynamic Pruning of CNN Networks", 2019 10th International Conference on IISA paper, 14 November 2019) discloses a CNN model for image recognition that dynamically prunes/disables individual kernels in each layer, depending on how well each kernel manages to learn data features during inferences (see section III, “Dynamic Pruning”). Although an entire layer can be interpreted to be removed if every kernel in said layer were to be disabled, and thus the number of convolutional layers can be determined based on the input data, Nikolaos fails to disclose doing so based on any specific aspects of training data, such as the color, saturation, brightness, transparency, or reflectance of a product, or of a specific a defect type, as claimed. Furthermore, the kernel pruning only appears to take place at the end of a collective training procedure, not in response to each individual image as claimed.
Chen et al (U.S. Patent US-11288789-B1) discloses an image analysis system for detecting vehicle damage in images using different CNNs (see figure 1, CNNs 134 and column 6, line 36 to column 7, line 5). The system selects a particular CNN for analyzing each part of the vehicle (see figure 2, change determiner 208 and column 12, line 66 to column 14, line 5; see figure 13 and column 27, lines 18-65 for an example). Selecting a particular CNN in this context can read on determining a number of convolution layers as claimed, assuming the CNNs have different numbers of layers, but Chen’s system only appears to choose the CNN based on the type of car part, and not characteristics of the car such as color, saturation, brightness, transparency, or reflectance, or the type of damage/defect on the car part, as required by the claim.
Regarding claim 12, this independent claim is not disclosed by the prior art for similar reasons to claim 1 above.
Regarding claims 2-11 and 13-16, these dependent claims are not disclosed by the prior art based on their dependence on claims 1/12 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS JOHN HELCO whose telephone number is (703)756-5539. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella, can be reached at telephone number 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/NICHOLAS JOHN HELCO/Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights