DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9/19/23 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims labeled 1, 1-14 and actually comprising a total of 15 claims are objected to because of the following: There are two claims labeled claim 1; All the claim numbering would appear to be off-count; Claim 3 self referentially depends from claim 3; and so does claim 14 as well, self referentially depending from claim 14. Appropriate correction is required.
Claim ”14” is objected to under 37 CFR 1.75 as being a substantial duplicate of claim “13”. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The preamble of claim 1 states:
“A method of detecting errors in the placement of elements in the PCB, carried out with the use of a computer, comprising the steps of:”
Which presents issues of antecedent basis.
Claim 1 recites the limitations "the placement of elements" and “the PCB” as the first instance of each in the Preamble. There is insufficient antecedent basis for these limitations in the claim.
Dependent claims 2-12 are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph, for their dependency to a rejected claim under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1 & 1-14 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US PG Pub 2023/0125477 to Gurumurthy et al.
Regarding claim 1. Gurumurthy discloses a method of detecting errors in the placement of elements in the PCB (Abstract), carried out with the use of a computer (Fig. 2, Computer 202), comprising the steps of:
a) providing an image of a PCB and an image of a reference PCB to an error detection module (“In at least one embodiment, an AOI system 200 bases, at least in part, its analysis of a manufactured object 206 on an ideal sample, which may be used to create an ideal image or ideal video (e.g., golden image, golden template, golden template image, golden video). In at least one embodiment, a manufactured objects 206 may comprise defects (e.g., scratches, soldering errors, components skews) to be used in pre-training one or more neural networks to detect defects in manufactured products. In at least one embodiment, a manufactured object such as a PCB may comprise over eight hundred components (e.g., capacitors, resistors); comprise over fifty different types of components; be as small as 0.1 mm×0.1 mm or smaller. In at least one embodiment, AOI system 200 can be used to count and inspect each component while detecting defects”, paragraph 73),
b) detecting, in the error detection module with the use of artificial intelligence algorithms, errors in elements mounting in the image of the PCB (“FIG. 3 depicts a neural network architecture 300 according to at least one embodiment. In at least one embodiment, a neural network architecture 300 can be used to configure one or more circuits of one or more processors to detect one or more defects in one or more manufactured objects based, at least in part, on one or more combined images of one or more manufactured objects. In at least one embodiment, a neural network architecture 300 comprises two identical neural networks that share use of a loss function 314 (e.g., Siamese neural network) and uses manufactured object images 302 (e.g., images or video of a manufactured object) and sample object images 304 (e.g., images or video of a sample object 212, golden sample object or an ideal or defect-free manufactured object) as input data. In at least one embodiment, a neural network architecture is a Siamese neural network comprising two similar neural networks that share use of a loss function 314”, paragraph 77),
c) indicating the mounting errors (“In at least one embodiment, one or more neural networks of a network architecture 300 identifies a defect if one or more features on a manufactured object do not match (e.g., calculate a distance greater than a threshold value for a distance) with features on a sample object. In at least one embodiment, one or more neural networks of a network architecture 300 identifies features on a manufactured object by drawing and displaying bounding boxes around said features on display 208”, paragraph 81).
Regarding claim 1. Gurumurthy discloses wherein before step a) step a1) is carried out to generate, with the use of a camera, of an image of the PCB (“In at least one embodiment, one or more trained neural networks 114 perform inferencing 110 using video 112 (e.g., images, digital photos, bit streams) containing information about a manufactured object (e.g., PCB) captured by a camera (e.g., microscope, x-ray sensor, CCD image sensor, CMOS image sensor, NMOS image sensor, infrared sensor, webcam)”, paragraph 63).
Regarding claim 2. Gurumurthy discloses wherein the error detection module verifies mounting correctness via verification of at least one quality of the list: lighting of a LED, mounting of a correct element, in particular a LED with correct light colour, mounting of all the elements on the PCB, correct orientation, polarization, in particular rotation, of the element, correctness of the readout of text or graphic codes, in particular of the board or element, determining absence of other elements than electronic elements, determining correct positioning of the PCB in the station, determining correct mounting variant on the PCB (“in at least one embodiment, an AOI system 200 bases, at least in part, its analysis of a manufactured object 206 on an ideal sample, which may be used to create an ideal image or ideal video (e.g., golden image, golden template, golden template image, golden video). In at least one embodiment, a manufactured objects 206 may comprise defects (e.g., scratches, soldering errors, components skews) to be used in pre-training one or more neural networks to detect defects in manufactured products. In at least one embodiment, a manufactured object such as a PCB may comprise over eight hundred components (e.g., capacitors, resistors); comprise over fifty different types of components; be as small as 0.1 mm×0.1 mm or smaller. In at least one embodiment, AOI system 200 can be used to count and inspect each component while detecting defects”, paragraph 73).
Regarding claim 3. Gurumurthy discloses wherein for verification of a quality at least one, separately learned artificial intelligence algorithm is used, preferably the artificial intelligence algorithm being based on a neural network (“In at least one embodiment, one or more neural networks of a network architecture 300 identifies a defect if one or more features on a manufactured object do not match (e.g., calculate a distance greater than a threshold value for a distance) with features on a sample object. In at least one embodiment, one or more neural networks of a network architecture 300 identifies features on a manufactured object by drawing and displaying bounding boxes around said features on display 208”, paragraph 81).
Regarding claim 4. Gurumurthy discloses wherein the error detection unit based on an artificial intelligence algorithm learns to detect errors on a database comprising images of elements correctly and erroneously mounted on a PCB (“In at least on embodiment, a manufactured objects 206 is an ideal sample (e.g., golden sample), which may comprise an ideal manufactured product free of defects and may be used in pre-training (e.g., training a machine learning model with tasks to apply said machine learning model to other tasks) one or more neural networks to detect defects in manufactured products”, paragraph 72, “In at least one embodiment, an AOI system 200 bases, at least in part, its analysis of a manufactured object 206 on an ideal sample, which may be used to create an ideal image or ideal video (e.g., golden image, golden template, golden template image, golden video). In at least one embodiment, a manufactured objects 206 may comprise defects (e.g., scratches, soldering errors, components skews) to be used in pre-training one or more neural networks to detect defects in manufactured products”, paragraph 73, “A training framework, in an embodiment, then backpropagates training loss 424 values to a generator in order to train one or more neural networks using input videos containing additional information. In at least one embodiment, if training is complete 426, training ends 428. Otherwise, in an embodiment, a process 400 for training one or more neural networks to detect defects in PCBs using input videos, continues by fetching additional input training videos 406. In at least one embodiment, training is complete 426 when one or more circuits of one or more processors use one or more neural networks to calculate losses (e.g., average squared loss), and said losses between a set of training images and a set of validation images are below a threshold value or exceed a threshold number of iterations (e.g., epochs) of training”, paragraph 91).
Regarding claim 5. Gurumurthy discloses wherein indication of mounting errors in step c) is carried out via generating an image of the PCB with marked frames to indicate regions with detected error (“In at least one embodiment, process 500 continues with one or more circuits of one or more processors using one or more neural networks to output results 510 from analyzing combined images 508. In at least one embodiment, outputting results 510 comprises one or more circuits of one or more processors: drawing a bounding box around inferred (e.g., detected) defects on a manufactured object 206 on display 208; displaying counts of many types of components inferred (e.g., counted) on a manufactured object 206 on display 208”, paragraph 95).
Regarding claim 6. Gurumurthy discloses wherein in step b) a dedicated artificial intelligence algorithm that learned to detect errors of a PCB, based on earlier series of images of sample PCBs, detects mounting errors on a PCB (“In at least one embodiment, an AOI system 200 bases, at least in part, its analysis of a manufactured object 206 on an ideal sample, which may be used to create an ideal image or ideal video (e.g., golden image, golden template, golden template image, golden video). In at least one embodiment, a manufactured objects 206 may comprise defects (e.g., scratches, soldering errors, components skews) to be used in pre-training one or more neural networks to detect defects in manufactured products. In at least one embodiment, a manufactured object such as a PCB may comprise over eight hundred components (e.g., capacitors, resistors); comprise over fifty different types of components; be as small as 0.1 mm×0.1 mm or smaller”, paragraph 73).
Regarding claim 7. Gurumurthy discloses wherein in step b) the error detection module compares an image of a reference PCB to the image of the PCB, where individual regions comprising a single element are analysed (“In at least one embodiment, an AOI system 200 bases, at least in part, its analysis of a manufactured object 206 on an ideal sample, which may be used to create an ideal image or ideal video (e.g., golden image, golden template, golden template image, golden video). In at least one embodiment, a manufactured objects 206 may comprise defects (e.g., scratches, soldering errors, components skews) to be used in pre-training one or more neural networks to detect defects in manufactured products. In at least one embodiment, a manufactured object such as a PCB may comprise over eight hundred components (e.g., capacitors, resistors); comprise over fifty different types of components; be as small as 0.1 mm×0.1 mm or smaller. In at least one embodiment, AOI system 200 can be used to count and inspect each component while detecting defects”, paragraph 73), and
before step b) step b1) is carried out consisting on separation from the image of the PCB of regions that comprise the singular element (“In at least one embodiment, supervision comprises input information that describes one or more aspects of training data 104, such as objects or styles, or a classification for said training data 104, to assist training one or more neural networks 108 by training framework 106. In at least one embodiment, supervision is strong, wherein input information provides direct identification of an object, style, or other aspect of an item, such as an image, in training data 104. In at least one embodiment, supervision is weak, wherein input information provides partial identification of an object, style, or other aspect of an input training data 104 item. In at least one embodiment, strong supervision is input information such as bounding boxes, where one or more objects are outlined in an input training data 104 item”, paragraph 61, “In at least one embodiment, a manufactured object such as a PCB may comprise over eight hundred components (e.g., capacitors, resistors); comprise over fifty different types of components; be as small as 0.1 mm×0.1 mm or smaller. In at least one embodiment, AOI system 200 can be used to count and inspect each component while detecting defects”, paragraph 73), and step b) is preferably carried out with the use of a Siamese neural network (“In at least one embodiment, a neural network architecture 300 comprises two identical neural networks that share use of a loss function 314 (e.g., Siamese neural network) and uses manufactured object images 302 (e.g., images or video of a manufactured object) and sample object images 304 (e.g., images or video of a sample object 212, golden sample object or an ideal or defect-free manufactured object) as input data. In at least one embodiment, a neural network architecture is a Siamese neural network comprising two similar neural networks”, paragraph 77).
Regarding claim 8. Gurumurthy discloses wherein in step b) the error detection module compares an image of a reference PCB with the image of the PCB, and an image obtained as a result of combination into one image of the image of the reference PCB and the image of the PCB is analysed (“In at least one embodiment, combined images comprise two or more images. In at least one embodiment, AOI system 200 analyzes a manufactured object 206 based, at least in part, on two or more images of different resolutions”. Paragraph 70, “In at least one embodiment, one or more circuits of one or more processors of network architecture 300 uses one or more neural networks to infer a count of components or defects on a manufactured object based, at least in part, on using a loss function 314 to compare distances between features extracted from images of a manufactured object and features extracted from a sample object such as a golden sample of a manufactured object”, paragraph 81, “In at least one embodiment, an AOI system 200 bases, at least in part, its analysis of a manufactured object 206 on an ideal sample, which may be used to create an ideal image or ideal video (e.g., golden image, golden template, golden template image, golden video). In at least one embodiment, a manufactured objects 206 may comprise defects (e.g., scratches, soldering errors, components skews) to be used in pre-training one or more neural networks to detect defects in manufactured products. In at least one embodiment, a manufactured object such as a PCB may comprise over eight hundred components (e.g., capacitors, resistors); comprise over fifty different types of components; be as small as 0.1 mm×0.1 mm or smaller. In at least one embodiment, AOI system 200 can be used to count and inspect each component while detecting defects”, paragraph 73), and the individual regions comprising a single element are analysed (“”, paragraph),
where before step b) step b1) is carried out consisting on separation from the image of the PCB of regions comprising a singular element (“In at least one embodiment, supervision comprises input information that describes one or more aspects of training data 104, such as objects or styles, or a classification for said training data 104, to assist training one or more neural networks 108 by training framework 106. In at least one embodiment, supervision is strong, wherein input information provides direct identification of an object, style, or other aspect of an item, such as an image, in training data 104. In at least one embodiment, supervision is weak, wherein input information provides partial identification of an object, style, or other aspect of an input training data 104 item. In at least one embodiment, strong supervision is input information such as bounding boxes, where one or more objects are outlined in an input training data 104 item”, paragraph 61, “In at least one embodiment, a manufactured object such as a PCB may comprise over eight hundred components (e.g., capacitors, resistors); comprise over fifty different types of components; be as small as 0.1 mm×0.1 mm or smaller. In at least one embodiment, AOI system 200 can be used to count and inspect each component while detecting defects”, paragraph 73), and step b) is preferably carried out with the use of a modified Siamese neural network in which the input images are combined and provided at the input of the network as one image (“In at least one embodiment, a neural network architecture 300 comprises two identical neural networks that share use of a loss function 314 (e.g., Siamese neural network) and uses manufactured object images 302 (e.g., images or video of a manufactured object) and sample object images 304 (e.g., images or video of a sample object 212, golden sample object or an ideal or defect-free manufactured object) as input data. In at least one embodiment, a neural network architecture is a Siamese neural network comprising two similar neural networks”, paragraph 77, “FIG. 4. In at least one embodiment, a neural network may comprise any neural network suitable for comparing two or more images. In at least one embodiment, one or more circuits are to use a loss function 314 to compare one or more combined images of a manufactured object with one or more other images of a sample object”, paragraph 79).
Regarding claim 9. Gurumurthy discloses wherein the method is implemented in at least two stages of assembling of the PCB (“In at least one embodiment, process 500 continues with one or more circuits of one or more processors using one or more neural networks to output results 510 from analyzing combined images 508. In at least one embodiment, outputting results 510 comprises one or more circuits of one or more processors: drawing a bounding box around inferred (e.g., detected) defects on a manufactured object 206 on display 208; displaying counts of many types of components inferred (e.g., counted) on a manufactured object 206 on display 208; triggering an assembly line to kick out a manufactured object 206 after detecting a defect on manufactured objects 206, or some combination thereof”, paragraph 95).
Regarding claim 10. Gurumurthy discloses wherein detection of a mounting error on a PCB triggers an alert (“In at least one embodiment, process 500 continues with one or more circuits of one or more processors using one or more neural networks to output results 510 from analyzing combined images 508. In at least one embodiment, outputting results 510 comprises one or more circuits of one or more processors: drawing a bounding box around inferred (e.g., detected) defects on a manufactured object 206 on display 208; displaying counts of many types of components inferred (e.g., counted) on a manufactured object 206 on display 208; triggering an assembly line to kick out a manufactured object 206 after detecting a defect on manufactured objects 206, or some combination thereof”, paragraph 95, note: a “kick out” is considered an alert).
Regarding claim 11. Gurumurthy discloses wherein before step information is introduced concerning new mounting on the PCB according to the following steps:
a31) introduction of an image of a reference PCB (“In at least one embodiment, images capture a manufactured object or golden sample (e.g., an ideal manufactured object free of defects, a manufactured object of suitable quality, a manufactured object with some defects but of suitable quality, or generally an object to be used as a reference for comparison)”, paragraph 70),
a32) marking of frames that comprise elements to be verified (“In at least one embodiment, strong supervision is input information such as bounding boxes, where one or more objects are outlined in an input training data 104 item”, paragraph 61),
a33) carrying out check tests to check whether the network operates properly (“A training framework, in an embodiment, then backpropagates training loss 424 values to a generator in order to train one or more neural networks using input videos containing additional information. In at least one embodiment, if training is complete 426, training ends 428. Otherwise, in an embodiment, a process 400 for training one or more neural networks to detect defects in PCBs using input videos, continues by fetching additional input training videos 406. In at least one embodiment, training is complete 426 when one or more circuits of one or more processors use one or more neural networks to calculate losses (e.g., average squared loss), and said losses between a set of training images and a set of validation images are below a threshold value or exceed a threshold number of iterations (e.g., epochs) of training”, paragraph 91).
Regarding claim 12. Gurumurthy discloses wherein at least one step is carried out on a server, and preferably data are stored on the server (“In at least one embodiment, as shown in FIG. 9, data center infrastructure layer 910 may include a resource orchestrator 912, grouped computing resources 914, and node computing resources (“node C.R.s”) 916(1)-916(N), where “N” represents a positive integer (which may be a different integer “N” than used in other figures). In at least one embodiment, node C.R.s 916(1)-916(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field programmable gate arrays (FPGAs), graphics processors, etc.), memory storage devices 918(1)-918(N) (e.g., dynamic read-only memory, solid state storage or disk drives), network input/output (“NW I/O”) devices, network switches, virtual machines (“VMs”), power modules, and cooling modules, etc. In at least one embodiment, one or more node C.R.s from among node C.R.s 916(1)-916(N) may be a server having one or more of above-mentioned computing resources”, paragraph 123).
Regarding claim 13. Gurumurthy discloses a computer program product, comprising software embodied on a non-transitory computer readable medium, that when executed on a computer is configured to execute the method according to claim 1 (“In at least one embodiment, code is stored on a computer-readable storage medium, for example, in form of a computer program comprising a plurality of instructions executable by one or more processors. In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein”, paragraph 623).
Regarding claim 14. Gurumurthy discloses a non-transitory computer readable data storage medium, comprising the computer program product according to claim 14 (“In at least one embodiment, code is stored on a computer-readable storage medium, for example, in form of a computer program comprising a plurality of instructions executable by one or more processors. In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein”, paragraph 623).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US PG Pub 2023/0053878 to Bonig et al. discloses a computer-implemented method of predicting a quality of a printed circuit board (PCB) assembly includes obtaining production data relating to production of the PCB assembly. The production data is mapped onto a latent vector of a latent space of a trained adaptive algorithm. The trained adaptive algorithm is trained on real X-ray images of PCB assemblies and/or serves for generating X-ray images of PCB assemblies. A subspace of the latent space related to the latent vector is determined. The subspace indicates a quality of the PCB assembly. Alternatively or additionally, an X-ray image of the PCB assembly is generated by the trained adaptive algorithm based on the latent vector in order to determine a quality of the PCB assembly.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER D. WAIT, Esq. whose telephone number is (571)270-5976. The examiner can normally be reached Monday-Friday, 9:30- 6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at 571 270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CHRISTOPHER D. WAIT, Esq.
Primary Examiner
Art Unit 2683
/CHRISTOPHER WAIT/Primary Examiner, Art Unit 2683