DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
2. This communication is in response to the application filled on 11/26/2024.
3. Claims 1-16 are pending.
4. Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Information Disclosure Statement
5. The information disclosure statement (IDS) submitted on 11/26/2024 has been considered by the examiner.
Claim Objections
6. Claim 2-3 and 9-10 objected to because of the following informalities:
Claim 2 and 9 recite “…one of a soldered joint and a welded joints”, consider correcting to “one of a soldered join or a welded joint”.
Claims 3 and 10 recite analogous issues, specifically, “…one of electrodes and pads…”, consider correcting to “…one of electrodes or pads…”
Appropriate correction is required.
Claim Rejections - 35 USC § 101
7. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
8. Claims 1-14 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis below follows Subject Matter Eligibility Test (See flowchart in MPEP 2106).
Step 1: Is the claim to a process, machine, manufacture or composition of matter? YES.
Step2A, Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon? YES. Claim 1 recites “A system for evaluating quality of bonding of wires to catheter elements for training an AI-based optical inspection model, the system comprising (1): a display (2); a user interface device (3); and a processor configured to: present images of the bonds to a user on the display (4); receive upon the user interface device scoring information related to the bonding quality based on the presented images, wherein the scoring information comprises an acceptance or rejection of each bond (5); and associate the images and scoring information and store the images and scoring information in a memory in a format configured for training an artificial intelligence (AI) based optical inspection model of bonding quality. (6)” [emphasis added].
Limitation (5) and (6) recite an abstract idea directed toward the mental process groupings of abstract ideas (MPEP 2106.04(a)(2)). Specifically, a person, given an image of a bond, can determine score a bond quality based on the image, wherein the scoring includes a acceptance or rejection of each bond. A person can likewise associate a score with an image and store said information (e.g., memorize scoring and/or store in filling cabinet etc.). Specifically, a person could perform such scoring, associating, and storing mentally or with a pen and paper.
Step2A, Prong 2: Does the claim recite an additional elements that integrate the judicial exception into a practical application? NO. Limitation (1), (2), (3), (4), and (6) recite additional limitations that fail to integrate the judicial exception into a practical application. Limitation (1) recites “A system for evaluating quality of bonding of wires to catheter elements for training an AI-based optical inspection model, the system comprising…” which constitutes mere instruction to apply the judicial exception to a particular field of technology (MPEP 2106.05(h)). Limitation (2) and (3) recite “a display; a user interface device”, which constitute mere instructions to implement the abstract idea of a mental process on a generic computer (MPEP 2106.05(f)). Limitation (4) recites “and a processor configured to: present images of the bonds to a user on the display”, which constitutes mere instructions to implement the abstract idea of a mental process on a generic computer (MPEP 2106.05(f)), as well as an insignificant extra-solution data gathering step (MPEP 2106.05(g)). Limitation (6) recites “and associate the images and scoring information and store the images and scoring information in a memory in a format configured for training an artificial intelligence (AI) based optical inspection model of bonding quality.”, which constitutes mere instruction to apply the judicial exception to a particular field of technology (MPEP 2106.05(h)). The examiner further notes this constitutes well-understood, routine and convention activity of training an AI on human annotated data (MPEP 2106.05(d)).
Claims 2-14 recite additional limitations that likewise fail to specifically integrate the judicial exception into a practical application. Claim 2 recites “wherein the bonding comprises one of a soldered joint and a welded joints.”, which constitutes mere instruction to apply the judicial exception to a particular field (MPEP 2106.05(h)). Claim 3 recites “wherein the catheter elements are one of electrodes and pads on a flexible PCB (fPCB).”, which constitutes mere instruction to apply the judicial exception to a particular field (MPEP 2106.05(h)). Claim 4 recites “wherein the scoring information further comprises a selection of one or more of a plurality of predefined bond defects.”, which is directed toward the mental process groupings of abstract ideas (MPEP 2106.04(a)(2)). Specifically, a person can select from a predefined group of bond defects when scoring an image, and can perform this function using a pen and paper (e.g., each defect is given a number, list of numbers is the defects identified by a person). Claim 5 recites “wherein the user interface comprises a touchscreen and the processor is further configured to render a graphical user interface (GUI) upon the display, wherein the GUI presents buttons upon which the user may select scoring information.”, which constitutes a generic computer recitation and routine and convention activity of touchscreen buttons (MPEP 2106.05(f), (d)). Claim 6 recites “wherein the user interface is further configured to receive image uploads from the user.”, which constitutes an insignificant extra-solution data gathering step (MPEP 2106.05(g)). Claim 7 recites “wherein the AI-based model is a neural network (NN) based model.”, which constitutes mere instruction to apply the judicial exception to a particular field (MPEP 2106.05(h)). Claims 8-14 recite analogous limitations to claims 2-7. Claim 15 and Claims 16 recite additional limitations that integrate the judicial exception into a practical application. Specifically, claim 15 and 16 recite both “…receiving from the user via the GUI scoring information for each bonding wherein the scoring information includes at least whether the bonding passes or fails inspection; enabling the user to select defect labels from a predefined list when a bonding joint is deemed defective… training the AI-based optical inspection model using the training dataset” and “execute an AI-based optical inspection model train in accordance with… claim 15”. The examiner specifically notes that the whole of claim 15 ties the GUI and user selection specifically into creating a training set, and subsequently recites that said training set is used to train a model, which integrates the judicial exception into a practical application by reciting a means of an improvement to the functioning of a training the AI. This is opposed to claim 1 and 8 where no training is actually performed based on the results of the mental process, the data is only stored in a format that would allow training of the AI.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? NO. The claim’s additional elements, as stated in Prong 2, do not amount to significantly more than the judicial exception. Limitations (1), (2), (3), (4) and (6) lack sufficient structure to amount to significantly more than the judicial exception as described in Step2A. Furthermore, annotating images using user input for training AI is well-understood and routine within the industry. Therefore, limitations (1), (2), (3), (4) and (6) use well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to accomplish the judicial exception.
At least for these reasons, claims 1-14 are ineligible under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
9. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
10. Claims 1-4, 6-11, and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over CN-116051558-A to Chen et al. (hereinafter Chen), in view of “Identification of Solder Joint Failure Modes Using Machine Learning” to Min et al. (hereinafter Min), and further in view of US. Publication No. 2016/0135749 to Chan et al. (hereinafter Chan).
11. Regarding Claim 1, Chen specifically discloses a system for evaluating quality of boding of wires to {catheter} elements for training an AI-based optical inspection model, the system comprising ([pg. 2, par. 7, ln. 1-14] “According to the technical solution of the embodiment of the present invention, by obtaining the defect type of the PCB defect image to be marked; then according to the defect type of the PCB defect image to be marked, determine the labeling rule corresponding to the PCB defect image to be marked; finally according to the defect type of the PCB defect image to be marked The corresponding labeling rules are used to label the PCB defect images to be marked, which solves the defect image labeling scheme in related technologies. According to the unified labeling rules, PCB defect images of different defect types are marked, which leads to the failure of the PCB defect detection model in the training process. Accurately distinguish the defect type of the PCB defect image and reduce the detection accuracy of the PCB defect detection model. According to the defect type of the PCB defect image to be marked, it will be able to effectively distinguish the defect type and the PCB defect image corresponding to the defect type close to the defect type. The labeling rules are determined as the labeling rules corresponding to the PCB defect images to be marked, and the PCB defect images to be marked are marked, so that the PCB defect detection model can accurately distinguish the defect type of the PCB defect image during the training process, and improve the detection accuracy of the PCB defect detection model beneficial effect.”, [Fig. 5-8] see wires, [pg. 2, par. 15, ln. 1-3] “FIG. 5 is a schematic diagram of a PCB defect image provided by an embodiment of the present invention after labeling a wafer with no bonding wire between the starting point and the ending point of the bonding wire in the PCB defect image.”):
a display ([pg. 13, par. 7, ln. 1-7] “To provide for interaction with the user, the systems and techniques described herein can be implemented on an electronic device having a display device (for example, a CRT (cathode ray tube) or LCD (liquid crystal display)) for displaying information to the user monitor); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to an electronic device. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be in any form (including Acoustic input, speech input, or tactile input) to receive input from the user.”); a user interface device ([pg. 13, par. 7, ln. 1-7] see keyboard and mouse, [pg. 13, par. 8, ln. 1 to pg. 14, par. 1, ln. 4] “The systems and techniques described herein can be implemented on a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein)”); and a processor configured to ([pg. 13, par. 2, ln. 1-6] “Processor 11 may be various general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the defect image labeling method.”):
present images of the bonds to a user on the display ([pg. 5, par. 6, ln. 1-6] “Optionally, extracting the PCB defect image to be marked from each of the PCB sub-images includes: sending each of the PCB sub-images to the terminal device of the target user, so that the target user intercepts the image to be marked from each of the PCB sub-images The PCB defect image, and upload the PCB defect image to be marked and the defect type of the PCB defect image to be marked to the electronic device. Optionally, the set of PCB defect images of each defect type is composed of a set number of PCB defect images including defect areas of the defect type.”);
receive upon the user interface device scoring information related to the bonding quality based on the presented images ([pg. 5, par. 6, ln. 1-6], [pg. 5, par. 7, ln. 1-5] “Optionally, the acquiring the PCB defect image collection of each defect type includes: acquiring the PCB defect image collection of each defect type uploaded by the target user. The target user uploads the PCB defect image collection of each defect type to the electronic device through the terminal device, so that the electronic device obtains the PCB defect image collection of each defect type.”, [pg. 5, par. 8, ln. 1 to pg. 6, par. 1, ln. 2] “Optionally, according to the set of PCB defect images of each defect type and the set of PCB defect images corresponding to each defect type close to the defect type, determining the labeling rules corresponding to each defect type includes: executing for each defect type The following operation: determine at least two candidate labeling rule combinations that match the defect type and the defect type corresponding to the defect type; wherein, each candidate labeling rule combination contains a candidate corresponding to the defect type Labeling rules and an alternative labeling rule corresponding to the close defect type; according to the combination of each of the candidate label rules, for the set of PCB defect images of the defect type and the set of PCB defect images of the close defect type Annotate the PCB defect image; according to the annotated PCB defect image, determine the dispersion of the PCB defect image set of the defect type and the PCB defect image set close to the defect type under different combinations of alternative labeling rules; the corresponding The candidate labeling rule corresponding to the defect type in the first target candidate labeling rule combination with the largest degree of dispersion is determined as the labeling rule corresponding to the defect type, and the first target candidate labeling rule combination The candidate labeling rule corresponding to the proximity defect type is determined as the labeling rule corresponding to the proximity defect type.”, [pg. 6, par. 5, ln. 1-12] “Optionally, according to each combination of the candidate labeling rules, label the PCB defect images in the set of PCB defect images of the defect type and the set of PCB defect images close to the defect type, including: for each candidate The combination of labeling rules performs the following operations: send the set of candidate labeling rules, the set of PCB defect images of the defect type, and the set of PCB defect images close to the defect type to the terminal equipment of the labeling personnel, so that the labeling personnel can select The alternative labeling rules corresponding to the defect types in the labeling rule combination are used to label the PCB defect image set of the defect type, and the close defects are labeled according to the candidate labeling rules corresponding to the close defect types in the candidate label rule combination Annotate the PCB defect image collection of the type, and return the PCB defect image collection of the defect type after annotation and the PCB defect image collection of the close defect type; obtain the PCB defect of the defect type after the annotation returned by the annotator A set of images and a set of PCB defect images of the proximate defect type. Annotators are technicians who are used to label PCB defect images according to specified labeling rules.”), {wherein the scoring information comprises an acceptance or rejection of each bond}; and
associate the images and scoring information and store the images and scoring information in a memory in a format configured for training an artificial intelligence (AI) based optical inspection model of bonding quality ([pg. 9, par. 6-7, ln. 1-7] “Optionally, after marking the PCB defect image to be marked according to the labeling rule corresponding to the PCB defect image to be marked, it also includes: storing the PCB defect image to be marked in a corresponding to the PCB defect detection model in the training sample set. Optionally, the PCB defect detection model is a model for detecting defects on the PCB based on the PCB image and determining a defect type of the PCB. The training sample set contains multiple annotated PCB defect images. Based on the labeled PCB defect images in the training sample set, the machine learning model is trained to obtain a PCB defect detection model.”).
Chen does not specifically disclose wherein the bonding is for catheter PCBs, or wherein the scoring information comprises an acceptance or rejection of each bond.
However, Min specifically teaches wherein the scoring information comprises an acceptance or rejection of each bond ([pg. 2032, col. 1, Abstract, par. 1, ln. 1-16] “The reliability of solder joints is one of the most critical factors that determine the lifecycle of electronic devices, and the identification of solder joint failure modes is necessary to enhance the performance and durability of electronic devices. In this study, solder joint failure modes were identified using the fine-tuned visual geometry group 19 (VGG 19) pretrained model. Raw images (57 images) were augmented into 428 images by sectioning to classify the solder joint failure mode into two classes (good or not-good mode) for the binary classification model, and 265 not-good data points obtained from the binary classification were employed as input to classify solder joint failure mode into six classes (failure modes 1–6) for the multiclass classification model. The binary and multiclass classification models were trained and validated, achieving 99% accuracy. The binary model classified shadows and small voids as defects, identifying the failure mode as “not-good.”, [pg. 2034, col. 1, B. Data Preparation, par. 2, ln. 1 to col. 2, par. 1, ln. 6] “In the binary classification approach, the output of the models represented the occurrence of failure and was coded as either 0 (good) or 1 (not-good). All 428 data points were utilized in the model training. For the multiclass classification, the models were designed to identify the specific failure mode, indexed as 1–6, as depicted in Fig. 1. The number of data points used for training in the multiclass classification was 265, with each data point having failures. After training the deep learning models, the models were validated using additional experimental results obtained from the literature. For this verification process, 15 raw data images were collected and augmented to create 102 images… All 102 data points were used in the binary classification model verification, and 65 data points were used in the multiclass model verification. The output values for the verification data points were coded using the same method as the training data points.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Chen and Min as within the same field of machine learning to verify quality of PCB bonding, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, and is disclosed in Min, wherein a failure of bonding can result in inoperability of the product ([pg. 2032, col. 1, par. 1, ln. 1 to col. 2, par. 1, ln. 3] “Electronic packaging technologies have witnessed significant advancements in the development of functional, miniature, and lightweight electronic devices. However, these devices still face challenges related to the damage of solder joints, which serve as crucial electrical and mechanical connections between integrated chips and printed circuit boards (PCBs) [1], [2], [3]. It is imperative that these solder joints possess the capability to endure external mechanical shocks and harsh chemical environments [1], [4], [5], [6], [7].”), and thus a binary classification of good or bad with regard to the bonding allows for determination of a product’s likely operability. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the system of Chen with the acceptance/rejection scoring information of Min through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the system of Chen with the acceptance/rejection scoring information of Min such that bonds that are scored “good” are acceptable and bonds that are scored “bad” are rejected.
Min does not specifically disclose wherein the bonding is for a catheter. Therefore, a combination of Chen and Min does not specifically teach wherein the bonding is for a catheter.
However, Chan specifically teaches a catheter assembly including a bonded wire and PCB board ([par. 0087, ln. 2-18] “…the sensor device 100 is packaged for in-vivo application as illustrated in FIG. 12A and FIG. 12B. In particular, there is provided an integrated multimodal sensor system 1200 for intracranial neuromonitoring as schematically depicted in FIG. 12A incorporating the sensor device 100… 1200 comprises a flexible catheter 1210, a flexible substrate 1214, a sensor device 100 disposed on… 1214 and within a sensing end portion 1222 of… 1210, and a guide tip member 1226 extending from… 1222 of… catheter 1210 to facilitate penetration and directional guidance… 1210… 100 is glued onto a flexible substrate 1214 (e.g., flexible PCB) which is biocompatible.”, [par. 0088, ln. 1-29] “…FIG. 12A, the flexible catheter 1218 has a power wire 1242 and a signal wire 1246 therein each extending between and connected to the sensor device 100 and the housing 1230 (i.e., the PCB 1240). The power wire 1242 is arranged to supply power from the power source 1238 to the sensor device 100 and the signal wire 1246 is arranged to transmit the sensed data from the sensor device 100 to the wireless communication module 1234. In this regard, the sensor device 100 comprises a plurality of contacts 128 (or I/O ports) which is configured to be bonded to the power wire 1242 and the signal wire 1246 in the sensor system 1200. FIG. 12C illustrates the same sensor device 100 as FIG. 1B but the power wire 1242 and signal wire(s) 1246 connected to the contacts 128 via bonding wires (only one bonding wire is shown in FIG. 12C). By way of example only, there are 8 contacts 128 for connecting to 8 bonding wires. As can be appreciated from FIG. 11, the contacts 128 are advantageously arranged so as to allow single-side wirebonding.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize the Chen, Min, and Chan as within the same field of PCB and wire bonding, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, in that the combination of Chen and Min teaches a system by which the bonding of the PCB and wire of the catheter of Chan can be tested for quality, and allows for automation of such a quality inspection. One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize that in the case of catheters and other medical devices this is particularly relevant, since a faulty catheter/in vivo sensor (e.g., due to bad wire bonding) can cause serious hazards to the individual being examined. One of ordinary skill in the art, therefore, would have combined the system of the combination of Chen and Min to automate inspection of the catheter of Chan by training an AI model on the wire bonds and PCB images of a catheter as taught in Chan through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min and the catheter including PCBs and wire bonds as taught in Chan to obtain the invention as specified in claim 1.
12. Regarding Claim 2, a combination of Chen, Min and Chan teaches the system of claim 1. Chen specifically discloses wherein the bonding comprises one of a {soldered joint and} a welded joints. ([pg. 5, par. 7, ln. 4-9] “Weld wire paths with no weld wire between the start point and the end point of the weld wire are marked. Exemplarily, the PCB defect image after labeling the wafer with no bonding wire between the starting point of the bonding wire and the ending point of the bonding wire in the PCB defect image is shown in Figure 5. For the starting point of the bonding wire in the PCB defect image The PCB defect image after marking the welding wire path with no welding wire between the termination point of the welding wire is shown in Figure 6.”). Min teaches wherein the bonding comprises a soldering joint ([pg. 2032, col. 1, Abstract, par. 1, ln. 1-16], [pg. 2034, col. 1, B. Data Preparation, par. 2, ln. 1 to col. 2, par. 1, ln. 6]). The motivation to combine would have been obvious to one of ordinary skill in the art, in that soldering and welding are both analogous means by which a wire can be bonded to a PCB during manufacturing. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min and the catheter including PCBs and wire bonds as taught in Chan to obtain the invention as specified in claim 2.
13. Regarding Claim 3, a combination of Chen, Min and Chan teaches the system of claim 1. Chen does not specifically disclose wherein the catheter elements are one of electrodes and pads on a flexible PCB (fPCB).
However, Min specifically teaches wherein the {catheter} elements are one of electrodes and pads on a {flexible} PCB (fPCB) ([pg. 2032, Fig. 1 see copper pads of PCB and Package substrate]). The motivation to combine would have been obvious to one of ordinary skill in the art, in that the bonding as performed in Chen and Min relies on electrical current being transmitted to and from the PCB, and as such, it is required that the bonding would be performed on an electrode or pad of the PCB since the remainder of the PCB is generally created using insulators (e.g., plastic or fiberglass). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the system of Chen with the acceptance/rejection scoring information and bonding to a pad of a PCB of Min through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Min does not specifically disclose wherein the elements are of a catheter or wherein the PCB is flexible. Therefore, a combination of Min and Chen does not specifically teach a catheter or wherein the PCB is flexible.
However, Chan specifically teaches wherein the elements are of a catheter and wherein the PCB is a flexible PCB ([par. 0087, ln. 2-18], [par. 0088, ln. 1-29]). The motivation to combine remains analogous to claim 1. One of ordinary skill in the art, therefore, would have combined the system of the combination of Chen and Min to automate inspection of the catheter of Chan by training an AI model on the wire bonds and flexible PCB images of a catheter as taught in Chan through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information and bonding to a pad of Min and the catheter including flexible PCBs and wire bonds as taught in Chan to obtain the invention as specified in claim 3.
14. Regarding Claim 4, a combination of Chen, Min, and Chan teaches the system of claim 1. Chen specifically teaches wherein the scoring information further comprises a selection of one or more of a plurality of predefined bond defects ([pg. 5, par. 6, ln. 1-6], [pg. 5, par. 7, ln. 1-5], [pg. 5, par. 8, ln. 1 to pg. 6, par. 1, ln. 2], [pg. 6, par. 5, ln. 1-12]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min and the catheter including PCBs and wire bonds as taught in Chan to obtain the invention as specified in claim 4.
15. Regarding Claim 6, a combination of Chen, Min, and Chan teaches the system of claim 1. Chen specifically discloses wherein the user interface is further configured to receive image uploads from the user ([pg. 5, par. 7, ln. 1-5]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min and the catheter including PCBs and wire bonds as taught in Chan to obtain the invention as specified in claim 6.
16. Regarding Claim 7, a combination of Chen, Min, and Chan teaches the system of claim 1. Chen does not specifically teach wherein the AI-based model is a neural network (NN) based model.
However, Min teaches wherein the AI-based model is a neural network (NN) based model ([pg. 2032, col. 1, Abstract, par. 1, ln. 1-16], [pg. 2034, col. 1, B. Data Preparation, par. 2, ln. 1 to col. 2, par. 1, ln. 6], [pg. 2033, col. 1 par. 2, ln. 11] “While various studies have focused on ML-based solder joint failure prediction models for life cycle and reliability, the identification of failure modes using classification models has not been thoroughly investigated. This study aims to classify solder joint failure modes using a CNN model. A dataset consisting of 428 images, comprising sound (good) connections and the six failure modes, was prepared for model training. Both a binary classification model for good or not-good classification and a multiclass classification model for failure mode estimation were developed and evaluated for their performance.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize the models used in Min as neural network models. The motivation to combine would have been obvious to one of ordinary skill in the art, in that the neural network model of Min shows high accuracy ([pg. 2032, col. 1, Abstract, par. 1, ln. 1-16], [pg. 2034, col. 1, B. Data Preparation, par. 2, ln. 1 to col. 2, par. 1, ln. 6]). Specifically, the examiner notes also that the primary concern of Chen (i.e., dataset creation for AI training purposes), would specifically be beneficial to the neural network of Min, since it is trained on a small dataset ([pg. 2032, col. 1, Abstract, par. 1, ln. 1-16], [pg. 2034, col. 1, B. Data Preparation, par. 2, ln. 1 to col. 2, par. 1, ln. 6]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the system of Chen with the acceptance/rejection scoring information and neural network of Min through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art, in combining Chen with Min, would have used the neural network of Min as the machine learning model to be trained as taught in Chen.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information and neural network of Min and the catheter including PCBs and wire bonds as taught in Chan to obtain the invention as specified in claim 7.
17. Regarding Claims 8-11 and 13-14, rejections analogous to claims 1-4 and 6-7 are further applicable in view of the analogous claim language. Specifically, Chen likewise discloses a method for training an AI-based optical inspection model ([pg. 1, par. 8, ln. 1-5] “The present invention provides a defect image labeling method, device, equipment, and medium to solve the defect image labeling scheme in the related art and to label PCB defect images of different defect types according to a unified labeling rule, resulting in a PCB defect detection model…”, [pg. 2, par. 7, ln. 1-14] [Fig. 5-8] see wires, [pg. 2, par. 15, ln. 1-3]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min and the catheter including PCBs and wire bonds as taught in Chan to obtain the invention as specified in claim 8-11 and 13-14.
18. Claims 5, 12 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over CN-116051558-A to Chen, in view of “Identification of Solder Joint Failure Modes Using Machine Learning” to Min, and further in view of US. Publication No. 2016/0135749 to Chan, and further in view of U.S. Publication No. 2021/0318673 to Kitchen et al. (hereinafter Kitchen).
19. Regarding Claim 5, a combination of Chen, Min, and Chan teaches the system of claim 1. Chen discloses wherein the user interface {comprises a touchscreen} and the processor is further configured to render a graphical user interface (GUI) upon the display, wherein the GUI presents buttons upon which the user may select scoring information ([pg. 13, par. 7, ln. 1-7], [pg. 13, par. 8, ln. 1 to pg. 14, par. 1, ln. 4]). Chen does not specifically disclose wherein the user interface comprises a touchscreen. Likewise, Min and Chan do not specifically disclose wherein the user interface comprises a touchscreen.
However, Kitchen specifically teaches wherein the user interface comprises a touchscreen ([par. 0043, ln. 1-10] “The computing device 200 may include a user interface 206 comprising a display device 208 and one or more input devices or mechanisms 210. In some implementations, the input device/mechanism includes a keyboard. In some implementations, the input device/mechanism includes a “soft” keyboard, which is displayed as needed on the display device 208, enabling a user to “press keys” that appear on the display 208. In some implementations, the display 208 and input device/mechanism 210 comprise a touch screen display (also called a touch sensitive display).”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize the system of the combination of Chen, Min, and Chan and Kitchen as within the same field of solder and welding analysis, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, in that a touchscreen may allow for easier annotation and input from a user. One of ordinary skill in the art, therefore, would have combined the system of the combination of Chen, Min, and Chan with the touchscreen user interface of Kitchen through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min, the catheter including PCBs and wire bonds as taught in Chan, and the touchscreen user interface as taught in Kitchen to obtain the invention as specified in claim 5.
20. Regarding Claim 12, a combination of Chen, Min, and Chan teaches the method of claim 8. Rejections analogous to claim 5 are further applicable to claim 12 in view of the analogous claim language. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Chen with the acceptance/rejection scoring information of Min, the catheter including PCBs and wire bonds as taught in Chan, and the touchscreen user interface as taught in Kitchen to obtain the invention as specified in claim 12.
21. Regarding Claim 15, Chen discloses a method for training an AI-based based optical inspection model for assessing quality of bonding of wires to {catheter} elements ([pg. 1, par. 8, ln. 1-5], [pg. 2, par. 7, ln. 1-14] [Fig. 5-8] see wires, [pg. 2, par. 15, ln. 1-3]), the method comprising:
{capturing, by an optical imaging system}, a plurality of images of bonding joints between wires and {catheter} elements automated bonding process ([pg. 2, par. 7, ln. 1-14] [Fig. 5-8] see wires, [pg. 2, par. 15, ln. 1-3]) {using an optical imaging system};
presenting captured images to a user through a graphical user interface (GUI) ([pg. 5, par. 6, ln. 1-6], [pg. 13, par. 7, ln. 1-7], [pg. 13, par. 8, ln. 1 to pg. 14, par. 1, ln. 4]);
receiving from the user via the GUI scoring information for each bonding ([pg. 5, par. 6, ln. 1-6], [pg. 5, par. 7, ln. 1-5], [pg. 5, par. 8, ln. 1 to pg. 6, par. 1, ln. 2], [pg. 6, par. 5, ln. 1-12]) {wherein the scoring information includes at least whether the bonding passes or fails inspection};
enabling the user to select defect labels from a predefined list when a bonding joint is deemed defective ([pg. 5, par. 6, ln. 1-6], [pg. 5, par. 7, ln. 1-5] [pg. 5, par. 8, ln. 1 to pg. 6, par. 1, ln. 2], [pg. 6, par. 5, ln. 1-12], [pg. 5, par. 3, ln. 1-8] “Optionally, determining at least two candidate labeling rule combinations matching the defect type and the defect type corresponding to the defect type includes: extracting the defect type and the defect type corresponding to the defect type from a preset file A combination of at least two candidate labeling rules that match. The preset file contains at least two candidate labeling rule combinations that match each defect type and the close defect type corresponding to each defect type. The target user uploads the preset file to the electronic device through the terminal device. For each defect type, the electronic device may extract at least two candidate labeling rule combinations matching the defect type and the defect type close to the defect type from the preset file.”, [pg. 7, par. 7, ln. 1 to pg. 8, par. 1, ln. 9] “Optionally, according to the set of PCB defect images of each defect type and the set of PCB defect images corresponding to each defect type close to the defect type, determining the labeling rules corresponding to each defect type includes: executing for each defect type The following operation: determine at least two candidate labeling rule combinations that match the defect type and the defect type corresponding to the defect type; wherein, each candidate labeling rule combination contains a candidate corresponding to the defect type Labeling rules and an alternative labeling rule corresponding to the close defect type; according to the combination of each of the candidate label rules, for the set of PCB defect images of the defect type and the set of PCB defect images of the close defect type Annotating the PCB defect image; for each candidate labeling rule combination, according to the preset clustering algorithm, the PCB defect images in the marked PCB defect image set and the PCB defect image set close to the defect type are respectively labeled The images are clustered to determine the cluster center distance of the PCB defect image set of the defect type and the PCB defect image set close to the defect type under different combinations of candidate labeling rules; the corresponding cluster center distance is the largest The candidate labeling rule corresponding to the defect type in the combination of two target candidate labeling rules is determined as the labeling rule corresponding to the defect type, and the labeling rule in the second target candidate labeling rule combination that is close to the The candidate labeling rule corresponding to the defect type is determined as the labeling rule corresponding to the approaching defect type.”);
storing the captured images, user-assigned scores, and selected defect labels in a memory as a training set ([pg. 9, par. 6-7, ln. 1-7]); and
training the AI-based optical inspection model using the training dataset ([pg. 9, par. 6-7, ln. 1-7]).
Chen does not specifically disclose the bonds are for a catheter, or wherein the scoring information includes at least whether the bonding passes or fails inspection. Likewise, Chen does not specifically disclose capturing the images by an optical imaging system, though this would have been required to obtain the images, and thus would have been obvious to one of ordinary skill in the art.
However, Min specifically teaches wherein the scoring information includes at least whether the bonding passes or fails inspection ([pg. 2032, col. 1, Abstract, par. 1, ln. 1-16], [pg. 2034, col. 1, B. Data Preparation, par. 2, ln. 1 to col. 2, par. 1, ln. 6], [pg. 2032, col. 1, par. 1, ln. 1 to col. 2, par. 1, ln. 3]). The motivation to combine remains analogous to claim 1. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the system of Chen with the acceptance/rejection scoring information of Min through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the system of Chen with the acceptance/rejection scoring information of Min such that bonds that are scored “good” are passing inspection and bonds that are scored “bad” are failing inspection.
Min does not specifically disclose a catheter, or an optical imaging system. Therefore, a combination of Chen and Min does not specifically disclose a catheter or an optical imaging system.
However, Chan specifically teaches a catheter assembly including a bonded wire and PCB board ([par. 0087, ln. 2-18], [par. 0088, ln. 1-29]). The motivation to combine remains analogous to claim 1. One of ordinary skill in the art, therefore, would have combined the system of the combination of Chen and Min to automate inspection of the catheter of Chan by training an AI model on the wire bonds and PCB images of a catheter as taught in Chan through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Chan does not specifically teach an optical imaging system for acquiring images of a bonding. Therefore, a combination of Chen, Min, and Chan does not specifically disclose an optical imaging system.
However, Kitchen specifically discloses an capturing, by an optical imaging system, images of a bonding ([par. 0007, ln. 1-18] “…the invention uses one or more cameras as sensors to capture sequenced imagery (e.g., still images or video) during welding of weld events (e.g., base metal and filler melt, cooling, and seam formation events). The sequenced images are processed as a multi-dimensional data array with computer vision and machine/deep learning techniques to produce pertinent analytics…”, [par. 0079, ln. 1-19] “FIG. 6B shows an example camera system (or image acquisition system) 608, according to some implementations. In some implementations, a camera capture system is placed in the vicinity (e.g., 1′-5′), on a tripod or on a mount fixed to a robot arm, or affixed to a rigid surface from above, or otherwise placed in a location, with visibility to a weld section. The camera system 608 collects imagery and/or video of a weld in progress… weld images record patterns and behavior of weld events, such as weld pool shape, size, intensity patterns, contours, depth, thermal gradients, changes over time, uniformity, spatter, alignment, and other hidden variables and interactions not explicitly defined as inputs, ultimately used to determine a relationship to final as-welded qualities. Some implementations use a high-speed optical camera or video camera 610… and transfer images… to an image capture and compute server 616…”, [Fig. 1, see Camera Devices 104]). The motivation to combine would have been obvious to one of ordinary skill in the art, in that the images used in the system of the combination of Chen, Min, and Chan would be acquired using an optical imaging system analogous to Kitchen, so that all the bonds would still be visible and faulty bonds could be more effectively flagged (i.e., by the trained model of the combination of Chen, Min, and Chan). Specifically, by acquiring the images during the manufacturing, it avoids any issues related to post production casings (e.g., PCBs are generally not openly visible, since exposure to elements can cause damage to electrical components, and thus images should be taken before casings/housings are applied) and allows for faster determination of defects (i.e., immediately after welding, if defects are detected, flag the PCB as failing inspection and dispose or redo the welding, etc.). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the system of the combination of Chen, Min, and Chan with the optical imaging system of Kitchen through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have used the images acquired in situ during manufacturing by an optical imaging system as taught in Kitchen as the images used for training and dataset creation as taught in the system of the combination of Chen, Min, and Chan.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min, the catheter including PCBs and wire bonds as taught in Chan, and optical imaging system as taught in Kitchen to obtain the invention as specified in claim 15.
22. Regarding Claim 16, rejections analogous to claim 15 are further applicable to claim 16. Specifically, a combination of Chen discloses a system for {real-time} assessment of quality of bonding of wires to {catheter} elements ([pg. 1, par. 8, ln. 1-5] [pg. 2, par. 7, ln. 1-14] [Fig. 5-8] see wires, [pg. 2, par. 15, ln. 1-3]), comprising:
{a sub-system for automated bonding of the wires};
{an optical system configured to capture} an image of the bonding portion of the bonded wires ([pg. 2, par. 7, ln. 1-14] [Fig. 5-8] see wires, [pg. 2, par. 15, ln. 1-3]); and
a processor configured to execute an AI-based optical inspection model trained in accordance with the method of claim 15 ([pg. 13, par. 2, ln. 1-6]).
Chen does not specifically disclose a real-time assessment of the bonding, a catheter, a sub-system for automated bonding of the wires, or an optical system configured to capture the images.
However, a combination of Chen, Min, Chan, and Kitchen disclose the system of claim 15, and arguments analogous to claim 15 are applicable to the shared elements of claim 16. Specifically, the system of the combination of Chen, Min, Chan, and Kitchen includes a catheter, and an optical system configured to capture the images (see claim 15 citations and arguments). Chen, Min, and Chan do not specifically disclose real-time assessment of the quality of the bonding, or a sub-system for automated bonding of the wires. However, Kitchen specifically discloses real-time assessment of the quality of the bonding using a trained AI model ([Fig. 1, see 102, Welding Equipment, and 112, in-situ inspection server/engine with deep learning data models 115], [par. 0059, ln. 1-21] “Some implementations perform real-time monitoring, identifying defects as they occur or soon after the defects occur. Some implementations use limited image parameters (e.g., shape of weld pool and/or a box-boundary around the shape). Some implementations process images based on a trained computer vision and machine/deep learning algorithm, produce intelligent image reconstruction and quality prediction, and/or produce dimensionally-accurate visual and quantitative weld defect characterization(s), during a welding process (or as the welding completes)”, [par. 0089, ln. 10-16] “…images captured by the camera are analyzed by a computer system (e.g., a system applying machine learning algorithms) to identify welding defects in real-time or when welding is in progress. Some implementations monitor one or more welding parameters, including transverse speed, rotation, gas flow, and any control variables that can be correlated to normal or good welding.”), and a sub-system for automated bonding of the wires ([Fig. 1, see 102, Welding Equipment, and 112, in-situ inspection server/engine with deep learning data models 115], [par. 0036, ln. 1-11] “FIG. 1 is a block diagram of a system 100 for in-situ inspection of welding processes using digital data models, in accordance with some implementations. Welding equipment 102 is monitored by one or more camera devices 104, each device 104 including one or more image sensors 106 and one or more image processors 108. Data collected by the camera devices is communicated to an in-situ inspection server 112 using a communication network 110. The welding equipment 102 uses a set of weld parameters 118, which can be updated dynamically by the in-situ inspection server 112.”, [Fig. 3A and B], [par. 0055, ln. 1-18] “According to some implementations, techniques disclosed herein apply to a wide range of weld processes. For example, the techniques can be used to inspect weld quality for gas tungsten arc welding or GTAW (sometimes called Tungsten-electrode inert gas welding or TIG), plasma arc welding, laser welding, electron beam welding, shielded metal, and gas metal welding, automated and/or manual welding, pulsed welds, and submerged welds. In some implementations, the techniques are applied during operations at multiple facilities, and/or on two or more types of welds (e.g., GTAW, where a weld torch moves across a fixed part, as is the case with most cladding, and some linear welds, and GTAW, where a weld torch is fixed and the part rotates, as is the case with circle seam welds, and some cladding). In some implementations, the techniques are used to simultaneously inspect weld quality for a large number of welds (e.g., a particular steam generator has 257 thick welds, with strict inspection criteria and a high reject rate).”, [par. 0057, ln. 1-19] “FIG. 3B is an example weld process 302, according to some implementations. The example shows a robotic arm welder 304, and views 306 of the welding process. Traditionally, robotic welding is monitored by a weld technician who observes weld melt and filler deposition (e.g., via a video monitor). The technician identifies abnormalities, and uses experience and observation to determine weld quality. Conventional systems do not capture weld process data, or do not use captured data to inspect quality. In some situation, pre-inspection quality control is performed using a qualified process based on pre-production mock-up where process parameters are determined. Most conventional systems require manual supervision, produce highly subjective and variable results, and/or detect only a small percentage of defects. FIG. 3C is another example weld process 308, according to some implementations. The example shows dual thru-shell nozzle welding stations 310, and a single platform 312 connecting two stations with desk and storage space in between.”). The motivation to combine would have been obvious to one of ordinary skill in the art, and is analogous to the motivation provided in claim 15, and is likewise disclosed in Kitchen, wherein it allows for the immediate determination of defective bonding and improves analysis of the cause of the welding issues ([par. 0003, ln. 1-16] “Manufacturing of components for safety critical systems, such as nuclear pressure vessels, is typically guided by strict requirements and design codes. Traditionally, such requirements are verified through costly non-destructive examination (NDE) after weld operations are complete, or through prequalification of weld process (to predict weld quality). After welding process is complete, routine repairs are performed to ensure quality (e.g., replacing or welding defective parts), sometimes without the knowledge of what caused the defects. Conventional techniques for welding quality control are error-prone, and cost-intensive.”, [par. 0004, ln. 1-15] “In addition to the problems set forth in the background section, there are other reasons where an improved system and method of inspecting welding quality are needed. For example, because existing techniques rely on postmortem analysis of welding failures, context information is absent for proper root-cause analysis. Some techniques only apply to a limited range of weld processes. Conventional systems for weld inspection rely on process method qualification, NDE post-weld inspection, or regressive techniques using weld process parameters, such as voltage, torch speed, amps, gas flow, but such conventional methods do not regress well to the desired quality features. The present disclosure describes a system and method that addresses at least some of the shortcomings of conventional methods and systems.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the system of the combination of Chen, Min, and Chan with the optical imaging system, sub-system for automated bonding, and real-time bonding quality assessment of Kitchen, though known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the system of the combination of Chen, Min, and Chan such that the images used for training the model were acquired from a optical imaging system connected to the sub-system for automated bonding inspection, as taught in Kitchen, and wherein the model was executed by the processor on in-situ images of bonding for the real-time analysis of the bonding quality as taught in Kitchen after having been trained.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Chen with the acceptance/rejection scoring information of Min, the catheter including PCBs and wire bonds as taught in Chan, and the optical imaging system, sub-system for automated bonding, and real-time bonding quality assessment of Kitchen to obtain the invention as specified in claim 16.
Conclusion
23. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights