DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 26th, 2026, has been entered.
Claim Status
Claims 1-20 in the claim set filed October 13th, 2025, were pending for examination in the Application No. 18/159,648 filed January 25th, 2023. In the remarks and amendments received on March 26th, 2026, claims 1 and 12-15 are amended and claims 21 and 22 are newly added. Accordingly, claims 1-22 are currently pending for examination in the application.
Response to Amendment
Applicant’s amendments filed March 26th, 2026, to the Claims have overcome each and every 35 U.S.C. § 112(a) and (b) rejection previously set forth in the Non-Final Office Action mailed January 2nd, 2026. Accordingly, the 35 U.S.C. § 112(a) and (b) rejection(s) are withdrawn in response to the remarks and amendments filed. Examiner warmly thanks Applicant for considering the suggested amendments to be made to the disclosure.
Response to Arguments
Applicant’s arguments filed March 26th, 2026, regarding the rejection(s) of the independent claims have been fully considered but are not persuasive.
The examiner respectfully disagrees with Applicant’s assertion that Song does not teach or suggest “feeding two separate images into a segmentation network, generating a mask based on inter-image deviation, and a neural network architecture processing anchor and test images in parallel branches” as currently recited in the independent claims (pgs. 9-10 of Applicant’s remarks). As detailed in the current rejection below, Song teaches in paragraph [0080] and Fig. 10 that two separate images (i.e., a “master” and “slave” image) are input into a trained neural network model (e.g., a “DNN”) which generates a “pixel by pixel segmentation output” mask (e.g., see output image 353 in Fig. 10) depicting the difference as “different pixels in the two images” (see the teachings of Song in the rejection of claim 1 below). Song further discloses in paragraphs [0095-0096] and Fig. 12 that the trained neural network performing the comparison may comprise of a “U network (U-net) structure”, which is an architecture comprising of ‘parallel encoding branches’ consistent with Applicant’s architecture depicted in Fig. 7 of Applicant’s drawings.
Further, the examiner respectfully disagrees with Applicant’s assertion that it would not have been reasonable to combine the teachings of Song with Ikeda because it would exceed ordinary skill and require fundamental alteration of the architecture of Ikeda’s system since Song’s system segments anomalies within a single image (pg. 10 of Applicant’s remarks). As detailed in the rejection below, each of Ikeda and Song disclose a system comprising a trained neural network to identify defects in an input image based on differences between the input image and at least one anchor image. Since Ikeda discloses in paragraphs [0123] and [0225] that this similarity comparison can be realized through machine learning image assessment such as by a segmentation-type machine learning that classifies pixels and Song discloses such a segmentation-type machine learning model in the same field of endeavor of defect detection, it would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to substitute the machine learning model used for the similarity comparison in the system of Ikeda with the machine learning model disclosed in the system of Song comprising ‘parallel encoding branches’. Since the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Ikeda, while the teaching of Song continues to perform the same function as originally taught prior to being combined, to produce the repeatable and predictable result as described previously above.
Priority (Previously Presented)
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 119(e) as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed application, Application No. 63/303,050 filed on February 3rd, 2022, fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application. The prior-filled application does not disclose the subject matter of a crimp machine comprising an anvil having a terminal support surface, a press having an actuator, a ram operably coupled to the actuator, or a crimp die coupled to the ram. Accordingly, claim 14 is not entitled to the benefit of the prior application.
Claim Interpretation (Previously Presented)
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier, as explained in MPEP § 2181, subsection I (note that the list of generic placeholders below is not exhaustive, and other generic placeholders may invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph):
A. The Claim Limitation Uses the Term "Means" or "Step" or a Generic Placeholder (A Term That Is Simply A Substitute for "Means")
With respect to the first prong of this analysis, a claim element that does not include the term "means" or "step" triggers a rebuttable presumption that 35 U.S.C. 112(f) does not apply. When the claim limitation does not use the term "means," examiners should determine whether the presumption that 35 U.S.C. 112(f) does not apply is overcome. The presumption may be overcome if the claim limitation uses a generic placeholder (a term that is simply a substitute for the term "means"). The following is a list of non-structural generic placeholders that may invoke 35 U.S.C. 112(f): "mechanism for," "module for," "device for," "unit for," "component for," "element for," "member for," "apparatus for," "machine for," or "system for." Welker Bearing Co., v. PHD, Inc., 550 F.3d 1090, 1096, 89 USPQ2d 1289, 1293-94 (Fed. Cir. 2008); Mass. Inst. of Tech. v. Abacus Software, 462 F.3d 1344, 1354, 80 USPQ2d 1225, 1228 (Fed. Cir. 2006); Personalized Media, 161 F.3d at 704, 48 USPQ2d at 1886–87; Mas-Hamilton Group v. LaGard, Inc., 156 F.3d 1206, 1214-1215, 48 USPQ2d 1010, 1017 (Fed. Cir. 1998). Note that there is no fixed list of generic placeholders that always result in 35 U.S.C. 112(f) interpretation, and likewise there is no fixed list of words that always avoid 35 U.S.C. 112(f) interpretation. Every case will turn on its own unique set of facts.
Such claim limitation(s) is/are:
"vision device configured to image…" in claims 1 and 14 implemented on hardware disclosed in paras. [0026] (e.g., "includes a camera…"), and thus, claim(s) 2-3 and 6-13 is/are similarly interpreted;
"terminal inspection module comparing… and performing…" in claims 1, 14, and 19 implemented on hardware disclosed in paras. [0032] (e.g., "the terminal inspection module 150 includes one or more memories 152 for storing data and/or executable instructions and one or more processors 154 configured to execute the executable instructions stored in the memory 152…"), and thus, claim(s) 2-3, 6-13, and 20 is/are similarly interpreted; and
"training module" in claims 7-13 “to train the terminal inspection module” or “training the terminal inspection module” as software implemented on hardware disclosed in paras. [0036] and [0037] (e.g., "the training module 180 may be part of the controller 130 such that training is performed on the controller 130 (for example, using internal processors)…").
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Objections
Claims 21-22 are objected to because of the following informalities:
In claims 21 and 22, the claims should end with a period (i.e., “.”) (see MPEP § 608.01(m));
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-14 and 22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 1 and 14, it is unclear to the examiner where the newly amended claim limitation “…compare the input image to the anchor image by processing the input image and the anchor image in parallel encoding branches of the trained neural network” (emphasis added) is supported in Applicant’s instant Specification. The examiner respectfully notes that Applicant has not pointed out where the amended (or new) claim is supported, nor does there appear to be a written description of “parallel encoding branches” in the application as filed (See, e.g., Hyatt v. Dudas, 492 F.3d 1365, 1370, n.4, 83 USPQ2d 1373, 1376, n.4 (Fed. Cir. 2007); and MPEP § 2163.04). The examiner notes that Applicant appears to be referring to the “U-net network architecture” recited in paras. [0035], [0050], and [0076] of Applicant’s instant Specification and depicted in Fig. 7 of Applicant’s drawings. However, it remains unclear to examiner based on Applicant’s instant Specification and drawings on what is considered “parallel encoding branches” as a “U-net network architecture” as known in the art is typically described as comprising an encoder and decoder path or part, which is not typically understood as ‘parallel encoding branches’ (e.g., see pertinent art Adaloglou in the conclusion section of the current Office Action below). Applicant is respectfully encouraged to specifically point out support for this claim limitation in Applicant’s disclosure. Dependent claim(s) 2-13 and 22 do not resolve or clarify this/these issue(s) and thus is/are similarly rejected under 35 U.S.C. 112(a) for the same reasons as above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8, 11-12, 15-18, and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (Ikeda; JP 2020052044 A) in view of Song et al. (Song; US 2020/0388021 A1).
Regarding claim 1, Ikeda discloses a terminal inspection system for a crimp machine (description, para(s). [0022], recite(s)
[0022] “[7] A manufacturing system for an electric wire with terminal (100, 100A, 100B, 100C) according to a representative embodiment of the present invention includes a terminal crimping device (11) that crimps a terminal (210) to an electric wire (201) to form an electric wire with terminal (200), an imaging device (8) that generates photographed image data (61) that captures an image of at least a portion of the terminal crimped portion of the electric wire with terminal formed by the terminal crimping device, and a plurality of image determination units (4, 5, 5A, 5B, 5C, 5E, 5E) that determine the quality of the terminal crimped portion based on the photographed image data generated by the imaging device, and the plurality of image determination units (4, 5, 5A, 5B, 5C, 5E, 5E) that determine the quality of the terminal crimped portion based on the photographed image data generated by the imaging device. the information output unit (7, 7A) generating and outputting information relating to the quality of the terminal crimping portion based on the judgment result, and a memory unit (6, 6A, 6B) storing the captured image data and the output of the information output unit, at least one of the plurality of image judgment units being a machine learning type image judgment unit (5, 5A, 5B, 5C, 5D, 5E), the memory unit further storing a learning model (63, 63B, 63C) by the machine learning type image judgment unit, and the machine learning type image judgment unit judging the quality of the terminal crimping portion based on the learning model and the captured image data.”
, where the “terminal crimping device” is a crimp machine) comprising:
a vision device configured to image a terminal being inspected and generate a digital image of the terminal (description, para(s). [0022]—see citation in above—, where the “imaging device” is a vision device and the “photographed image data generated” is a digital image);
a terminal inspection module communicatively coupled to the vision device (para(s). [0022]—see citation above—, where the “image determination units… that determine the quality of the terminal crimped portion based on the photographed image data generated by the imaging device” are terminal inspection modules communicatively coupled to the vision device; such as description, para(s). [0114] and [0050], recite(s)
[0114] “…The communication device 42 is a device that communicates with devices outside the terminal-fitted electric wire manufacturing system 100 and the communication device 24 in the control device 20 via a wired or wireless network.
For example, the learning model generation unit 41 in the information processing device 40 communicates with the communication device 24 of the control device 20 via the communication device 42, thereby enabling transmission and reception of various data between the image assessment device 1.”
[0050] “…The image assessment device 1 controls the imaging device 8 in response to the command to capture an image of the terminal crimped portion and obtains the captured image of the terminal crimped portion.”
) and receiving the digital image of the terminal as an input image (description, para(s). [0092-0093], recite(s)
[0092] “The image judgment unit 5 judges whether the terminal crimping portion of the inspection target is good or bad based on captured image data 61 including an image of at least a portion of the terminal crimping portion of the terminal-attached electric wire 200 to be inspected and a pre-registered learning model 63.”
[0093] “Specifically, when the photographed image data 61 to be judged is input, the image judgment unit 5 first reads out the AI inspection master information 64 including an identification code that matches the identification code of the photographed image data 61 from the memory unit 6.
Next, the image judgment unit 5 judges the quality of the terminal crimping portion based on the learning model 63 for the image information of the inspection range specified by the read AI inspection master information 64 in the image of the input captured image data 61, and outputs a judgment result that the terminal crimping portion of the terminal-attached electric wire 200 being inspected is a good product, or a judgment result that the terminal crimping portion of the terminal-attached electric wire 200 being inspected is a defective product.
As described above, the image determination unit 5 executes a machine learning type image determination process.”
, where the “photographed image data” is “input” into at least an “image judgment unit” and/or a “learning model” is receiving the digital image of the terminal as an input image), the terminal inspection module including a trained neural network and a stored anchor image representing a defect-free terminal (description, para(s). [0085-0086], [0091], and [0123], recite(s)
[0085] “(2) Machine learning type image judgment processing by image judgment unit 5 Machine learning type image judgment processing is processing for judging the quality of the terminal crimping portion of the terminal-attached electric wire 200 based on a learning model 63 by machine learning based on a predetermined technique (algorithm) and an image included in the captured image data 61.”
[0086] “Here, the predetermined technique (algorithm) may be, for example, a neural network such as a Convolutional Neural Network (CNN).”
[0091] “In the image determination device 1, for example, multiple images (for example, several dozen to several hundred) of the terminal crimping portion of a good quality terminal-equipped electric wire 200, as shown in Figures 3A and 3B, are prepared and input into the image determination unit 5 for training, thereby generating a learning model 63.
The generated learning model 63 is pre-stored in the storage unit 6 along with the AI inspection master information 64 described above.”
[0123] “In addition, machine learning image assessment processing makes it possible to classify differences from a reference image through self-learning…”
, where the “learning model” is at least a trained “neural network” and the “good quality terminal-equipped electric wire” images are anchor images representing defect-free (i.e., “good quality”) terminals), the trained neural network of the terminal inspection module configured to receive both the input image and the anchor image as separate images (description, para(s). [0085-0086], [0091], and [0123]—see preceding citation immediately above—, where the “learning model” first receiving “good quality terminal-equipped electric wire” includes receiving at least an anchor image separately; and para(s). [0092] discloses receiving an input image separately as an “image of at least part of the terminal crimping portion of the terminal-equipped electric wire 200 to be inspected”:
[0092] “The image determination unit 5 determines whether the terminal crimping portion of the terminal to be inspected is good or bad based on the captured image data 61, which includes an image of at least a part of the terminal crimping portion of the terminal-equipped electric wire 200 to be inspected, and a pre-registered learning model 63.”
) and compare the input image to the anchor image by processing the input image and the anchor image in(description, para(s). [0093] and [0225], recite(s)
[0093] “…Next, the image judgment unit 5 judges the quality of the terminal crimping portion based on the learning model 63 for the image information of the inspection range specified by the read AI inspection master information 64 in the image of the input captured image data 61, and outputs a judgment result that the terminal crimping portion of the terminal-attached electric wire 200 being inspected is a good product, or a judgment result that the terminal crimping portion of the terminal-attached electric wire 200 being inspected is a defective product…”
[0225] “Here, the first image judgment unit 5B may, for example, perform image judgment processing based on autoencoder-type machine learning that self-learns using an automatic encoder, and the second image judgment unit 5C may, for example, perform image judgment processing based on segmentation-type machine learning that classifies pixels.”
, where “perform[ing] image judgment processing” by comparing the image information obtained from at least one anchor image (i.e., “master information”) with the image information of the input image (i.e., “input captured image data”) is comparing the input image to the anchor image by processing the input image and the anchor image in the trained neural network (i.e., “learning model”) to generate an output of “a judgment result that the terminal crimping portion of the terminal-attached electric wire 200 being inspected is a good product, or… a defective product”), the output(description, para(s). [0093] and [0225]—see citations in limitation “, and performing semantic segmentation between…” above—, where the output of a “judgment result that that the terminal crimping portion of the terminal-attached electrical wire 200 being inspected is a defective product” is the output identifying differences between the input image and at least one anchor image as defects to determine if the terminal is “defective”).
Where Ikeda does not specifically disclose
…the trained neural network …configured to receive both the input image and the anchor image as separate images and compares the input image to the anchor image by processing the input image and the anchor image in parallel encoding branches of the trained neural network to perform semantic segmentation between the input image and the anchor image to generate a pixel-wise difference mask as an output image,
the output image identifying regions in which the input image deviates from the anchor image to show differences between the input image and the anchor image to identify any potential defects on the terminal to determine whether the terminal is defective based on the generated pixel-wise difference mask;
Song teaches in the same field of endeavor of machine learning defect detection
……the trained neural network …configured to receive both the input image and the anchor image as separate images (para(s). [0077-0078] and [0080] and Fig. 10, recite(s)
[0077] “More particularly, the neural network of the present disclosure may be used for classifying an anomaly product and a normal product in a production process. Further, the neural network of the present disclosure may be used for image segmentation. …Further, the image segmentation may also include a process of visualizing and displaying a part of an image discriminated from another part. For example, the neural network of the present disclosure may be used for displaying the anomaly part in the image.”
[0078] “The master image 310 is an image that is a basis of the determination of anomaly of input data. The slave image 330 is an image that is a target for determination of anomaly. In the present disclosure, the anomaly data may be abnormal data deviating from a normal pattern of data. The master image 310 may be an image including only normal state image data, not anomaly data. The slave image 330 may have a normal pattern and may also be abnormal data, and may be a target image of which anomaly is determined by the neural network. For example, the master image 310 may be an image of a normal product, and the slave image 330 may be an image of a product to be examined.”
[0080] “The computing device 100 may extract a feature from each of the master image 310 and the slave image 330 input to the neural network 200. The computing device 100 may compare the respective features and determine similarity of the master image 310 and the slave image 330. The computing device 100 may determine similarity by calculating a mathematical distance between the feature of the master image 310 and the feature of the slave image 330. For example, when the similarity of the master image 310 and the slave image 330 is equal to or smaller than a predetermined threshold value, the computing device 100 may determine the slave image 330 as an image including anomaly data. Further, the computing device 100 may compare the feature of the master image 310 and the feature of the slave image 330 through the DNN, and generate a pixel by pixel segmentation output, which expresses actually different pixels in the two images by the unit of a pixel. The computing device 100 may compare outputs of the subnetwork through a predetermined comparison algorithm. The computing device 100 may output the degree of difference between both features, a location of a pixel of a part having the difference, and the like by using the neural network in the comparison of both features, as well as the simple comparison of the difference between both features.”
PNG
media_image1.png
505
771
media_image1.png
Greyscale
, where the “neural network” used for “classifying an anomaly product and a normal product in a production process” is a trained neural network configured to receive both an input image (e.g., “slave image”) and an anchor image (e.g., “master image”) separately (i.e., “compare the feature of the master image 310 and the feature of the slave image 330 through the DNN”—see Fig. 10 above)) and compares the input image to the anchor image by processing the input image and the anchor image in parallel encoding branches of the trained neural network to perform semantic segmentation between the input image and the anchor image to generate a pixel-wise difference mask as an output image (para(s). [0077-0078] and [0080] and Fig. 10—see citations in the preceding limitation immediately above—, where determining “different” and “normal” pixels between the input and anchor image is through “pixel by pixel segmentation” is comparing the input image to the anchor image by processing the input image and the anchor image through the trained neural network to perform the intended use/result of performing semantic segmentation (i.e., “pixel-by-pixel segmentation”) between the input image and the anchor image (i.e., grouping of all pixels of the same class of either “different” or “normal” into a single segment of “different” pixels and “normal” pixels such as depicted in Fig. 10) to generate a pixel-wise difference mask as an output image (i.e., the “pixel by pixel segmentation output, which expresses actually different pixels in the two images by the unit of a pixel”);
where Fig. 12 and para(s). [0058] and [0095-0096] further recite(s) processing the input image and the anchor image in parallel encoding branches of the trained neural network:
[0058] “A deep neural network (DNN) may mean a neural network including a plurality of hidden layers, in addition to an input layer and an output layer. When the DNN is used, it is possible to recognize a latent structure of data. That is, it is possible to recognize a latent structure (for example, the kind of object included in a picture, contents and emotion included in writing, contents and emotion included in a voice, and the like) of a picture, writing, a video, a voice, and music. The DNN may include a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network Siamese network, and the like.”
[0095] “The comparison module 250 may form at least a part of a U network structure with at least one of the subnetworks. A U network 500 will be described with reference to FIG. 12. The entirety or a part of the subnetworks 210 and 230 and the entirety or a part of the comparison module 250 may form the U network. Further, the combination of the subnetworks 210 and 230 and the comparison module 250 may also be a part of the U network.”
[0096] “The U network 500 may have a DNN structure, which is capable of performing image segmentation. A left part of the U network illustrated in FIG. 12 may have a DNN structure, which is capable of decreasing a dimension of input data, and a right part of the U network may have a DNN structure, which is capable of increasing a dimension of input data. More particularly, a dimension decreasing network 510 of the U network 500 may have a CNN structure, and a dimension increasing network 530 of the U network 500 may have a DCNN structure. A part illustrated with a rectangle in FIG. 12 may be each layer of the U network. In the illustration of FIG. 12, a number of a part 501 of each layer may be an example of the number of channels of each layer. In the illustration of FIG. 12, a number of a part 503 of each layer may mean the illustrative number of pixels of an image processed in each layer, and the example of the number of pixels of the image is decreased and then increased in a direction of an arrow of the calculation of the U network illustrated in FIG. 12, so that it can be confirmed that the dimension of the image is decreased and then increased again. …”
PNG
media_image2.png
692
842
media_image2.png
Greyscale
, where the trained network (i.e., the “DNN”) receiving the input image and the anchor image separately may comprise of a “U network structure” such as depicted in Fig. 12 is the trained network comprising parallel encoding branches),
the output image identifying regions in which the input image deviates from the anchor image to show differences between the input image and the anchor image to identify any potential defects on the terminal to determine whether the terminal is defective based on the generated pixel-wise difference mask (para(s). [0077-0078] and [0080]—see citations above—, where the output image (i.e., “pixel by pixel segmentation output”) identifies regions in which the input image deviates from the anchor image as “different pixels” as depicted in Fig. 10; and thus satisfies the intended use/result limitation of showing differences between the input image and the anchor image to identify potential defects (e.g., anomalies) to determine whether the product captured in the input image is defective (e.g., an “anomaly product”) based on the generated pixel-wise difference mask (i.e., the “pixel by pixel segmentation output” depicting the differences between the images as defects)).
Since Ikeda and Song each disclose a system capturing an input image and at least one anchor image separately and using a trained neural network to identify defects in the input image based on differences between the input image and at least one anchor image, a person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that the trained neural network of Ikeda could have been substituted for the trained neural network of Song comprising parallel encoding branches generating a pixel-wise difference mask output image by performing at least semantic segmentation between the input image and the anchor image because both the trained neural networks of Ikeda and Song serve the purpose of providing a neural network to identify any potential defects in an input image by comparison between the input image and anchor image. Furthermore, a person of ordinary skill in the art would have been able to carry out the substitution without changing a “fundamental” operating principle of Ikeda since the substitution achieves the predictable result of identifying any potential defects on the terminal to determine whether the terminal is defective.
Regarding claim 2, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Song further teaches the trained neural network of the terminal inspection module is trained to suppress differences attributable to positional variation and material variation while identifying differences corresponding to structural defects (para(s). [0098-0099] and [0102], recite(s)
[0098] “The comparison module 250 may include the dimension increasing network and form at least a part of the U network structure with at least one of the subnetworks 210 and 230 . FIG. 13 is a diagram of an example of the Siamese network for implementing the image comparison method according to another exemplary embodiment of the present disclosure, and illustrates the case where the subnetworks 210 and 230 form the U network structure with the comparison module 250 . …”
[0099] “In FIG. 13, a left side of a dotted line may form the dimension decreasing network 510 , which decreases a dimension of input data, of the U network, and a right side of the dotted line may form the dimension increasing network 530 , which restores the dimension of the input data.”
[0102] “The neural network including the application of the Siamese network is used, so that the image comparison method of the exemplary embodiment of the present disclosure may learn both the case where the master image is partially different from the slave image (the case where the two images are much similar, but anomaly partially exists, so that details are partially different, and the like) and the case where the master is much different from the slave image (the case where the two image are considerably different due to lens distortion, light change, texture difference, and the like, but the anomaly does not exist, so that details are similar, and the like) and classify the cases. The image comparison method of the present disclosure may determine whether anomaly data exists in the slave image by the difference in the detail when the master image and the slave image belong to the same domain. Particularly, when both the master image and the slave image are the images related to fabric including a flower pattern (the case where the patterns are the same and the textures are the same), different portions in the two images may be anomaly data. Further, the image comparison method of the present disclosure may compare the details even when the master image and the slave image belong to the different domains, and determine whether anomaly data exists in the slave image. Particularly, when the master image is an image related to fabric including a flower pattern and the slave image is an image related to leather including a star pattern (the case where the patterns are different and the textures are different), the image comparison method may ignore a large difference portion in the two images generated due to the different domains and determine whether anomaly data exists in the slave image by examining the details. Accordingly, the image comparison method of the present disclosure may have general recognition performance despite a rotation and transformation of image data, an error by lens distortion, the domain change, and the like, thereby having an effect in that training data is secured for each domain and a limit of the existing neural network, in which learning needs to be performed for each domain, is overcome.”
, where the trained neural network comprising a “U network structure” such as in a “Siamese network” lets the image comparison method to “ignore a large difference portion in the two images generated due to the different domains” such as different positional variations (e.g., “rotation and transformation”) and different “textures” of material (e.g., “fabric”) is training the trained neural network to suppress differences attributable to positional variation and material variation while identifying differences corresponding to structural defects (e.g., “anomaly data exists” between different fabric domains through the disclosed neural network architecture are structural defects)).
Regarding claim 3, Ikeda in view of Song discloses the terminal inspection system of claim 1, Song further teaches wherein the terminal inspection module directly compares the input image with the anchor image (para(s). [0077-0078] and [0080] and Fig. 10—see citation in claim 1 limitation “, the terminal inspection module comparing…” above—, where determining “different” and “normal” pixels between the input and anchor image is through “pixel by pixel segmentation” is directly comparing said images).
Regarding claim 4, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Song further teaches the semantic segmentation performs a pixel-by-pixel comparison of the input image and the anchor image (para(s). [0080]—see citation in claim 1 above—, where a “a pixel by pixel segmentation output, which expresses actually different pixels in the two images by the unit of a pixel” is a semantic segmentation performing a pixel-by-pixel comparison).
Regarding claim 5, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Song further teaches the output image highlights differences between the input image and the anchor image by displaying pixels without differences with a first color in the output image and displaying pixels with differences with a second color in the output image (para(s). [0092] and Fig. 10, recite(s)
[0092] “As illustrated in FIG. 10, a neural network 200 in still another exemplary embodiment of the present disclosure may learn a normal image 410 and a target image 430 as training data 420 and be trained so as to output an output 353 related to location information about a pixel, at which the anomaly exists, in input data. That is, in the learning scheme illustrated in FIG. 10, the normal image 410 and the target image 430 are input to the neural network 200, and the neural network 200 may output data related to the location information about the pixel, at which the anomaly exists, in the target image 430. The computing device 100 may compare an output of the neural network and a target (for example, whether the anomaly labelled to the target image 430 exists), calculate an error, and back-propagate the error to train the neural network 200 so as to output the data related to the location information about the pixel, at which the anomaly exists, in the slave image 330. That is, the computing device 100 may train the neural network 200 so as to output an output 353 related to the location information about the pixel, at which the anomaly exists, in the image data by using the training data 420 labelled with whether anomaly data exists.”
PNG
media_image1.png
505
771
media_image1.png
Greyscale
, where the “output 353” is an output image highlighting differences between the input image (e.g., “slave image” 330) and the anchor image (e.g., “master image” 310) by displaying pixels without differences with a first color (e.g., white pixels as depicted in output 353 in Fig. 10) and displaying pixels with differences with a second color in the output image (e.g., black pixels as depicted in output 353 in Fig. 10)).
Regarding claim 6, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Song further teaches the terminal inspection module includes a U-net network architecture for comparing the input image and the anchor image and generating the output image (para(s). [0095-0096], recite(s)
[0095] “The comparison module 250 may form at least a part of a U network structure with at least one of the subnetworks. A U network 500 will be described with reference to FIG. 12. The entirety or a part of the subnetworks 210 and 230 and the entirety or a part of the comparison module 250 may form the U network. Further, the combination of the subnetworks 210 and 230 and the comparison module 250 may also be a part of the U network.”
[0096] “The U network 500 may have a DNN structure, which is capable of performing image segmentation. A left part of the U network illustrated in FIG. 12 may have a DNN structure, which is capable of decreasing a dimension of input data, and a right part of the U network may have a DNN structure, which is capable of increasing a dimension of input data. More particularly, a dimension decreasing network 510 of the U network 500 may have a CNN structure, and a dimension increasing network 530 of the U network 500 may have a DCNN structure. A part illustrated with a rectangle in FIG. 12 may be each layer of the U network. In the illustration of FIG. 12, a number of a part 501 of each layer may be an example of the number of channels of each layer. In the illustration of FIG. 12, a number of a part 503 of each layer may mean the illustrative number of pixels of an image processed in each layer, and the example of the number of pixels of the image is decreased and then increased in a direction of an arrow of the calculation of the U network illustrated in FIG. 12, so that it can be confirmed that the dimension of the image is decreased and then increased again. Arrow 511 illustrated in FIG. 12 may mean a convolutional operation of applying the convolutional filter to the image. For example, arrow 511 may be the convolutional operation of applying the 3×3 convolutional filter to the image, but the present disclosure is not limited thereto. Arrow 513 illustrated in FIG. 12 may indicate an operation of transmitting information required for increasing the dimension of the dimension-decreased image from the dimension decreasing network 510 to the corresponding dimension increasing network 530. Arrow 515 illustrated in FIG. 12 may mean a pooling operation for decreasing the pixel of the image. For example, arrow 515 may be max pooling of extracting a maximum value, but the present disclosure is not limited thereto. Arrow 517 illustrated in FIG. 12 may mean a convolutional operation of increasing the dimension of the image. For example, arrow 517 may be the convolutional operation using the 2×2 convolutional filter, but the present disclosure is not limited thereto. Arrow 519 illustrated in FIG. 12 may mean a convolutional operation for transferring an output to a complete connection layer. For example, arrow 519 may be the convolutional operation using the 1×1 convolutional filter. In the illustration of FIG. 12, a rectangle with deviant crease lines included in the dimension increasing network 530 may mean the reception of information for increasing the dimension of the image from a corresponding layer of the dimension decreasing network 510.”
, where a “U Network” is a U-Net).
A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that the “machine learning image assessment processing” for comparing the input image to the anchor image disclosed by Ikeda (para(s). [0003]—see claim 1 limitation “, the terminal inspection module comparing…” above) can incorporate a U-Net to improve performing the semantic segmentation for comparing the input image and the anchor image by transferring local information about a feature to corresponding layers in the neural network as taught by Song (para(s). [0097], recite(s)
[0097] “The U network 500 may have a structure (arrow 513 illustrated in FIG. 12), in which information (for example, the location information about the pixel and a high-level feature) for increasing the dimension is transferred to a process of increasing the dimension of the image in a process of decreasing the dimension of the image for the image segmentation. That is, each layer of the dimension decreasing network 510 of the U network may transfer location information about the feature to a corresponding layer of the dimension increasing network 530. Accordingly, it is possible to restore the location information about the pixel, which may be lost during the process of decreasing the dimension of the image and then increasing the dimension of the image. Accordingly, the location information about the pixel may be restored during the process of increasing the dimension of the image, so that the U network may use the location information about the pixel in the essential image segmentation…”
) while still yielding the predictable result of identifying any potential defects by comparing the input image and the anchor image.
Regarding claim 7, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Ikeda further discloses the terminal inspection module includes a training module using a training data set to train the terminal inspection module, the training data set including the anchor image, (and) a plurality of positive images(para(s). , recite(s)
[0007] “…Therefore, when introducing a machine learning type image inspection device, it is necessary to prepare a large amount of learning images (sample data) in advance. In other words, when using a machine learning type image inspection device to determine whether the terminal crimp portion of an electric wire with a terminal is good or bad during the manufacturing process of an electric wire with a terminal as described above, it is necessary to prepare a large number of images of the terminal crimp portion of a good product and an image of the terminal crimp portion of a defective product in advance and allow the machine learning type image inspection device to thoroughly learn from them.”
[0091] “In the image assessment device 1, for example, multiple images (e.g., tens to hundreds) of the terminal crimping portion of a good terminal-attached electric wire 200 as shown in Figures 3A and 3B are prepared, and these are input into the image assessment unit 5 for learning, thereby generating a learning model 63…”
[0192] “For example, the learning model generation unit 32 generates a learning image (learning data) for the photographic image data 61 of the inspection target by labeling the judgment result of the non-machine learning type image judgment unit 4 regarding the photographic image data 61 as correct answer information, and updates the learning model 63 by re-learning the learning data.”
, where the “learning image for the photographic image data” is an anchor image and the “multiple images… of a good terminal-attached electric wire… for learning” are a plurality of positive images).
Where Ikeda does not specifically disclose
…and a plurality of negative images;
Ikeda further teaches
…and a plurality of negative images (para(s). [0007], recite(s)
[0007] “…Therefore, when introducing a machine learning type image inspection device, it is necessary to prepare a large amount of learning images (sample data) in advance. In other words, when using a machine learning type image inspection device to determine whether the terminal crimp portion of an electric wire with a terminal is good or bad during the manufacturing process of an electric wire with a terminal as described above, it is necessary to prepare a large number of images of the terminal crimp portion of a good product and an image of the terminal crimp portion of a defective product in advance and allow the machine learning type image inspection device to thoroughly learn from them.”
, where “a large number of images… of the terminal crimp portion of a defective product” for machine learning is a plurality of negative images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Ikeda in view of Song to incorporate a plurality of negative images in a training data set to improve training the machine learning type image inspection of a terminal crimp portion of an electric wire.
Regarding claim 8, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Song further teaches the terminal inspection module includes a training module training the terminal inspection module to ignore differences relating to material differences between the input image and the anchor image (para(s). [0102], recite(s)
[0102] “The neural network including the application of the Siamese network is used, so that the image comparison method of the exemplary embodiment of the present disclosure may learn both the case where the master image is partially different from the slave image (the case where the two images are much similar, but anomaly partially exists, so that details are partially different, and the like) and the case where the master is much different from the slave image (the case where the two image are considerably different due to lens distortion, light change, texture difference, and the like, but the anomaly does not exist, so that details are similar, and the like) and classify the cases. The image comparison method of the present disclosure may determine whether anomaly data exists in the slave image by the difference in the detail when the master image and the slave image belong to the same domain. Particularly, when both the master image and the slave image are the images related to fabric including a flower pattern (the case where the patterns are the same and the textures are the same), different portions in the two images may be anomaly data. Further, the image comparison method of the present disclosure may compare the details even when the master image and the slave image belong to the different domains, and determine whether anomaly data exists in the slave image. Particularly, when the master image is an image related to fabric including a flower pattern and the slave image is an image related to leather including a star pattern (the case where the patterns are different and the textures are different), the image comparison method may ignore a large difference portion in the two images generated due to the different domains and determine whether anomaly data exists in the slave image by examining the details. Accordingly, the image comparison method of the present disclosure may have general recognition performance despite a rotation and transformation of image data, an error by lens distortion, the domain change, and the like, thereby having an effect in that training data is secured for each domain and a limit of the existing neural network, in which learning needs to be performed for each domain, is overcome.”
, where the inclusion of a “Siamese network” lets the image comparison method to “ignore a large difference portion in the two images generated due to the different domains” such as different “textures” of material (e.g., “fabric”) is ignoring differences relating to material differences between the two images).
Regarding claim 11, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Ikeda further discloses the terminal inspection module includes a training module training the terminal inspection module to identify differences relating to differences in shapes between the terminal in the input image and the terminal in the anchor image (para(s). [0161], recite(s)
[0161] “First, in the terminal-fitted electric wire manufacturing system 100A, a learning model 653 relating to the shape of the bell-mouth is generated as the learning model 65 for inspecting the terminal crimped portion.
For example, multiple images (e.g., tens to hundreds) of images of the terminal crimping portion as shown in FIG. 11A are prepared as images of the terminal crimping portion where the bellmouths 214, 215 are appropriately formed, and multiple images (e.g., tens to hundreds) of images of the terminal crimping portion as shown in FIG. 11B and FIG. 11C are prepared as images of the terminal crimping portion where the bellmouths 214, 215 have inappropriate shapes, and these images are input to image judgment unit 5A for learning, thereby generating learning model 653.”
).
Regarding claim 12, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Ikeda further discloses the terminal inspection module includes a training module for training the terminal inspection module, wherein the training module uses two-dimensional images (para(s). [0091], recite(s)
[0091] “In the image assessment device 1, for example, multiple images (e.g., tens to hundreds) of the terminal crimping portion of a good terminal-attached electric wire 200 as shown in Figures 3A and 3B are prepared, and these are input into the image assessment unit 5 for learning, thereby generating a learning model 63.
The generated learning model 63 is pre-stored in the memory unit 6 together with the above-mentioned AI inspection master information 64.”
, where the “multiple images” for learning are two-dimensional images).
Regarding claim 15, the claim is a method reciting limitations all performed by the system of claim 1. Therefore, claim 15 recites similar limitations to claim 1 and is rejected for similar rationale and reasoning (see the analysis for claim 1 above).
Regarding claim 16, the claim recites similar limitations to claim 4 and is rejected for similar rationale and reasoning (see the analysis for claim 4 above).
Regarding claim 17, the claim recites similar limitations to claim 5 and is rejected for similar rationale and reasoning (see the analysis for claim 5 above).
Regarding claim 18, the claim recites similar limitations to claim 6 and is rejected for similar rationale and reasoning (see the analysis for claim 6 above).
Regarding claim 21, the claim recites similar limitations to claim 2 and is rejected for similar rationale and reasoning (see the analysis for claim 2 above).
Regarding claim 22, the claim recites similar limitations to claim 2 and is rejected for similar rationale and reasoning (see the analysis for claim 2 above).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Ikeda in view of Song as applied to claim(s) 1 above, and further in view of Ortega (US 2021/0049754 A1).
Regarding claim 9, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Ikeda further discloses the terminal inspection module includes a training module training the terminal inspection module(detect) differences between the input image and the anchor image (para(s). [0091], recite(s)
[0091] “In the image assessment device 1, for example, multiple images (e.g., tens to hundreds) of the terminal crimping portion of a good terminal-attached electric wire 200 as shown in Figures 3A and 3B are prepared, and these are input into the image assessment unit 5 for learning, thereby generating a learning model 63.
The generated learning model 63 is pre-stored in the memory unit 6 together with the above-mentioned AI inspection master information 64.”
, where the “image assessment device” comprises at least a training module for “generating a learning model” to detect differences (e.g., defects) between the input image and the anchor image).
Where Ikeda in view of Song does not specifically disclose
…to ignore differences relating to positional differences…;
Ortega teaches in the same field of endeavor of terminal inspection
…to ignore differences relating to positional differences…(para(s). [0031] and [0059], recite(s)
[0031] “Embodiments of the invention resolve the problems with conventional vision inspection systems by introducing an artificial intelligence program that significantly reduces the complexity of system set up and operation without sacrificing quality. In an embodiment, an artificial intelligence program is utilized to identify the object of interest, e.g., a terminal element added to the end of an insulated wire. This object identification is not sensitive to the alignment or orientation of the wire and terminal combination, i.e., so long as the object of interest is presented in the field of view of the camera(s), the inspection process may be successfully completed. Further, precise timing is not essential, and embodiments can use video cameras to detect defects using images of objects of interest, e.g., even if the object of interest is simply manually waived in the field of view of the camera(s) by an operator.”
[0059] “An object to be analyzed, e.g., a wire 506 and terminal 507, is placed in the center of the imaging device 500, e.g., manually by an operator. For manual operation, a pair of stabilizing elements 504a, 504b optionally may be included to assist the operator in placing the object in view of the cameras. This orients the object to be imaged roughly in the middle of the two cameras 503a, 503b. Again, this positioning can be accomplished either manually or by some mechanical device, as the imaging analysis is not sensitive to precise alignment. In this way, the cameras 503a, 503b will take images of the top and bottom of the object to have a 360-degree view of the object.”
, where the “object identification is not sensitive to the alignment or orientation of the wire and terminal combination” is ignoring differences relating to positional differences).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Ikeda in view of Song to incorporate training the inspection module to ignore differences relating to material differences between the input image and the anchor image to improve defect detection by significantly reducing the complexity of system set up and operation without sacrificing quality as taught by Ortega above.
Claims 10, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda in view of Song as applied to claim(s) 1 above, and further in view of Walser et al. (Walser; US 2023/0056526 A1).
Regarding claim 10, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Ikeda further discloses the terminal inspection module includes a training module training the terminal inspection module to identify differences relating to objects identified in the input image (para(s). [0161], recite(s)
[0161] “First, in the terminal-fitted electric wire manufacturing system 100A, a learning model 653 relating to the shape of the bell-mouth is generated as the learning model 65 for inspecting the terminal crimped portion.
For example, multiple images (e.g., tens to hundreds) of images of the terminal crimping portion as shown in FIG. 11A are prepared as images of the terminal crimping portion where the bellmouths 214, 215 are appropriately formed, and multiple images (e.g., tens to hundreds) of images of the terminal crimping portion as shown in FIG. 11B and FIG. 11C are prepared as images of the terminal crimping portion where the bellmouths 214, 215 have inappropriate shapes, and these images are input to image judgment unit 5A for learning, thereby generating learning model 653.”
, where the “terminal crimping portion” is an object identified in the input image).
Where Ikeda in view of Song does not specifically disclose
…identify differences relating to foreign objects identified in the input image;
Walser teaches in the same field of endeavor of terminal inspection
…identify differences relating to foreign objects identified in the input image (para(s). [0030], recite(s),
[0030] “…additionally, the image measuring algorithm is configured to identify contaminants or production residues on the detected image, so that a further quality criterion for an ideal cable processing process may be satisfied.”
, where “contaminants or production residues” are foreign objects).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Ikeda in view of Song to incorporate training the inspection module to identify differences relating to foreign objects identified in the input image to improve the quality of terminal inspection as taught by Walser above.
Regarding claim 13, Ikeda in view of Song discloses the terminal inspection system of claim 1, wherein Song further teaches the terminal inspection module includes a training module for training the terminal inspection module, images used by the training module being images (para(s). [0091]—see citation in claim 12 above).
Where Ikeda in view of Song does not specifically disclose
…being three-dimensional images;
Walser teaches in the same field of endeavor of terminal inspection
…being three-dimensional images (para(s). [0001] and [0010], recite(s),
[0001] “The invention relates to a cable processing station, a cable machine with cable processing stations, and a computer-implemented method according to the independent claims. Optical cable inspection devices are known per se. They are used for performing quality control in cable processing, for spot checking finished cable systems consisting of a cable end and a terminal, such as a plug for electrically or optically connecting a cable with an end device…”
[0010] “…The camera may in particular detect multiple images of the cable end of the cable in order to provide a larger selection of detected images to the image processing system. In this context, the camera may record images of the cable or the cable end from various perspectives, and in particular at least one image for a frontal view of the cable axis in order to record the layer construction of the cable. In this way, for example, a twining (twist) of the electrical conductor is rendered identifiable on the detected image. The camera represented here typically has a zoomable lens and various standard commercial filter elements, and is configured to detect three-dimensional images.”
, where the “camera” capturing a terminal detects “three-dimensional images” is a terminal inspection system comprising a vision device configured to image a terminal being inspected and generating a digital image of the terminal as a three-dimensional image).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the presently filed invention to try training the terminal inspection module using three-dimensional images instead of two-dimensional images because a person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that a terminal inspection system capturing three-dimensional images as disclosed by Walser would comprise of training the terminal inspection module on three-dimensional images in order to train the terminal inspection system to identify defects in captured three-dimensional images.
Regarding claim 20, the claim recites similar limitations to claim 13 and is rejected for similar rationale and reasoning (see the analysis for claim 13 above).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Ikeda in view of Song as applied to claim(s) 1 above, and further in view of Bluemmel et al. (Bluemmel; US 2019/0221949 A1).
Regarding claim 14, the claim differs from claim 1 in that the claim is in the form of a crimp machine comprising:
an anvil having a terminal support surface at a crimp zone configured to support a terminal during a crimping operation;
a press having an actuator, a ram operably coupled to the actuator, and a crimp die coupled to the ram, the actuator moving the ram in a pressing direction during the crimping operation to move the crimp die relative to the anvil, the crimp die having a forming surface configured to crimp the terminal in the crimp zone during the crimping operation; and
the terminal inspection system of claim 1.
Ikeda in view of Song discloses a crimp machine comprising: …the terminal inspection system of claim 1 (see the rejection of claim 1 above).
Where Ikeda in view of Song does not specifically disclose said crimp machine comprising:
an anvil having a terminal support surface at a crimp zone configured to support a terminal during a crimping operation; and
a press having an actuator, a ram operably coupled to the actuator, and a crimp die coupled to the ram, the actuator moving the ram in a pressing direction during the crimping operation to move the crimp die relative to the anvil, the crimp die having a forming surface configured to crimp the terminal in the crimp zone during the crimping operation;
Bluemmel teaches in the same field of endeavor of terminal inspection said crimp machine comprising:
an anvil having a terminal support surface at a crimp zone configured to support a terminal during a crimping operation (para(s). [0025] and [0036], recite(s)
[0025] “The crimp barrel 20 is configured to be crimped around the end of the wire to mechanically and electrically connect the wire to a terminal…”
[0036] “The crimping device includes an anvil and a crimp tooling member. The anvil has a top surface that receives the crimp segment 14, 16 thereon. The electrical conductors of the wire are received in the crimp barrel 20 on the anvil. The crimp tooling member includes a forming profile that is selectively shaped to form or crimp the barrel 20 around the conductors when the forming profile engages the crimp segment 14, 16. The forming profile defines part of a crimp zone in which the crimp segment 14, 16 and wire are received during the crimping operation. The top surface of the anvil also defines a part of the crimp zone, as the crimp segment 14, 16 is crimped to the wire between the crimp tooling member and the anvil.”
, where the “anvil has a top surface that receives the crimp segment” is a an anvil having a terminal support surface); and
a press having an actuator, a ram operably coupled to the actuator, and a crimp die coupled to the ram, the actuator moving the ram in a pressing direction during the crimping operation to move the crimp die relative to the anvil, the crimp die having a forming surface configured to crimp the terminal in the crimp zone during the crimping operation (para(s). [0007] and [0037], recite(s)
[0007] “The crimp connection is produced by a crimping die, which consists of an anvil and a crimping stamp. For crimping, the crimping base is positioned centrally on the anvil, and the electrical conductor is placed between crimping legs on the crimping barrel. Subsequently, the crimping stamp descends onto the anvil and bends the crimp flanks around the electrical conductor in order to compress it tightly and to fix it in a force-locking manner with the crimping barrel. In the transition area from the crimp base to the crimp side walls, the so-called crimping roots, as well as laterally at the crimp side walls, zones of high bending stresses are formed in the crimp barrel. The force connection between the crimp barrel and the electrical conductor can be improved by providing additional form-fitting elements for example, recesses or depressions on the inner side of the crimp barrel facing the conductor for the creation of locking elements, wherein displaced conductor material can penetrate into the recesses during compression. The pressed zones of a crimping connection may have better electrical properties and the less heavily pressed areas have a higher mechanical stability. The crimping barrel and the electrical conductor can be locally reinforced by steps or projections in the crimping die.”
[0037] “The crimp tooling member is movable towards and away from the anvil along a crimp stroke. The crimp stroke has an upward component away from the anvil and a downward component towards the anvil. The crimp tooling member moves bi-directionally, towards and away from the anvil, along a crimp axis. The crimp tooling member forms the crimp segment 14, 16 around the electrical conductors during the downward component of the crimp stroke as the crimp tooling member moves towards the anvil. Although not shown, the crimp tooling member may be coupled to a mechanical actuator that propels the movement of the crimp tooling member along the crimp stroke. For example, the crimp tooling member may be coupled to a movable ram of an applicator or lead-maker machine. In addition, the applicator or the lead-maker machine may also include or be coupled to the anvil and the base support of the crimping device.”
, where the “crimp tooling member” is a ram operably coupled to the actuator (e.g., “mechanical actuator”); and the “crimping die” is a crimp die coupled to the ram (e.g., the “crimp tooling member” or “crimping stamp”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Ikeda in view of Song to incorporate the mechanical elements listed in the current claim above in the crimp machine disclosed by Ikeda to perform terminal inspection for a crimp machine comprising the mechanical elements taught by Bluemmel above.
Claims 19 is rejected under 35 U.S.C. 103 as being unpatentable over Ikeda in view of Song as applied to claim(s) 1 above, and further in view of Ortega (US 2021/0049754 A1), and further more in view of Walser et al. (Walser; US 2023/0056526 A1).
Regarding claim 19, Ikeda in view of Song discloses the terminal inspection method of claim 15, further comprising training a terminal inspection module used for the image comparison and the output image generation (Ikeda; para(s). [0161]—see the citation in claim 10 above—, where training the “learning model… for inspecting the terminal crimped portion” is training the terminal inspection module), said training comprising:
training the terminal inspection module to ignore differences relating to material differences between the input image and the anchor image (Song; para(s). [0102]—see similar limitation in claim 8 above);
training the terminal inspection module(detect) differences between the input image and the anchor image (Ikeda; para(s). [0161]—see similar limitation in claim 9 above);
training the terminal inspection module to identify differences relating to(Ikeda; para(s). [0161]—see similar limitation in claim 10 above); and
training the terminal inspection module to identify differences relating to differences in shapes between the terminal in the input image and the terminal in the anchor image (Ikeda; para(s). [0161]—see similar limitation in claim 11 above).
Where Ikeda in view of Song does not specifically disclose
…to ignore differences relating to positional differences…;
Ortega teaches in the same field of endeavor of terminal inspection
…to ignore differences relating to positional differences…(para(s). [0031] and [0059]—see similar limitation in claim 9 above).
Claim 14 recites similar limitations to claim 9 and is rejected for similar rationale and reasoning (see the rejection of claim 9 above).
Where Ikeda, as modified by Song and Ortega, does not specifically disclose
…identify differences relating to foreign objects identified in the input image;
Walser teaches in the same field of endeavor of terminal inspection
…identify differences relating to foreign objects identified in the input image (para(s). [0030]—see similar limitation in claim 10 above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Ikeda, as modified by Song and Ortega, to incorporate training the inspection module to identify differences relating to foreign objects identified in the input image to improve the quality of terminal inspection as taught by Walser above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Adaloglou (“An overview of U-net architectures for semantic segmentation and biomedical image segmentation,” 2021) discloses in section “U-Net and 3D U-Net”:
PNG
media_image3.png
479
664
media_image3.png
Greyscale
[U-Net and 3D U-Net] “It can be divided into an encoder-decoder path or contracting-expansive path equivalently.
Encoder (left side): It consists of the repeated application of two 3x3 convolutions. Each conv is followed by a ReLU and batch normalization. Then a 2x2 max pooling operation is applied to reduce the spatial dimensions. Again, at each downsampling step, we double the number of feature channels, while we cut in half the spatial dimensions.
Decoder path (right side): Every step in the expansive path consists of an upsampling of the feature map followed by a 2x2 transpose convolution, which halves the number of feature channels. We also have a concatenation with the corresponding feature map from the contracting path, and usually a 3x3 convolutional (each followed by a ReLU). At the final layer, a 1x1 convolution is used to map the channels to the desired number of classes.”
Ali et al. (“Detection of Steel Surface Defects Using U-Net with Pre-trained Encoder,” 2021) discloses on pg. 186 and Fig. 2:
[1. Introduction] “…U-Net - called in this way due to its U shape - is a type of convolutional neural network that performs the task of semantic segmentation; it has symmetric architecture, having encoder that extracts spatial features from the images, and a decoder that forms the segmentation map from the encoded features [8], see Fig. 2.”
PNG
media_image4.png
573
784
media_image4.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA Z YAO whose telephone number is (571)272-2870. The examiner can normally be reached Monday - Friday (8:30AM - 5PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.Z.Y./Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights