CTNF 18/719,976 CTNF 84736 Detailed Action 1. Claims 17-32 are pending in this Application. Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia 2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103, which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 3. Claims 17, 19, 23, 25, 28 and 30 are rejected r under 35 U.S.C. 103(a) as being unpatentable over Tsuyoshi Nagato et al., (hereafter Tsuyoshi),” Defect Inspection Technology for a Gloss-Coated Surface Using Patterned Illumination” pub. 2013, in view of NIIHARA YOSHIMI (hereafter NIIHARA), JP H0569319 A, pub. 03/20/1993. As to claim 17, Tsuyoshi teaches a surface inspection device ( Fig.6 page 5 section 4, coating appearance inspection apparatus ) comprising: a hardware processor that acquires a plurality of images of an inspected site on a coated surface of a workpiece imaged by an imaging device ( Figs.4 and 7, page 5 section 4, Figure 6 shows the appearance of the apparatus, and Figure 7 shows the structure of the internal optics system. The internal structure includes a camera, a projector and screen. As shown in Fig. 4 the camera move in the coated layers and capture a plurality of image ) in a state where a bright and dark pattern of an illumination device that illuminates the inspected site is moved relative to the workpiece ( Figs. 4 and 5, page 4, 2 nd paragraph, the amplitude of intensity modulation on the coated surface is calculated by the four-step phase-shifting method [3] as shown in Fig. 5. In this method, the amplitude Iamp(x, y) is calculated according to the equations below by obtaining the images I 1 to I 4 taken in the four patterns, each with a different initial phase δ of the illumination pattern by π/2. ); calculates a feature representing a surface defect for the plurality of images that has been acquired, ( Fig. 8, tabel.1, Section 4.1, Table 1 illustrates feature of the observation methods for defects on the coated surface. Fig. 8 shows the appearance of the intensity modulation at individual defects with the phase shift of the stripe pattern. It was found that a convex defect has reduction in amplitude. In the case of scratches also, the amplitude was reduced. In the case of dirt, the amplitude was hardly observed.); and estimates a depth from the coated surface of a foreign substance causing the surface defect using change in the feature that has been calculated ( Fig. 8b page 6, last par. using change in the feature calculated by the calculation means (c "phase shift" and reduction in amplitude in view of figure 8b would determine a convex defect which is attributed to foreign matter inside the coating as disclosed on page 1, last paragraph, line 1); however , it is noted that Tsuyoshi does not specifically teach the underline section of the limitation “ estimates a depth from the coated surface of a foreign substance ” On the other hand in the same field of endeavor a method of polishing coating defects of NIIHARA teaches estimates a depth from the coated surface of a foreign substance ([0019], 5 th par., The depth of the defect in the concave defect or the height of the defect in the convex defect and the size of the coating defect are calculated, and the allowable polishing time for each coating defect site is calculated based on the line tact, and the result is imaged.) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method of calculating the dimension of the defect that includes depth and size of the defect taught by NIIHARA into Tsuyoshi. The suggestion/motivation for doing so would have been allows user of Tsuyoshi to follow the standardized procedures for automated defect characterization and volume-based severity assessment. As to claim 19, Tsuyoshi teaches the hardware processor estimates the depth from the coated surface of the foreign substance using a difference between a maximum value and a minimum value of the feature ( page 4 2 nd par., the amplitude of intensity modulation on the coated surface is calculated by the four-step phase-shifting method [3] as shown in Fig. 5. In this method, the amplitude Iamp(x, y) is calculated according to the equations below by obtaining the images I1 to I4 taken in the four patterns, each with a different initial phase δ of the illumination pattern by π/2.Thus the maximum and minimum amplitudes identify pixels that are too bright or too dark compared to their surroundings, and systems can highlight defect such as scratches, cracks, or spots based the maximum, minimum amplitudes and their surrounding area ). Claim 23 is rejected the same as claim 17 except claim 23 is directed to a method claim. Thus, argument analogous to that presented above for claim 17 is applicable to claim 23. Claim 25 is rejected the same as claim 19 except claim 25 is directed to a method claim. Thus, argument analogous to that presented above for claim 19 is applicable to claim 25. As to claim 28, both Tsuyoshi and NIIHARA teaches A non-transitory recording medium storing a program for causing a computer to execute (Tsuyoshi: computer simulation results shown in Fig.8 and Fig 9, where Fig.8 illustrate the computer simulation graphs of the appearance of phase and intensity modulation at defects of different images; NIIHARA A later-described polishing control program for determining and polishing is stored, and the RAM is provided with a memory for storing various data necessary for control such as coating defect data, see [0019] 5 th and 6 th pars., ); regarding the remaining limitations of claim 28, all the remaining limitations is rejected the same as claim 17 except claim 28 is directed to a computer programing claim. Thus, argument analogous to that presented above for claim 17 is applicable to remaining limitation of claim 28. Claim 30 is rejected the same as claim 17 except claim 30 is directed to a computer program claim. Thus, argument analogous to that presented above for claim 17 and 28 are applicable to claim 30. 4. Claim 22 is rejected r under 35 U.S.C. 103(a) as being unpatentable over Tsuyoshi, , in view of NIIHARA, JP H0569319 A, further in view of Matthews et al., ( hereafter Matthews), US 20030139836 A1, pub. 07/24/2003 As claim 22,Tsuyoshi teaches An automatic defect repair system (see Fig.6) comprising: an imaging device that images a plurality of images of an inspected site on a coated surface of a workpiece in a state where a bright and dark pattern of an illumination device that illuminates the inspected site is moved relative to the workpiece; the surface inspection device according to claim 17 (all this limitation discussed in claim 17) that acquires the plurality of images and estimates a depth from a coated surface of a foreign substance causing a surface defect ( this limitation is also discussed in claim 17 above ); however, it is noted that the combination of Tsuyoshi and NIIHARA does not teach “An automatic defect repair system comprising a repair device that repairs the surface defect based on an estimation result by the surface inspection device” On the other hand in the same field of endeavor a method of detecting and repairing paint defects on a vehicle body of Matthews teaches an automatic defect repair system comprising a repair device that repairs the surface defect based on an estimation result by the surface inspection device a repair device that repairs the surface defect based on an estimation result by the surface inspection device ( claims 4-5, 19, An assembly for automated paint defect detection and repair on a vehicle body, comprising: an optical system developing paint defect data by electronically imaging the vehicle body; a vision cell controller in communication with said optical wherein said paint defect data includes the size, location of said paint defects., the size and type of said paint defects). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the automated paint defect detection and repair apparatus taught by Matthews into Tsuyoshi. The suggestion/motivation for doing so would have been allows user of Tsuyoshi to repair defect such as defects on a gloss-coated surface, scratches, and sticking dirt of adhesive materials ( see Tsuyoshi section 4 page 5). 07-21 AIA 5. Claim s 20-21, 26-27 and 31-32 are rejected under 35 U.S.C. 103(a) as being unpatentable over Tsuyoshi, in view of NIIHARA, JP H0569319 A, further in view of TAKAHASHI, IPPE(hereafter TAKAHASHI), TW 201310012 A, pub., 03/01/2003 As to claim 20, Tsuyoshi teaches the feature is set, in a case where a set of pixels constituting a defect portion is defined as a defect region in images ( Figure 8 shows Each shot images in a particular phase and the appearance of phase and intensity modulation at defects ,, where the intensity describe the intensity of the pixels associated to the image ); however, it is noted that the combination of Tsuyoshi and NIIHARA does not teach “ obtained by binarizing the plurality of images, to the number of pixels in the defect region”. On the other hand TAKAHASHI teaches “ obtained by binarizing the plurality of images, to the number of pixels in the defect region ( page 5 3 rd par ., Preferably, the image processing unit includes a binarization step of binarizing the captured luminance image with a predetermined threshold value, and setting each pixel to a clear pixel or more below a threshold value. a dark pixel of the threshold value ; and a candidate defect extraction step, wherein a region in which a plurality of bright pixels are connected is a candidate defect region, and the detection step is in the direction in which the boundary lines are arranged, and the candidate defect region and the adjacent other The number of pixels between the candidate defect regions is counted, and when the number of pixels counted is equal to or less than a predetermined value, the relevant candidate defect region is set as a defect ). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a well-known method of binarizing an image taught by TAKAHASHI into Tsuyoshi. The suggestion/motivation for doing so would have been allows user of Tsuyoshi to maximize speed, accuracy, and computational efficiency, by simplifying the image, it highlights the region of interest (defects) and separates it from the background. Claim 26 is rejected the same as claim 20 except claim 26 is directed to a method claim. Thus, argument analogous to that presented above for claim 20 is applicable to claim 26. As to claim 21, TAKAHASHI the feature is set, in a case where a set of pixels constituting a defect portion is defined as a defect region in images obtained by binarizing the plurality of images, to a minimum value of a pixel value in the defect region ( page 5 3 rd par., the binarization step of binarizing the captured luminance image with a predetermined threshold value, and setting each pixel to a clear pixel or more below a threshold value a dark pixel of the threshold value. Thus, th e system identifies the absolute lowest brightness value within the flawed area (the darkest point) to set its threshold). Claim 27 is rejected the same as claim 21 except claim 27 is directed to a method claim. Thus, argument analogous to that presented above for claim 21is applicable to claim 27. Claim 31 is rejected the same as claim 20 except claim 31 is directed to a computer program claim. Thus, argument analogous to that presented above for claim 20 and 28 are applicable to claim 31. Claim 32 is rejected the same as claim 21 except claim 32 is directed to a computer program claim. Thus, argument analogous to that presented above for claim 21 and 28 are applicable to claim 32. Allowable Subject Matter 07-43 6. Claims 18, 24 and 29 are objected to as being dependent upon a rejected base claims but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claim. 7. Regarding claims 18, 24 and 29 no prior art is found to anticipate or render the following limitation obvious: “ wherein the hardware processor estimates the depth from the coated surface of the foreign substance using a value of a coefficient a when a quadratic curve of y = ax 2 + bx + c is fitted to the change in the feature with a frame number of the plurality of images represented as x and the feature represented as y.” Prior art document but not applied in the rejection “ Quality inspection of machined metal parts using an image fusion technique ” journal homepage: www.elsevier.com/locate/measuremen , pub. 2017, to S. Satorres Martínez et al. disclosed” This paper has been concerned with the automated inspection of machined metallic surfaces. An industrial Proto-Station (PS) based on computer vision has been developed. Particular attention has been paid to the lighting system, specifically designed for the PS, and the way in which lighting features have been included in the image processing. Multiple lighting sources have been utilized to ensure that, at least in one image, defects can be enhanced and subsequently extracted from the background. Images acquired under different lighting conditions are processed separately, labelled—following a predefined code—and merged into one. Features, obtained from this image, provide valuable information about the lighting technique involved in the defect detection . Additional features that incorporate geometrical and texture data have been added and also considered to distinguish between defective and non- defective regions. This distinction is performed by a supervised learning classifier in particular, an artificial neural network. The PS is assessed in terms of precision and repeatability and also compared with the achievements accomplished in previous works. According to the obtained results, the PS meets the requirements established in MSA which makes it suitable for industrial applications ( see section 5 and abstract). Contact Information Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday -Friday from 9:00AM to 6:50 PM Eastern Time. If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR. Status information for unpublished application is available through Privet PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217-919 (tool-free) /MEKONEN T BEKELE/Primary Examiner, Art Unit 2699 Application/Control Number: 18/719,976 Page 2 Art Unit: 2699 Application/Control Number: 18/719,976 Page 3 Art Unit: 2699 Application/Control Number: 18/719,976 Page 4 Art Unit: 2699 Application/Control Number: 18/719,976 Page 5 Art Unit: 2699 Application/Control Number: 18/719,976 Page 6 Art Unit: 2699 Application/Control Number: 18/719,976 Page 7 Art Unit: 2699 Application/Control Number: 18/719,976 Page 8 Art Unit: 2699 Application/Control Number: 18/719,976 Page 9 Art Unit: 2699 Application/Control Number: 18/719,976 Page 10 Art Unit: 2699
Prosecution Insights