DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of this application being a National Stage of the International Application No. PCT/US2022/015169, filed on February 4, 2022.
Information Disclosure Statement
The information disclosure statement (“IDS”) filed on July 24, 2024 was reviewed and the listed references were noted.
Drawings
The 6-page drawings have been considered and placed on record in the file.
Status of Claims
Claims 1-15 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5 and 9-13 are rejected under 35 U.S.C. 103 as being unpatentable over Gutierrez et al. (“Synthetic Training Data Generation for Deep Learning based Quality Inspection” – IDS) in view of Dal Mutto et al. (US 2019/0096135).
Consider Claim 1, Gutierrez discloses “A method comprising: generating, by an anomaly texture generator, first (Gutierrez, Fig. 3, Texture and defect generation; and Section 3.2 discloses: “First, we build a texture dictionary from scanned normal samples (Figure 3a). Next, we manually choose small texture patches which were not included in the dictionary: they are called seed points (Figure 3b, first column)”) “obtaining 3D models of objects associated with the first (Gutierrez, Section 3.1 discloses obtaining a 3D model, creating 3D reconstructed shape and creating a reconstructed normal maps as bump maps (fake details) to texture 3D models, see also Fig. 2b-2d. Section 3.3 discloses defect generation on the synthetic images. This section also discloses “Because we use bump maps35 (fake details) for part texturing and displacement maps (true details) for defect generation, the two types of maps do not impede each other”); “and training an anomaly segmentation network, to detect anomalies, with the first synthetic images” (Gutierrez, Section 3.4, where defect detection with deep learning using either segmentation or object detection is being disclosed; and Section 4.1 discloses the training procedure for the anomaly detection is disclosed using generated simulated image containing at least one defect). Although Gutierrez discloses in its synthetically created images, by capturing fine-grained texture details of the model (Gutierrez, Section 3.1) and it also discloses “To synthesize new textures of typical casting surface finish, we apply a dictionary-based approach of exemplar-based inpainting.36,37 This technique is especially well-suited for situations where only few texture scans are available” (Gutierrez, Section 3.2 Texture Randomization), it does not explicitly disclose that its defect training and detection relies on “color texture” of a surface. However, in an analogous field of endeavor, Dal Mutto discloses “the defects can be detected and characterized in the extent or magnitude of the differences in geometric shape, geometric dimensions and sizes, surface texture and color from a known good (or “reference” sample) or other based on similarity to known defective samples” (Emphasis added) (Dal Mutto, Paragraph [0168]).
Accordingly, it would have been obvious to a person of ordinary skill, before the effective filing date of the instant application, to combine Gutierrez with the teachings of Dal Mutto to establish a training system for detection of surface anomaly based on the color texture of the generated synthetic images. One of ordinary skill in the art would have been motivated to substitute the surface texture taught by Gutierrez with the color as taught by Dal Mutto to obtain the training anomaly network to detect anomalies with the synthetic images. Accordingly, the combination of Gutierrez and Dal Mutto discloses the invention of Claim 1.
Consider Claim 2, the combination of Gutierrez and Dal Mutto discloses “The method as recited in claim 1, the method further comprising: capturing a real image of a target object; inputting the real image of the target object into the anomaly segmentation network; detecting, by the anomaly segmentation network, at least one anomaly on a surface of the target object” (Gutierrez, Section 4.1 Training and Evaluation Data discloses: “To evaluate the model in real life conditions, we construct a real test set of 700 images (468 defectives, 232 healthy) using different physical parts than the ones used to create the real training set.” And, Section 4.2, Experiments and Results: Table 1ig. 5 shows the curve for threshold visibility and Fig. 8 shows that regardless of the visibility, the target tissue is highlighted in all subsequent images).
Consider Claim 3, the combination of Gutierrez and Dal Mutto discloses “The method as recited in claim 2, wherein the at least one anomaly defines a stain or unclean portion of the target object” (Dal Mutto discloses features such as defects of blemished surfaces may be detected in the defect detection system, “a blemish surface” is interpreted as a stain on the surface). The proposed combination as well as the motivation for combining the Gutierrez and Dal Mutto references presented in the rejection of Claim 1, apply to Claim 3 and are incorporated herein by reference. Thus, the method recited in Claim 3 is met by Millioni and St. Pierre.
Consider Claim 4, the combination of Gutierrez and Dal Mutto discloses “The method as recited in claim 2, wherein the target object is not one of the objects defined by the first synthetic images” (Gutierrez, Section 4.1 Training and Evaluation Data discloses: “In total, we generated 9530 simulated images of parts from our client, which are split into a train (8576 images) and a test (954 images) set with a 9:1 ratio.” And, “To evaluate the model in real life conditions, we construct a real test set of 700 images (468 defectives, 232 healthy) using different physical parts than the ones used to create the real training set).
Consider Claim 5, the combination of Gutierrez and Dal Mutto discloses “The method as recited in claim 1, the method further comprising: obtaining, by the rendering module, second color texture images associated with the 3D models, the second color texture images including no surface anomalies so as to define non- anomalous color texture images; and based on the 3D models and the second color texture images associated with the 3D models, the rendering module generating second synthetic images of the respective objects, the second synthetic images defining the objects in a realistic scene, the objects of the second synthetic images each including no surface anomalies so such that the second synthetic images define non-anomalous synthetic images.” (Gutierrez, Figs. 2b-2d and Section 3.1 discloses “To compute surface normal maps from the captured image stacks, we use the photometric stereo library available in Substance Designer tool (Figure 2b-2d, second row). The reconstructed normal maps are used as bump maps35 to texture the 3D models (Figure 2b-2d, third row)”).
Claims 9-13 recite systems with elements corresponding to the steps of the methods recited in Claims 1-5, respectively. Therefore, the recited elements of these claims are mapped to the proposed combination in the same manner as the corresponding steps in their corresponding method claims. Additionally, the rationale and motivation to combine the Gutierrez and Dal Mutto references, presented in rejection of Claim 1, apply to these claims. In addition, the combination of Gutierrez and Dal Mutto discloses a memory and a processor to act on the instructions stored in the memory (see for example, Dal Mutto, Fig. 2:108 and 110).
Allowable Subject Matter
Claims 6-8 and 14-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: none of the cited prior art, alone or in combination, provides a motivation to teach the ordered combination of the recited limitations of these claims with the limitations of the claims to which they depend.
Conclusion and Contact Information
The prior art made of record and not relied upon are considered pertinent to Applicant’s disclosure: Kadambi et al. (US 2021/0264147), Paragraphs [0050] and [0148]); and Erol et al. (US 2023/0230224), Figs. 1 and 8.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Siamak HARANDI whose telephone number is (571)270-1832. The examiner can normally be reached Monday - Friday 9:30 - 6:00 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights