DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 3/12/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Claim Objections
Claim 7 is objected to because of the following informalities: Claim 7 depends upon claim 6 which states “the inspection item includes a color, a surface shape, and a gloss”. Claim 7 goes on to separate these into different categories which implies they are not all included in the inspection item as claim 6 states. “set the reference number as one or more in a case where the inspection item is the gloss, and set the reference number as three or more in a case where the inspection item is the color or the surface shape” Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 and 5-13 are rejected under 35 U.S.C. 103 as being unpatentable over Hyatt (U.S. Patent Pub. No. 2022/0044379) in view of Onodera (U.S. Patent Pub. No. 2014/0211404).
Regarding Claim 1, Hyatt teaches an image processing apparatus comprising:
at least one processor; and at least one memory that is in communication with the at least one processor, wherein the at least one memory stores instructions for causing the at least one processor and the at least one memory to (¶38 Exemplary system 100, which may be used for automated visual inspection of an item on an inspection line, includes a processor 102 in communication with one or more camera(s) 103 and with a device, such as a user interface device 106 and/or other devices, such as storage device 108:)
acquire captured images obtained by imaging an object (Fig. 2B; ¶62 an image of an item surrounded by a bounding shape is displayed to a user (202) and initial user confirmation is received (204); ¶63 Additional images of same-type objects are then acquired (206))
determine whether a number of the acquired captured images is a reference number for an inspection of the object (Fig. 2B; ¶63 Once a predetermined or desired amount of additional images are acquired (207);)
in a case where it is determined that the number of the acquired captured images is the reference number, detect a defect in the object based on the acquired captured images; and (¶63 Once a predetermined or desired amount of additional images are acquired (207) or once a predetermined time has passed, images of suspected items (e.g., images of items determined to differ from the defect-free items in the reference image database) are displayed to the user (208) to obtain user confirmation or correction; User then determines if images contain defects.)
in a case where it is determined that the number of the acquired captured images is not the reference number, output a warning (Hyatt does not explicitly disclose sending a warning, it just repeats the step of obtaining images. One with ordinary skill in the art could use this predetermined evaluation number not being met to send a warning. See Onodera below.)
Hyatt does not explicitly disclose in a case where it is determined that the number of the acquired captured images is not the reference number, output a warning.
Onodera is in the same field of art of image analysis. Further, Onodera teaches in a case where it is determined that the number of the acquired captured images is not the reference number, output a warning (¶111 when the acquired maximum evaluation value (number of images taught by Hyatt) is determined to be the prescribed value or more (step S8, YES), the control unit 22 displays a warning message on the through image displayed at the display unit 26 (liquid crystal monitor screen 4).)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Hyatt by outputting a warning when a predetermined value (number of images taught by Hyatt) is met that is taught by Onodera; thus, one of ordinary skilled in the art would be motivated to combine the references to prevent erroneously capturing images (Onodera ¶9).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Hyatt in view of Onodera discloses the image processing apparatus according to claim 1, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to output a result of the inspection (Hyatt ¶69 once user confirmation is received the system may switch temporarily to inspection mode in order to simulate parameters of inspection (e.g., camera and/or illumination parameters). Namely, processor 102 may apply inspection algorithms on images of the object in response to receiving user confirmation. The optimal image may then be identified while using parameters of inspection, which will ensure that the optimal image which will be used to update the reference database, will be similar to images that will be obtained during the inspection stage, thereby providing a more accurate and efficient visual inspection process.)
Regarding Claim 5, Hyatt in view of Onodera discloses the image processing apparatus according to claim 1, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to set the reference number in accordance with an inspection item (Hyatt, ¶63 Additional images of same-type objects are then acquired (206), for example, enough images are acquired to obtain a full representation of an item (as described above). Once a predetermined or desired amount of additional images are acquired (207).)
Regarding Claim 6, Hyatt in view of Onodera discloses the image processing apparatus according to claim 5, wherein the inspection item includes a color, a surface shape, and a gloss (Hyatt, ¶64 objects may have texture (gloss), pattern or color differences on the object surface, which are not considered to be defects by a user. Also, same-type objects may have permitted shape tolerances. For example, an object may be imaged from different angles and/or at different locations on the inspection line without showing perspective distortion and may thus be within the permitted shape tolerance. Due to these and other permitted differences objects may be determined to be defected or with suspected defects, even though they are not considered defective by the user.)
Regarding Claim 7, Hyatt in view of Onodera discloses the image processing apparatus according to claim 6, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to set the reference number as one or more in a case where the inspection item is the gloss, and set the reference number as three or more in a case where the inspection item is the color or the surface shape (See objection above. This is a simple design choice a user with ordinary skill in the art could make. Hyatt, ¶31 “Enough set up images” are collected when an essentially complete representation of a type of item is achieved. For example, when enough images are collected to enable determining the spatial range in which each reference image can be used as a distortion-less reference, as described above. Analysis of the set up images may be performed to collect information regarding possible 2D shapes and 3D characteristics (e.g., rotations on the inspection line) of an item or to find uniquely discriminative features of the item and the spatial relation between these unique features, as preserved between the set up images.)
Regarding Claim 8, Hyatt in view of Onodera discloses the image processing apparatus according to claim 1, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to, in a case where it is determined that the number of the images captured acquired is not the reference number, output a warning indicating an inability to conduct the inspection (Onodera, ¶111 when the acquired maximum evaluation value (number of images taught by Hyatt) is determined to be the prescribed value or more (step S8, YES), the control unit 22 displays a warning message on the through image displayed at the display unit 26 (liquid crystal monitor screen 4); ¶115 A message such as "it is not possible to perform automatic image capture correctly in the current photographing environment"; Therefore the inspection process would not be able to be conducted.)
The reasons for combining Hyatt and Onodera are similar to those stated in the rejection of claim 1. In addition, this same reasoning is pertinent and applicable to the rejections of claims 9-10 and 13 below.
Regarding Claim 9, Hyatt in view of Onodera discloses the image processing apparatus according to claim 1, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to, in a case where the number of the images acquired captured is greater than the reference number and an imaging unit performs continuous imaging, output a warning indicating an abnormality in the imaging unit and displays an indication for prompting an inspection or a recovery of the imaging unit (Onodera, ¶114 When the maximum evaluation value is greater than or equal to the prescribed value decided in advance (step S8, YES), after a warning is issued to the user (step S9), step S2 is returned to, and the user is invited to reset the auto shutter area (recovery of imaging unit).)
Regarding Claim 10, Hyatt in view of Onodera discloses the image processing apparatus according to claim 1, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to, in a case where the number of the acquired captured images is greater than the reference number, delete excessive images (This is a simple design choice a user with ordinary skill in the art could implement for computational efficiency. Onodera, ¶127 After additional storage of a prescribed number (for example, 10) of frame images to the DRAM 21 is complete, the oldest frame image of the plurality of frame images stored in the DRAM 21 is deleted every time a frame image is acquired and the newly acquired frame image is stored.)
Regarding Claim 11, Hyatt in view of Onodera discloses the image processing apparatus according to claim 1, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to control a conveyance unit to convey the object corresponding to the warning distinguishing the object from an inspected object (Hyatt, ¶43 Camera(s) 103, which are configured to obtain an image of an inspection line, are typically placed or positioned in relation to an inspection line 105 (e.g., a conveyer belt), such that items 104 placed on the inspection line 105 are within the FOV 103′ of the camera 103; ¶45 Processor 102 receives image data (which may include data such as pixel values that represent the intensity of reflected light as well as partial or full images or videos) of objects on the inspection line 105 from the one or more camera(s) 103 and runs processes according to embodiments of the invention.)
Regarding claim 12, claim 12 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Hyatt further teaching on: An image processing method comprising (Hyatt, ¶1 The present invention relates to automated visual inspection processes, for example, inspection of items during a production process.)
Regarding claim 13, claim 13 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Onodera further teaching on: A non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform an image processing method (Onodera, Claim 3: A non-transitory computer-readable recording medium storing a program which controls a computer of an image capturing device)
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Hyatt (U.S. Patent Pub. No. 2022/0044379) in view of Onodera (U.S. Patent Pub. No. 2014/0211404) in view of Yamagami (U.S. Patent Pub. No. 2020/0320680).
Regarding Claim 3, Hyatt in view of Onodera teaches the image processing apparatus according to claim 1, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to (see claim 1.)
Hyatt in view of Onodera does not explicitly disclose detect a defect in the object based on an image obtained by combining the acquired captured image.
Yamagami is in the same field of art of image analysis. Further, Yamagami teaches detect a defect in the object based on an image obtained by combining the acquired captured images (¶219 since the plurality of captured images can be combined by the imaging device 3 conformable to the standardization standard, it is possible to perform the generation of the inspection image using the principle of the photometric stereo or the deflectometry and the multispectral imaging, for example.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Hyatt in view of Onodera by inspecting the image by combining the plurality of images that is taught by Yamagami; thus, one of ordinary skilled in the art would be motivated to combine the references to improve accuracy of image inspection (Yamagami ¶220).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 4, Hyatt in view of Onodera in view of Yamagami discloses the image processing apparatus according to claim 3, wherein the at least one memory further stores instructions for causing the at least one processor and the at least one memory to combine the acquired captured images by using a photometric stereo method (Yamagami, ¶219 since the plurality of captured images can be combined by the imaging device 3 conformable to the standardization standard, it is possible to perform the generation of the inspection image using the principle of the photometric stereo or the deflectometry and the multispectral imaging, for example.)
The reasons for combining Hyatt and Onodera are similar to that stated in the rejection of claim 3.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
Prosecution Insights