Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 4, 13, 16-18 are rejected under 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) as being anticipated by Saphier et al. (US 11113803 B2), hereinafter Saphier.
Regarding claim 1, Saphier teaches A method for operating a machine vision system suitable for use with a microassembler system for inspection of assembly of micro-objects on a planar working surface, the method comprising: (Abstract see "Apparatus for inspection includes an imaging assembly, including a plurality of cameras, which are mounted in different, respective locations in the imaging assembly and are configured to capture respective images of a sample." Col. 4, Ln. 14-19 see "system 20 for automated optical inspection of a sample 22, in accordance with an embodiment of the present invention. In the illustrated embodiment, the sample is a flat panel display (FPD), which comprises a large glass substrate with appropriate circuit components formed on its upper surface."). providing a plurality of optical-image-capture modules (optical modules) arranged over, or under, a planar working surface, (Col. 7, Ln. 1-3 see "Cameras 40 are arrayed over the surface of sample 22, typically in one or more rows along the direction transverse to the scan."). wherein each module having a module field-of-view (FOV) and a module captured-image resolution both associated with a module FOV region on the planar working surface, (Col. 7, Ln. 3-5 see "Each camera 40 captures images of a respective field of view (FOV) 42." Col. 8, Ln. 6-11 see "This sensor is an array of 1280×1024 detector elements with a pitch of 5.2 μm. It is designed to output image data at roughly 40 megapixels/sec (giving full-frame output at 30 frames/sec). Assuming optics 58 are designed so that FOV has a width of about 6-10 mm, sensor 60 will give image resolution at the object plane of about 6-10."). and wherein each module in the plurality including a receiving optical train and a camera device optically coupled with the receiving optical train, the receiving optical train configured to couple light signals received from the module FOV region on the planar working surface to the camera device, (Col.7, Ln. 52-60 see "The beam emitted by the light source may be shaped by appropriate illumination optics 54 and is cast onto FOV 42 by objective optics 58 via a beamsplitter 56. Optics 58 form an image of the circuit components in FOV 42 onto an image sensor 60 via beamsplitter 56. The image sensor is connected via a suitable camera circuit 62 to image processor 36, which thus receives electronic images (in the form of analog signals or digital image data) from all of the cameras in imaging assembly 32."). and wherein a plurality of module FOV regions on the planar working surface, respectively associated with the plurality of modules, collectively forming a working FOV and a working captured image resolution both associated with a working FOV region on the planar working surface for a machine vision system; (Col. 7, Ln. 35-40 see "the swaths imaged by the individual cameras can be made to overlap in the X-direction so as to permit reliable registration between the images notwithstanding the loose motion tolerance. The number and placement of the cameras may be such that the entire width of the active area of sample 22 is covered." Col. 9, Ln. 11-13 see "the image processor combines the images into a single, large-scale picture of sample 22."). capturing by the camera device of each module in the plurality a respective module-captured image of a module FOV region on the planar working surface; (Col. 2, Ln. 29-32 see "an imaging assembly, including a plurality of cameras, which are mounted in different, respective locations in the imaging assembly and are configured to capture respective images of a sample." Col. 7, Ln. 6-8 see "Each camera captures multiple images as sample 22 progresses in the Y (scan) direction."). adjusting the module-captured image of at least one module in the plurality, wherein the adjusting includes adjusting a module-captured image resolution; (Col. 11, Ln. 30-34 see "the pixel data may be resampled on a common grid, with resolution finer than the pixel pitch of the individual images, and the resampled data may be interleaved or otherwise combined to give a single image with enhanced resolution."). and stitching together a plurality of module-captured images of adjacent modules in the plurality; (Col. 9, Ln. 8-10 see "Image processor 36 combines images 70, 72, 74, . . . , by stitching together neighboring images in the areas of overlap."). and forming, based on the plurality of module-captured images stitched together, the working FOV region associated with the working captured image resolution for the machine vision system, wherein the working FOV region is larger than each module FOV region (Col. 9, Ln. 11-13 see "the image processor combines the images into a single, large-scale picture of sample 22." Col. 7, Ln. 21-27 see "FOV 42 may therefore be considerably smaller than the outer dimensions of the camera itself, as illustrated in FIG. 2. Therefore, a single row of cameras along the X-direction will not be able to image the entire active area of the sample (i.e., the entire area on which circuit components are formed, possibly excluding the margins) in a single pass along the Y direction."). and the working captured image resolution is at least equal to or greater than each module-captured image resolution respectively associated with each module FOV region. (Col. 11, Ln. 30-34 see "the pixel data may be resampled on a common grid, with resolution finer than the pixel pitch of the individual images, and the resampled data may be interleaved or otherwise combined to give a single image with enhanced resolution.").
Regarding claim 2, Saphier teaches The method of claim 1, wherein the stitching comprises side-by-side stitching of module-captured images of adjacent module FOV regions to form a working captured image of the working FOV region. (Col. 9, Ln. 8-10 see "Image processor 36 combines images 70, 72, 74, . . . , by stitching together neighboring images in the areas of overlap." Col. 7, Ln. 1-3 see "Cameras 40 are arrayed over the surface of sample 22, typically in one or more rows along the direction transverse to the scan.").
Regarding claim 4, Saphier teaches The method of claim 1, wherein the stitching comprises staggered stitching of module-captured images of adjacent module FOV regions to form a working captured image of the working FOV region. (Col. 9, Ln. 8-10 see "Image processor 36 combines images 70, 72, 74, . . . , by stitching together neighboring images in the areas of overlap." Col. 7, Ln. 31-34 see "cameras 40 are arrayed in imaging assembly 32 in multiple rows, as shown in FIG. 2, wherein the cameras in each row are offset in the X-direction relative to the other rows.").
Regarding claim 13, Saphier teaches The method of claim 1, wherein adjusting the module-captured image comprises performing super-resolution imaging on sets of sub-pixels in the module-captured image to produce a higher resolution image of the module-captured image. (Col. 11, Ln. 24-35 see "the offset between successive images will typically not be a whole number of pixels. Thus, in the region of overlap between images 80a, 80b, 80c and 80d (or between any other four successive images), each point on sample 22 is captured in four different, slightly offset pixels. Using the known, sub-pixel offset between the four images, the pixel data may be resampled on a common grid, with resolution finer than the pixel pitch of the individual images, and the resampled data may be interleaved or otherwise combined to give a single image with enhanced resolution (commonly referred to as super resolution).").
Regarding claim 16, Saphier teaches The method of claim 1, wherein the adjusting the module-captured image comprises performing pixel shifting on sets of sub-pixels in the module-captured image to produce a higher resolution image of the module-captured image. (Col. 11, Ln. 24-35 see "the offset between successive images will typically not be a whole number of pixels. Thus, in the region of overlap between images 80a, 80b, 80c and 80d (or between any other four successive images), each point on sample 22 is captured in four different, slightly offset pixels. Using the known, sub-pixel offset between the four images, the pixel data may be resampled on a common grid, with resolution finer than the pixel pitch of the individual images, and the resampled data may be interleaved or otherwise combined to give a single image with enhanced resolution (commonly referred to as super resolution)." Col. 9, Ln. 49-50 see "For each camera, the image processor computes an estimated shift (X,Y) and angular skew." Col. 9, Ln. 8-11 see "Image processor 36 combines images 70, 72, 74, . . . , by stitching together neighboring images in the areas of overlap in order to determine the exact shift of each image relative to a given reference point." Col. 2, Ln. 33-35 see "a motion assembly, which is configured to move at least one of the imaging assembly and the sample so as to cause the imaging assembly to scan the sample.").
Regarding claim 17, Saphier teaches The method of claim 1, wherein the adjusting the module-captured image comprises performing a combination of at least two image processing methods selected from the following list of image processing methods: using a micro-lens array to create a foveal region increasing a native resolution in a neighborhood of desired device position datums or locations in the module-captured image; performing grayscale imaging to detect a centroid of each of at least one micro-object in the module-captured-image and adjusting a position of the at least one micro-object on the planar working surface; performing super-resolution imaging on sets of sub-pixels in the module-captured image to produce a higher resolution image of the module-captured image; (Col. 11, Ln. 24-35 see "the offset between successive images will typically not be a whole number of pixels. Thus, in the region of overlap between images 80a, 80b, 80c and 80d (or between any other four successive images), each point on sample 22 is captured in four different, slightly offset pixels. Using the known, sub-pixel offset between the four images, the pixel data may be resampled on a common grid, with resolution finer than the pixel pitch of the individual images, and the resampled data may be interleaved or otherwise combined to give a single image with enhanced resolution (commonly referred to as super resolution)."). or performing pixel shifting on sets of sub-pixels in the module-captured image to produce a higher resolution image which is higher than a native resolution of the module-captured image. (Col. 11, Ln. 24-35 see "the offset between successive images will typically not be a whole number of pixels. Thus, in the region of overlap between images 80a, 80b, 80c and 80d (or between any other four successive images), each point on sample 22 is captured in four different, slightly offset pixels. Using the known, sub-pixel offset between the four images, the pixel data may be resampled on a common grid, with resolution finer than the pixel pitch of the individual images, and the resampled data may be interleaved or otherwise combined to give a single image with enhanced resolution (commonly referred to as super resolution)." Col. 9, Ln. 49-50 see "For each camera, the image processor computes an estimated shift (X,Y) and angular skew." Col. 9, Ln. 8-11 see "Image processor 36 combines images 70, 72, 74, . . . , by stitching together neighboring images in the areas of overlap in order to determine the exact shift of each image relative to a given reference point." Col. 2, Ln. 33-35 see "a motion assembly, which is configured to move at least one of the imaging assembly and the sample so as to cause the imaging assembly to scan the sample.").
Regarding claim 18, Saphier teaches The method of claim 1, wherein a width of an overall working FOV region on the planar working surface is at least ten times larger than a width of any one module FOV region in the plurality of module FOV regions on the planar working surface, and a resolution of the captured-image of the overall working FOV region is at least equal to or greater than the resolution of any module FOV captured image from the plurality of module FOV regions on the planar working surface. (Col. 4, Ln. 19-21 see "The dimensions of the glass substrates that are currently used in manufacturing FPDs may be as large as 246×216 cm." Col. 8, Lin. 9-10 see "optics 58 are designed so that FOV has a width of about 6-10 mm." (200cm is 200 times larger than 10mm.) Col. 11, Ln. 30-34 see "the pixel data may be resampled on a common grid, with resolution finer than the pixel pitch of the individual images, and the resampled data may be interleaved or otherwise combined to give a single image with enhanced resolution.").
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Saphier et al. (US 11113803 B2), hereinafter Saphier, in view of Ryu et al. (KR 20230127659 A), hereinafter Ryu.
Regarding claim 3, Saphier teaches The method of claim 1. wherein the stitching comprises stitching of module-captured images of adjacent module FOV regions to form a working captured image of the working FOV region. (Col. 9, Ln. 8-10 see "Image processor 36 combines images 70, 72, 74, . . . , by stitching together neighboring images in the areas of overlap." Col. 9, Ln. 15-18 see "To stitch the pictures together, the image processor may, for example, use a digital filter that is matched to the shape of an image feature appearing in the overlap area.").
Saphier does not teach feathered stitching.
However, Ryu teaches feathered stitching (Para. 14 see "In addition, the image stitching unit performs stitching by blending an overlap area, which is an overlapping area between the captured images placed on the three-dimensional space region, and the overlap is measured by performing a raycast from the origin of the three-dimensional space region. Blending between the captured images can be performed by measuring the planar distance between the captured images in an area and adjusting the slope of the overlap area of the captured images based on the measured plane distance.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saphier to incorporate the teachings of Ryu to stitch the images together by feathering or "blending" the overlapping areas. Doing so would predictably increase the quality of the stitched image by reducing sharp edges that would appear if one image was placed over the other.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Saphier et al. (US 11113803 B2), hereinafter Saphier, in view of Hagen et al. (US 6003223 A), hereinafter Hagen.
Regarding claim 5, Saphier teaches The method of claim 4.
Saphier does not teach wherein the staggered stitching comprises a stitching geometry based on a step-and-repeat assembly process to stitch module-captured images of adjacent module FOV regions.
However, Hagen teaches wherein the staggered stitching comprises a stitching geometry based on a step-and-repeat assembly process to stitch module-captured images of adjacent module FOV regions. (Col. 2, Ln. 13-18 see "The common alignment target is placed in the field stitch area 28 between two adjacent fields 28A 28B. See FIGS. 6A & 6B. The stepper alignment system uses the wafer alignment target 42 placed in the field stitch area between two adjacent fields and the alignment target for that particular field to align the reticle." Col. 4, Ln 10-24 see "4. FIG. 5A shows using a stepper, exposing through a stepper aperture the first slider area 12A with a first reticle image field 70A. The first reticle image field 70A has spaced first and second reticle alignment keys 72A 74A. The first alignment key 72A is aligned with the first wafer alignment target 40 and the second reticle alignment key 74A is aligned with the center alignment target 42. 5. FIG. 5B shows stepping and exposing the second slider area 12B through the stepper aperture with a second reticle image field 70B. The second reticle image field 70B having spaced first and second alignment targets 72B 74B. The first reticle alignment key 72B is aligned with the center wafer alignment target 42 and the second reticle alignment key 74B is aligned with the second wafer alignment target 44.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saphier to incorporate the teachings of Hagen to perform stitching in a step-and-repeat process. Doing so would predictably allow each image captured by each module to be stitched together to form the overall image.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Saphier et al. (US 11113803 B2), hereinafter Saphier, in view of Thiele et al.: "3D-printed eagle eye: Compound microlens system for foveated imaging", Researchgate, submitted February 2017, [retrieved on 12-9-2025]. Retrieved from the internet <https://www.researchgate.net/publication/313784716_3D-printed_eagle_eye_Compound_microlens_system_for_foveated_imaging>, hereinafter Thiele.
Regarding claim 9, Saphier teaches The method of claim 1.
Saphier does not teach wherein the adjusting the module-captured image comprises using a micro-lens array to create a foveal region increasing a native resolution in a neighborhood of desired device position datums or locations in the module-captured image.
However, Thiele teaches wherein the adjusting the module-captured image comprises using a micro-lens array to create a foveal region increasing a native resolution in a neighborhood of desired device position datums or locations in the module-captured image. (Pg. 3, Col. 2, Para. 1 see "Because the chip does not perfectly record the image, there is a notable difference in quality if it is compared to the imaging on a glass slide. This effect can be explained by the chip plane being not perfectly aligned with the focal plane and by the fact that the microlens array on the chip, which is important for the imaging performance, was removed before 3D printing. After creating the foveated images (Fig. 4D), the imaging resolution is considerably increased toward the center of the images.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saphier to incorporate the teachings of Thiele to use a micro-lens array to create a foveal region to increase a native resolution. Doing so would predictably increase the resolution of a region in the image thereby allowing a higher accuracy in detecting faults on the planar surface.
Claims 10, 12 are rejected under 35 U.S.C. 103 as being unpatentable over Saphier et al. (US 11113803 B2), hereinafter Saphier, in view of Kasaya et al.: "Image-based autonomous micromanipulation system for arrangement of spheres in a scanning electron microscope", AIP Publishing, submitted 1 June 2004, [retrieved on 12-9-2025]. Retrieved from the internet <https://pubs.aip.org/aip/rsi/article/75/6/2033/452652/Image-based-autonomous-micromanipulation-system>, hereinafter Kasaya.
Regarding claim 10, Saphier teaches The method of claim 1. wherein the adjusting the module-captured image comprises performing grayscale imaging to detect each of at least one micro-object in the module-captured-image of the module FOV region on the planar working surface (Col. 9, Ln. 31-37 see "By precisely matching location coordinates of the feature in question in both overlapping images, the image processor is able to register the images in the combined picture. (As noted earlier, the term “feature” should be interpreted broadly to refers to any and all recognizable characteristics of the pixel values in the overlap region, including gray-scale gradations.").
Saphier does not teach a centroid of and adjusting a position of the at least one micro-object on the planar working surface in a micro-assembly process.
However, Kasaya teaches a centroid of (Pg. 5, Section A. Micromanipulation system, Para. 1 see "has a configuration that the probe tip is always located at the center of the field of view to guarantee the consistent acquisition of well-focused images of the manipulation."). and adjusting a position of the at least one micro-object on the planar working surface in a micro-assembly process. (Pg. 9, Col. 2, Para. 1 see "Every time the probe is rotated by a unit angle, the probe and the sphere are detected and the position is corrected." Pg. 9, Col. 2, Para. 2 see "the position of the sphere sometimes shifts away from the target position because of the instability in the rotation center of the probe. If the error exceeds a certain tolerable value, the substrate is lowered and the position of the sphere is corrected.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saphier to incorporate the teachings of Kasaya to detect the central location of the micro-object in the image and adjust its position. Doing so would predictably allow the system to accurately determine an adjustment that must be made to the micro-object as well as perform the adjustment by locating the position of the micro-object.
Regarding claim 12, Saphier in view of Kasaya teaches The method of claim 10.
Saphier does not teach wherein the at least one micro-object comprises a plurality of micro-objects, and adjusting the position of the at least one micro-object includes performing rough alignment of the plurality of micro-objects to each other in the micro-assembly process.
However, Kasaya teaches wherein the at least one micro-object comprises a plurality of micro-objects, and adjusting the position of the at least one micro-object includes performing rough alignment of the plurality of micro-objects to each other in the micro-assembly process. (Pg. 10, Col. 1, Fig. 8 see "FIG. 8. Autonomous arrangement of five microspheres into a linear lattice realized by the developed system. The target arrangement given by the operator. In this model, the spheres are displayed as circles with a diameter of 25 mm. The result of the automatic arrangement." (Any collection of micro-objects can be considered a micro-object comprising a plurality of micro-objects.)).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saphier to incorporate the teachings of Kasaya to perform alignment on micro-objects relative to each other. Doing so would predictably increase quality of the overall object being assembled by ensuring its parts are positioned correctly.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Saphier et al. (US 11113803 B2), hereinafter Saphier, in view of Kasaya et al.: "Image-based autonomous micromanipulation system for arrangement of spheres in a scanning electron microscope", AIP Publishing, submitted 1 June 2004, [retrieved on 12-9-2025]. Retrieved from the internet <https://pubs.aip.org/aip/rsi/article/75/6/2033/452652/Image-based-autonomous-micromanipulation-system>, hereinafter Kasaya, and Cappelleri et al.: "Caging micromanipulation for automated microassembly", IEEE, submitted 18 August 2011 , [retrieved on 12-9-2025]. Retrieved from the internet <https://ieeexplore.ieee.org/document/5980190>, hereinafter Cappelleri.
Regarding claim 11, Saphier in view of Kasaya teaches The method of claim 10.
Saphier does not teach wherein adjusting the position of the at least one micro-object includes rotation of the micro-object on the planar working surface in the micro-assembly process.
However, Cappelleri teaches wherein adjusting the position of the at least one micro-object includes rotation of the micro-object on the planar working surface in the micro-assembly process. (Pg. 2, Section B. Approach, Para. 1 see "A rotation primitive follows to orientate the part to the final goal orientation.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saphier in view of Kasaya to incorporate the teachings of Cappelleri to rotate the micro object. Doing so would predictably increase quality of the overall object being assembled by ensuring its parts are oriented correctly.
Allowable Subject Matter
Claim(s) 6-8, 14-15, 19-20 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Bower et al. (US 9444015 B2) discloses a technology that provides micro-assembled micro-LED displays and lighting elements that use arrays of micro-LEDs that are too small, numerous, or fragile to assemble by conventional means.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER J VAUGHN whose telephone number is (571) 272-5253. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDREW MOYER can be reached on (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER JOSEPH VAUGHN/Examiner, Art Unit 2675
/EDWARD PARK/Primary Examiner, Art Unit 2675