DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending.
Claim Objections
Claim(s) 9 and 10 is/are objected to because of the following reasons:
Claim 9 ends with a semicolon “;”. It should always end with a period “.”.
Claim 10 ends with nothing. It should always end with a period “.”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1, 5-11 and 15-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fridge (US5638461A) in view of Pentex et al (US20220276359A1)).
Regarding claims 1, 11 and 18, Fridge teaches a system comprising:
a lighting apparatus comprising a plurality of lights;
a calibration object having a shape with a size;
(Fridge, "The present invention is directed to a three-dimensional optical inspection system", [c1:65-end]; "an accurately known artwork image of cross-hatch lines", [c2:15-20]; teaches a calibration object with a known shape and size used for calibrating the system)
a camera aimed at the conveyor belt;
(Fridge, "high-resolution inspection cameras 10, 12", [c2:50-55]; "actual display device 14 moving on belt 18", [c2:50-55]; teaches cameras aimed at a moving belt/conveyor)
a computer system comprising at least one hardware processor and a non-transitory computer-readable medium storing instructions that, when executed by the at least one hardware processor, perform operations comprising:
(Fridge, "stored in one or more electronic memories or "frame grabbers" and then fed to a pre-programmed computer system.", [c3:20-25]; teaches a computer system with memories (medium) executing pre-programmed operations)
adjusting lighting from the lighting apparatus and taking a plurality of images from the camera of the calibration object under different lighting conditions;
(Fridge, "turn on or off a "bezel illuminator" light source, if necessary.", [c4:5-10]; "A plurality of cameras are mounted in a spaced arrangement", [c2:1-5]; Pentex, "produce images J1, J2", [0042]; Fridge teaches lighting adjustment and hardware setup. Specifically, it discloses turning a "bezel illuminator" light source on or off and utilizing a plurality of cameras in a spaced arrangement; Pentex teaches the capture of multiple sequential images, specifically producing plural images (e.g., "images J1, J2"))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Pentex into the system or method of Fridge in order to actively adjust the state of the lighting source while utilizing the cameras to sequentially capture plural images of the object under the manipulated conditions. The combination of Fridge and Pentex also teaches other enhanced capabilities.
The combination of Fridge and Pentex further teaches:
based on timestamps associated with each image of the plurality of images, organizing the plurality of images in chronological order;
(Pentex, "The data captured by the various devices constituting the measuring system 5 (images, distance and orientation of the beam, location, attitude) are all identified temporally by way of time-stamping information", [0044]; incorporating Pentex into Fridge would reliably track and chronologically correlate the image frames captured as the target object moves dynamically through the inspection space)
passing the plurality of images into a camera calibration function to obtain a camera matrix, distortion coefficients, and rotational vectors;
(Fridge, "Calibration software, using an accurately known artwork image, has established the position and perspective distortion associated with each camera.", [c2:55-60]; Pentex, "translation vector T36 ... rotation matrix R36", [0048]; incorporating Pentex into Fridge would mathematically map the optical perspectives using robust extrinsic components, thereby providing complete rotation, translation, and distortion parameters for the cameras)
computing a distance between the calibration object and the camera using the rotation vectors and the size of the shape in the calibration object;
(Fridge, "compute and correct for angular and distance misalignments", [Abstract]; "calculate the three-dimensional coordinates", [c3:60-65]; using the captured perspectives of the known-sized artwork to compute the 3D distances/coordinates)
computing speed and direction of the calibration object using the camera matrix, distortion coefficients, and rotational vectors and the plurality of images;
(Pentex, "vehicle 1 moves along a trajectory t", [0042]; "The location and attitude data supplied by the unit 6 make it possible to identify the position and angle of the measuring system 5 in a georeferenced frame of reference G.", [0047]; incorporating Pentex into Fridge would lead to utilize the time-stamped images combined with the rotational vectors and matrices to compute the spatial trajectory (speed and direction) of the continuously moving belt/object)
using the computed direction and the computed speed to modify the plurality of images so that the calibration object in all of the images are aligned in a single plane.
(Fridge, "The complete three-dimensional map of the display surface is then rotated by the software to create an image of the UUT display image as it would appear to an "ideal observer" viewing the display from a perfectly centered location", [c3:6-end]; mathematically rotating/modifying the captured images across the motion path to align them to a single, perfectly centered standard viewing plane, independent of their actual moving orientation)
Regarding claims 5 and 15, the combination of Fridge and Pentex teaches its/their respective base claim(s).
The combination further teaches the system of claim 1, wherein the operations further comprise: for each pair of successive images in the plurality of images:
performing perspective warping on a first image of the pair of successive images to simulate a top-down view of the calibration object, based on the rotational vectors.
(Fridge, "The complete three-dimensional map of the display surface is then rotated by the software to create an image of the UUT display image as it would appear to an "ideal observer" viewing the display from a perfectly centered location at a standard distance.", [c2:60-end]; mathematically rotating (perspective warping) the captured 3D map/images of the moving object to simulate a perfectly centered, ideal top-down viewpoint independent of the object's actual orientation)
Regarding claims 6 and 16, the combination of Fridge and Pentex teaches its/their respective base claim(s).
The combination further teaches the system of claim 1, wherein the operations further comprise creating a three-dimensional offset vector using the aligned images.
(Fridge, "compute a correction function which compensates for misalignment in angle and/or distance between the inspection system and the UUT", [c1:40-45]; Pentex, "translation vector T36 and a rotation matrix R36", [0048]; incorporating Pentex into Fridge would represent the computed 3D distance and angular misalignment corrections as defined translation (offset) vectors and rotation matrices to align the views)
Regarding claims 7 and 17, the combination of Fridge and Pentex teaches its/their respective base claim(s).
The combination further teaches the system of claim 6, wherein the operations further comprise:
causing a part to analyze to pass under the camera;
as the part is passing under the camera, adjusting lighting from the lighting apparatus and taking a second plurality of images from the camera under different lighting conditions; and
aligning images in the second plurality of images using the three-dimensional offset vector.
(Fridge, "when an actual display device 14 moving on belt 18 is presented to the system, each feature in the image displayed on the CRT can be viewed by both cameras", [c2:55-60]; "independent of the actual position and orientation of each display device as it is presented to the system", [c3:1-5]; incorporating Pentex into Fridge would adjust the lighting state and capture a second plurality of images of the actual part passing under the camera, utilizing the previously computed 3D offset vectors to mathematically align the captured part images)
Regarding claim 8, the combination of Fridge and Pentex teaches its/their respective base claim(s).
The combination further teaches the system of claim 1 wherein the camera matrix defines a mapping of three-dimensional points to two-dimensional points.
(Fridge, "associate a particular line in space with each pixel in each camera's image space.", [c3:40-45]; Pentex, "the intrinsic parameters of the image capture device 3 make it possible to connect or project a point defined in the first frame of reference 3′ in a point or a pixel of an image provided by this device 3", [0057]; Fridge and Pentex teach the camera matrix parameters providing a mathematical mapping from 3D space points to 2D image pixels)
Regarding claim 9, the combination of Fridge and Pentex teaches its/their respective base claim(s).
The combination further teaches the system of claim 1, wherein the distortion coefficients describe an amount of distortion in each of the plurality of images(;)
(Fridge, "Calibration software, using an accurately known artwork image, has established the position and perspective distortion associated with each camera.", [c2:55-60]; calculating coefficients that describe the perspective distortion in the images associated with each camera)
Regarding claim 10, the combination of Fridge and Pentex teaches its/their respective base claim(s).
The combination further teaches the system of claim 1, wherein the rotational vectors describe an amount of rotation of the calibration object in each of the plurality of images(.)
(Pentex, "translation vector T36 and a rotation matrix R36", [0048]; using a rotation matrix/vector to describe the rotational orientation and adjustments required for alignment)
Claim(s) 2-4, 12-14 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fridge (US5638461A) in view of Pentex et al (US20220276359A1) and further in view of Zhang et al (US20200151908A1) and Raag et al (US20190295291A1).
Regarding claims 2, 12 and 19, the combination of Fridge and Pentex teaches its/their respective base claim(s).
The combination does not expressly disclose but Zhang and Raag teach the system of claim 1, wherein the operations further comprise: for each image in the plurality of images:
computing a second-order derivative of image intensity of a corresponding image; and
(Zhang, "the horizontal gradient du and the vertical gradient dv of the pixel may be extracted", [0084]; Raag, "it determines the derivative of the image.", [0030]; "Ix and Iy be partial derivatives of I", [0033]; While Zhang and Raag explicitly compute first-order partial derivatives of image intensity to detect corners, it would have been obvious to a person of ordinary skill in the art to modify the algorithms to compute a second-order derivative of image intensity. Applying second-order derivatives (e.g., using Hessian or Laplacian of Gaussian matrices) is a widely known and predictable alternative technique in computer vision for robust feature and corner detection. Second-order derivatives (like Hessian or Laplacian matrices) are a predictable alternative because they replace broad gradient peaks with precise zero-crossings and surface curvature. This standard technique isolates corner geometry, optimizes scale invariance, and improves illumination invariance, making it a routine, textbook modification for achieving highly accurate, sub-pixel feature detection)
identifying corners of the calibration object in a corresponding image based on the second-order derivative.
(Raag, "Corners are then characterized by large variations in S", [0034]; Zhang, "determining the first checkerboard corners in the calibration image based on a corner likelihood of each pixel." [0060]; As discussed above, identifying corners based on a computed second-order derivative is a known equivalent design choice to using the first-order derivative methods taught by Zhang and Raag, yielding the predictable result of accurately identifying target features for camera calibration)
Regarding claims 3, 13 and 20, the combination of Fridge, Pentex, Zhang and Raag teaches its/their respective base claim(s).
The combination further teaches the system of claim 2, wherein the shape is a square with a checkerboard pattern.
(Zhang, "multiple checkerboard targets", [0008]; "intersection point between a black checkerboard and a white checkerboard", [0055]; Raag, "capture multiple images of a particular pattern, for example a checkerboard pattern"[0006]; "Preferably, however, the features are squares ... array comprises a plurality of squares", [0039]; Zhang and Raag teach a calibration object having a shape of squares arranged in a checkerboard pattern)
Regarding claims 4 and 14, the combination of Fridge, Pentex, Zhang and Raag teaches its/their respective base claim(s).
The combination further teaches the system of claim 3, wherein the identifying corners includes finding a grouping of corner locations with consistent spacing and then iteratively growing the calibration object in all directions by searching for corner locations within small windows that would continue the corner locations, until no peak in a search window meets a threshold or an edge of a corresponding image is detected.
(Zhang, "checkerboard growth may be performed based on the checkerboard corners",[0089]; "successively determining a current corner from each of the first checkerboard corners", [0091]; "determining, for each current corner, a first preset number of adjacent points of the current corner", [0092]; "for each of four growth directions of the initial unit, a candidate corner in the direction may be determined ... energy of a growth unit formed by the candidate corner and the second preset number of adjacent points is calculated", [0101]; "if the energy of the growth unit is less than the energy of the initial unit, the growth result is accepted, and the growth is continued based on the growth result. Otherwise, the growth is stopped in this direction.", [0104]; teaches the iterative growth of the calibration checkerboard object in all directions by evaluating grouped adjacent corner locations (small windows) until the energy condition (threshold) is no longer met, which halts the directional growth)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 8/22/2026
Prosecution Insights