Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This Office Action is sent in response to Applicant’s Communication received on April 14, 2024, April 16, 2024, August 28, 2024 and October 4, 2024 for application number 18/701,238. This Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, and Claims.
3. Claims 1-13 are presented for examination. Claims 12 and 13 have been amended. Claims 14-17 have been canceled.
Priority
4. Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. IN 202141046889, filed on October 14, 2021.
Claim Rejections - 35 USC § 101
5. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
6. Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because claim 13 is directed to a computer program per se. Therefore, the claim is directed to a non-statutory subject matter.
Claim Rejections - 35 USC § 103
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1-13 are rejected under 35 U.S.C. 103 as being unpatentable over FUKUMA et al.(US 2014/0320809 A1)(hereinafter Fukuma) in view of CLAVEAU et al.(US 2017/0287166 A1)(hereinafter Claveau).
Regarding claims 1, 12 and 13, Fukuma discloses a computer-implemented method of calibration of a movement of a movable platform upon which is mounted a stereo camera, a system comprising means adapted for carrying out a computer-implemented method of calibration of a movement of a movable platform upon which is mounted a stereo camera [See Fukuma: at least Figs 1-3, par. 60, 100-102 regarding ophthalmologic apparatus comprising a retinal camera unit 2. The retinal camera unit 2 shown in FIG. 1 is provided with an optical system for forming a two-dimensional image (fundus image) representing the surface morphology of the fundus Ef of the eye E...the optical system driver 2A of the present embodiment moves the optical system installed in the retinal camera unit 2. The optical system driver 2A is an example of a "moving mechanism."..] and a computer program comprising instructions for carrying out all the steps of a method, when said computer program is executed on a computer system, the method being a computer-implemented method of calibration of a movement of a movable platform upon which is mounted a stereo camera[See Fukuma: at least Figs. 1-3, par. 59-60, 95, 166, 168, 322 regarding An ophthalmologic apparatus 1, as shown in FIG. 1, comprises a retinal camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The arithmetic and control unit 200 is provided with a computer that executes various arithmetic processes, control processes, and so on. The arithmetic and control unit 200 includes a microprocessor, a RAM, a ROM, a hard disk drive, a communication interface, and so on, as in conventional computers. The storage device such as a hard disk drive stores computer programs for controlling the ophthalmologic apparatus 1. ], for use in a system for automatically calibrating a camera device [See Fukuma: at least Figs 1-7B, par. 189-207 regarding method for automatic alignment regarding an ophthalmologic apparatus for imaging a fundus using OCT…],
wherein the camera device is a fundus camera device including the stereo camera, a fundus camera, an illumination source and the movable platform, wherein the fundus camera, the stereo camera and the illumination source are mounted on the movable platform [See Fukuma: at least Figs. 1-3, par. 59-62, 77-78, 93, 100-102, 269 regarding The retinal camera unit 2 shown in FIG. 1 is provided with an optical system for forming a two-dimensional image (fundus image) representing the surface morphology of the fundus Ef of the eye E. The retinal camera unit 2 is provided with an illumination optical system 10 and an imaging optical system 30…The retinal camera unit 2 is provided with anterior eye cameras 300. The anterior eye cameras 300 substantially and simultaneously photograph an anterior eye part Ea from different directions… the optical system driver 2A of the present embodiment moves the optical system installed in the retinal camera unit 2. The optical system driver 2A is an example of a "moving mechanism."… Moreover, the anterior eye cameras 300 of the present embodiment are provided on the case of the retinal camera unit 2; accordingly, the anterior eye cameras 300 can be moved by means of controlling the optical system driver 2A (camera moving part)… In such configurations, it is possible to perform stereo photographing using two imaging parts. The two imaging parts are, for example, the anterior eye cameras 300A and 300B…]; and
wherein the system for automatically calibrating the camera device includes a multi-planar calibration target which comprises a fiducial marker and the marker comprises at least one object point and when an image of the marker is captured it contains one or more image points corresponding to the marker[See Fukuma: at least Fig. 8I, par. 69, 28, 112, 138, 142, 156, 234-243 regarding The alignment optical system 50 generates a target (alignment target) for position matching of the optical system (examination optical system) with respect to the eye E (alignment)… Further, the image processor 230 may form a cross sectional image at an arbitrary cross section based on a three-dimensional image data. This process is called MPR (Multi-Planar Reconstruction) and the like, and includes a process of extracting picture elements (voxels) located at a designated cross section and a process of arranging the extracted picture elements. The measurement depth marker 1102 is provided in the right end part of the OCT image display part 1101...].
Fukuma does not explicitly disclose wherein the system for automatically calibrating the camera device includes a multi-planar calibration target which comprises multiple fiducial markers and each of the markers comprises at least one object point and when an image of the marker is captured it contains one or more image points corresponding to the marker.
However, the use of multiple fiducial markers for alignment or calibration of camera devices was well known in the art at the time of the invention was filed as evident from the teaching of Claveau[See Claveau: at least Fig. 1-5, par. 5, 28, 85-89, 108-113, 121, 131, 142, regarding Conventional camera calibration techniques use one or more images of a specifically designed calibration target or object. The calibration target includes several readily detectable fiducial markers or features with known relative 3D positions. By fixing the world coordinate system in the calibration object, point correspondences between 3D world points and 2D image points can be established… In some implementations, the searching step includes looking for one or more fiducial features or markers on the calibration target… ].
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Fukuma with Claveau teachings by including “wherein the system for automatically calibrating the camera device includes a multi-planar calibration target which comprises multiple fiducial markers and each of the markers comprises at least one object point and when an image of the marker is captured it contains one or more image points corresponding to the marker” because this combination has the benefit of providing an efficient camera calibration process[See Claveau: at least par. 4-18].
Further on, when combined, Fukuma and Claveau teach wherein the method further comprises the steps of: capturing one or more images of one or more planes of the multi- planar calibration target using the stereo camera, wherein the one or more images are captured in one instance of calibration [See Fukuma: at least par. 60-65, 80, 135-139, 212, 234-243, regarding the image processor 230 executes various image processes and analysis on images (fundus images, anterior eye images, etc.) obtained by the retinal camera unit 2… Further, the image processor 230 may form a cross sectional image at an arbitrary cross section based on a three-dimensional image data. This process is called MPR (Multi-Planar Reconstruction) and the like, and includes a process of extracting picture elements (voxels) located at a designated cross section and a process of arranging the extracted picture elements…See Claveau: at least Fig. 6, par. 90, 96-101 regarding the method 200 of FIG. 6 includes a step 202 of providing a plurality of target images acquired with the camera, where each target image represents a view of the calibration target in a respective one of a plurality of target poses, each target pose corresponding to a given location and orientation of the calibration target relative to the camera...];
extracting at least one of the fiducial markers and image points from the captured image and storing them in a database wherein one object point and at least one image point is mapped with each of the fiducial markers based on a predefined location on the plane they are associated with [See Fukuma: at least Fig. 3, par. 103-109 regarding The main controller 211 executes a process of writing data into the storage 212, and a process of reading out data from the storage 212. The storage 212 stores various kinds of data. The data stored in the storage 212 is, for example, image data of OCT images, image data of fundus images, and eye information… an extraction procedure for extracting image regions corresponding to the reference points in each photograph image; a distribution state calculating procedure for calculating the distribution state (coordinates) of the image regions corresponding to the reference points in each photograph image;…See Claveau: at least par. 86-89, 100-106, 127-129, 138, 140, 147, 156-157, 160 regarding Each captured image of the target, or for brevity, target image, generally represents a view of the calibration target in a specific target pose corresponding to a certain location and orientation of the calibration target relative to the camera. In the present description, the term “target pose”, or simply “pose”, is used to describe a specific combination of location (or distance) and orientation of the calibration target relative to the camera or another reference frame. In general, not all captured target images are selected as reference images in the calculations of the camera parameters. Characteristics of the fiducial markers such as edges, corners, and the like may be extracted from the reference images to provide a mapping between the scene plane defined by the planar calibration target and its perspective image on the image plane… Alternatively, in other implementations, the providing step 302 can involve retrieving or receiving previously acquired target images, for example from a database or a storage medium. In all cases, all the target images are or have been acquired with the cameras to be calibrated…];
grouping the fiducial markers based on the plane they are associated with[ See Claveau: at least par. 85-89, 100-108, 121, 127-131, 138, 140, 147, 156-157, 160 regarding Each captured image of the target, or for brevity, target image, generally represents a view of the calibration target in a specific target pose corresponding to a certain location and orientation of the calibration target relative to the camera. In the present description, the term “target pose”, or simply “pose”, is used to describe a specific combination of location (or distance) and orientation of the calibration target relative to the camera or another reference frame. In general, not all captured target images are selected as reference images in the calculations of the camera parameters. Characteristics of the fiducial markers such as edges, corners, and the like may be extracted from the reference images to provide a mapping between the scene plane defined by the planar calibration target and its perspective image on the image plane … As mentioned above regarding the intrinsic calibration method, an exemplary image quality metric for qualifying the target images can be a weighted sum or average of the following quality factors: the number of points or fiducial markers detected on the target image; the level of blur in the image where the points or fiducial markers have been detected; the saturation of the image; and the contrast of the image.]; and
selecting planes having at least a predefined threshold number of the grouped fiducial markers followed by calibration of intrinsic and extrinsic properties of the stereo camera [See Claveau: at least Figs. 1-15, par. 4-9, 78-80, 85-89, 95-108, 121, 124--131, 138, 140, 147, 156-157, 160 regarding The calibration target includes several readily detectable fiducial markers or features with known relative 3D positions. By fixing the world coordinate system in the calibration object, point correspondences between 3D world points and 2D image points can be established. The intrinsic and extrinsic camera parameters can be computed by solving the system of equations resulting from these point correspondences. In the present techniques, obtaining the intrinsic and extrinsic camera parameters to solve the calibration problem generally involves identifying so-called “reference” images among a plurality of captured images of a calibration target…Characteristics of the fiducial markers such as edges, corners, and the like may be extracted from the reference images to provide a mapping between the scene plane defined by the planar calibration target and its perspective image on the image plane. Such mapping can be referred to as a “planar homography”…In Fig. 6, The method 200 also includes a step 204 of partitioning a volume of interest of the scene into a set of volume bins, and a step 206 of defining a set of angle bins, each angle bin encompassing a respective range of possible orientation values for the calibration target. The method 200 further includes a step 208 of identifying, among the plurality of target images, reference images of the calibration target, and a step 210 of assigning each reference image to one of the volume bins and/or one of the angle bins based on the respective target pose corresponding to the reference image. The method 200 also includes a step 212 of obtaining intrinsic camera parameters based on the reference images… Referring to FIG. 13, there is provided a flow diagram of an embodiment of a method 300 for extrinsically calibrating a network of cameras using a calibration target…];
wherein the method further comprises the steps of: identifying an (x, y, z) axis of the movable platform and the same (x, y, z) axis of the stereo camera; upon selecting a particular axis (x, y or z), determining a variation in movement between the corresponding selected axis of the movable platform and the corresponding selected axis of the camera to ascertain an error [See Fukuma: at least par. 27-29, 146-162 regarding Based on this calculation result of the three-dimensional position, the controller 210 controls the optical system driver 2A such that the optical axis of the examination optical system matches the axis of the eye E and such that the distance of the examination optical system with respect to the eye E becomes the predetermined working distance… The analyzer 231 may derive the displacement between the eye E and the examination optical system based on the three-dimensional position obtained by the three-dimensional position calculating part 2313. This process may be carried out by utilizing the fact that the positions of the anterior eye cameras 300 and the position of the examination optical system are known. Here, the position of the examination optical system is a position given in advance, and is, for example, the intersecting position of the front surface (surface facing the eye E) of the objective lens 22 and the optical axis of the examination optical system. The analyzer 231 that carries out this process corresponds to a "first displacement calculator."];
wherein the variation in movement is determined by moving the movable platform by a first fixed distance along a first axis, and simultaneously capturing images of the multi- planar calibration target using the stereo camera [See Fukuma: at least par. 27-29, 127-130, 146-162, 199-206, 269-270 regarding wherein the information obtaining part is configured to include two imaging parts non-coaxially arranged to the optical system, and a first displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing two images substantially simultaneously obtained by the two imaging parts, and the controller is configured to carry out the first control based on the displacement calculated by the first displacement calculator…wherein the imaging part is coaxially arranged with the optical system and obtains a moving image of the anterior eye part, and the information obtaining part is configured to include an alignment optical system configured to project an alignment target on the anterior eye part, and a second displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing the moving image of the anterior eye part on which the alignment target is being projected, and the controller is configured to carry out the first control based on the displacement calculated by the second displacement calculator…wherein the controller is configured to, in the second control, interlockingly carry out control of causing the optical system to repeatedly perform acquisition of the optical information and control of causing the moving mechanism to stepwise move the optical system, and determine whether this interlocking control is continued or ended based on information successively obtained by the optical system during this interlocking control…];
wherein the stereo camera is used to calculate a shift in the position of markers to determine a second fixed distance along the first axis; wherein the variation in movement is calculated as the difference between the second fixed distance and the first fixed distance [See Fukuma: at least par. 27-29, 127-130, 146-162, 199-206, 269-270 regarding a first displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing two images substantially simultaneously obtained by the two imaging parts, and the controller is configured to carry out the first control based on the displacement calculated by the first displacement calculator…wherein the imaging part is coaxially arranged with the optical system and obtains a moving image of the anterior eye part, and the information obtaining part is configured to include an alignment optical system configured to project an alignment target on the anterior eye part, and a second displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing the moving image of the anterior eye part on which the alignment target is being projected, and the controller is configured to carry out the first control based on the displacement calculated by the second displacement calculator… See Claveau: par. 80-89, 101 regarding The extrinsic parameters can include, without limitation, the distance and relative angle between the world reference frame and the camera reference frame. Knowledge of the intrinsic and extrinsic camera parameters is required to use a camera model and can be achieved by camera calibration. Camera calibration can involve acquiring multiple reference images of a calibration target from different distances and angles with respect to the camera to establish a mapping between 3D world points and the corresponding 2D images points, and performing a calibration calculation based on these correspondences… In some implementations, orientation markers are present on the target, so that it may be possible to recognize the target in any direction and/or when the target is only partially present in the camera field of view, provided that at least two markers are present in the image… In some implementations, the fiducial markers can also include orientation markers. Orientation markers can allow the calibration target to be recognized in any orientation and/or when the calibration target is only partially present in the field of view of a camera, provided that at least two orientation markers are present in the target image. That is, once the at least two orientations makers are detected, it is possible to define and position a 2D target reference frame on the target image... In some implementations, the calibration target is portable and is intended to be held and moved within the scene by an operator such that the camera gradually captures images of the calibration target from varying distances and/or orientations relative to the camera…];
calculating an average error for all the axes based on the ascertained error; and minimising the average error by virtually rotating the stereo camera axes to match the movable platform axes [See Fukuma: at least Figs. 5A-5B, par. 27-29, 127-130, 146-162, 199-206, 269-270 regarding a first displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing two images substantially simultaneously obtained by the two imaging parts, and the controller is configured to carry out the first control based on the displacement calculated by the first displacement calculator…wherein the imaging part is coaxially arranged with the optical system and obtains a moving image of the anterior eye part, and the information obtaining part is configured to include an alignment optical system configured to project an alignment target on the anterior eye part, and a second displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing the moving image of the anterior eye part on which the alignment target is being projected, and the controller is configured to carry out the first control based on the displacement calculated by the second displacement calculator… See Claveau: at least Figs. 1-13, par. 121-122, 142-154 regarding In some implementations, the intrinsic camera parameters can be further refined by using a non-linear least-squares method in which the objective function to be minimized is the reprojection error of the target pose fiducials… In some implementations, the obtaining step 212 can include a sub-step of determining a calibration error associated with the obtained intrinsic camera parameters. One possible approach to assess the completion level of the intrinsic calibration is to continuously or repeatedly (e.g., periodically) compute the average reprojection error of target points in the reference images and then compare the computed error with a predetermined threshold below which the intrinsic calibration is considered complete or satisfactory. In such implementations, the obtaining step 212 can be performed iteratively until the calibration error gets lower than a predetermined error value, at which point the providing step 202, identifying step 208 and assigning step 210 can also be stopped… In some implementations, the method 300 of FIG. 13 can allow the user to monitor the progress of the extrinsic camera calibration. One possible approach to assess the completion level of the extrinsic calibration is to continuously or repeatedly (e.g., periodically) compute the average reprojection error of target points in the reference images and then compare the computed error with a predetermined threshold below which extrinsic calibration is considered complete or satisfactory. In such implementations, the obtaining steps 310 and 312 can be performed iteratively until the calibration error gets lower than a predetermined error value, at which point the providing step 302, identifying step 306 and assigning step 308 can also be stopped.].
Regarding claim 2, Fukuma and Claveau teach all of the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Further on, Fukuma and Claveau teach or suggest wherein the moveable platform is determined to be calibrated when the average error is within a predefined threshold[See Fukuma: at least Figs. 5A-5B, par. 27-29, 127-130, 146-162, 199-206, 269-270 regarding a first displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing two images substantially simultaneously obtained by the two imaging parts, and the controller is configured to carry out the first control based on the displacement calculated by the first displacement calculator…wherein the imaging part is coaxially arranged with the optical system and obtains a moving image of the anterior eye part, and the information obtaining part is configured to include an alignment optical system configured to project an alignment target on the anterior eye part, and a second displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing the moving image of the anterior eye part on which the alignment target is being projected, and the controller is configured to carry out the first control based on the displacement calculated by the second displacement calculator… See Claveau: at least Figs. 1-13, par. 121-122, 142-154 regarding In some implementations, the intrinsic camera parameters can be further refined by using a non-linear least-squares method in which the objective function to be minimized is the reprojection error of the target pose fiducials… In some implementations, the obtaining step 212 can include a sub-step of determining a calibration error associated with the obtained intrinsic camera parameters. One possible approach to assess the completion level of the intrinsic calibration is to continuously or repeatedly (e.g., periodically) compute the average reprojection error of target points in the reference images and then compare the computed error with a predetermined threshold below which the intrinsic calibration is considered complete or satisfactory. In such implementations, the obtaining step 212 can be performed iteratively until the calibration error gets lower than a predetermined error value, at which point the providing step 202, identifying step 208 and assigning step 210 can also be stopped… In some implementations, the method 300 of FIG. 13 can allow the user to monitor the progress of the extrinsic camera calibration. One possible approach to assess the completion level of the extrinsic calibration is to continuously or repeatedly (e.g., periodically) compute the average reprojection error of target points in the reference images and then compare the computed error with a predetermined threshold below which extrinsic calibration is considered complete or satisfactory. In such implementations, the obtaining steps 310 and 312 can be performed iteratively until the calibration error gets lower than a predetermined error value, at which point the providing step 302, identifying step 306 and assigning step 308 can also be stopped.].
Regarding claim 3, Fukuma and Claveau teach all of the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Further on, Fukuma and Claveau teach or suggest wherein the variation along a second axis is calculated[See Fukuma: at least par. 27-29, 127-130, 146-162, 199-206, 269-270 regarding a first displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing two images substantially simultaneously obtained by the two imaging parts, and the controller is configured to carry out the first control based on the displacement calculated by the first displacement calculator…wherein the imaging part is coaxially arranged with the optical system and obtains a moving image of the anterior eye part, and the information obtaining part is configured to include an alignment optical system configured to project an alignment target on the anterior eye part, and a second displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing the moving image of the anterior eye part on which the alignment target is being projected, and the controller is configured to carry out the first control based on the displacement calculated by the second displacement calculator… See Claveau: par. 80-89, 101 regarding The extrinsic parameters can include, without limitation, the distance and relative angle between the world reference frame and the camera reference frame. Knowledge of the intrinsic and extrinsic camera parameters is required to use a camera model and can be achieved by camera calibration. Camera calibration can involve acquiring multiple reference images of a calibration target from different distances and angles with respect to the camera to establish a mapping between 3D world points and the corresponding 2D images points, and performing a calibration calculation based on these correspondences… In some implementations, orientation markers are present on the target, so that it may be possible to recognize the target in any direction and/or when the target is only partially present in the camera field of view, provided that at least two markers are present in the image… In some implementations, the fiducial markers can also include orientation markers. Orientation markers can allow the calibration target to be recognized in any orientation and/or when the calibration target is only partially present in the field of view of a camera, provided that at least two orientation markers are present in the target image. That is, once the at least two orientations makers are detected, it is possible to define and position a 2D target reference frame on the target image... In some implementations, the calibration target is portable and is intended to be held and moved within the scene by an operator such that the camera gradually captures images of the calibration target from varying distances and/or orientations relative to the camera…].
Regarding claim 4, Fukuma and Claveau teach all of the limitations of claim 3, and are analyzed as previously discussed with respect to that claim. Further on, Fukuma and Claveau teach or suggest wherein the variation along a third axis is calculated[See Fukuma: at least par. 6, 27-29, 127-130, 146-162, 199-206, 269-270 regarding Alignment includes the action of aligning the light axis of the optical system of the apparatus with respect to the axis of an eye (xy alignment), as well as the action of adjusting the distance between the eye and the optical system of the apparatus (z alignment)… The image determination part 232 transmits this determination result to the controller 210. When it is determined that the image of the anterior eye part Ea is not included in the predetermined area, the controller 210 controls the optical system driver 2A (camera moving part) to move the anterior eye cameras 300 in a direction away from the supporter 440 (that is, the face of the subject) and/or a direction outwards of the supporter 440. The direction away from the supporter 440 is the -z direction in the coordinate system indicated in FIG. 1, etc. Moreover, the direction outwards of the supporter 440 is the direction in which the anterior eye cameras 300 moves away from the optical axis of the examination optical system. The direction away from the examination optical system may be defined horizontally (.+-.x direction) and/or vertically (.+-.y direction). That is, the direction away from the examination optical system may be defined in any direction in the xy plane…See Claveau: at least Figs. 5-13, par. 27, 68, 80-89, 92-93, 101-106 regarding In such implementations, the orientation of the calibration target can be defined by the angle made between the planar calibration surface and a plane normal to the optical axis of the camera…Knowledge of the intrinsic and extrinsic camera parameters is required to use a camera model and can be achieved by camera calibration… In some implementations, the fiducial markers can also include orientation markers. Orientation markers can allow the calibration target to be recognized in any orientation and/or when the calibration target is only partially present in the field of view of a camera, provided that at least two orientation markers are present in the target image. That is, once the at least two orientations makers are detected, it is possible to define and position a 2D target reference frame on the target image... In some implementations, the calibration target is portable and is intended to be held and moved within the scene by an operator such that the camera gradually captures images of the calibration target from varying distances and/or orientations relative to the camera…].
Regarding claim 5, Fukuma and Claveau teach all of the limitations of claim 3, and are analyzed as previously discussed with respect to that claim. Further on, Claveau teaches or suggests wherein scalar multiplications are used to minimize errors in each axis[See Claveau: at least Figs. 1-13, par. 80, 88, 121-122, 142-154 regarding In some implementations, the intrinsic camera parameters can be further refined by using a non-linear least-squares method in which the objective function to be minimized is the reprojection error of the target pose fiducials… In some implementations, the obtaining step 212 can include a sub-step of determining a calibration error associated with the obtained intrinsic camera parameters. One possible approach to assess the completion level of the intrinsic calibration is to continuously or repeatedly (e.g., periodically) compute the average reprojection error of target points in the reference images and then compare the computed error with a predetermined threshold below which the intrinsic calibration is considered complete or satisfactory. In such implementations, the obtaining step 212 can be performed iteratively until the calibration error gets lower than a predetermined error value, at which point the providing step 202, identifying step 208 and assigning step 210 can also be stopped… In some implementations, the method 300 of FIG. 13 can allow the user to monitor the progress of the extrinsic camera calibration. One possible approach to assess the completion level of the extrinsic calibration is to continuously or repeatedly (e.g., periodically) compute the average reprojection error of target points in the reference images and then compare the computed error with a predetermined threshold below which extrinsic calibration is considered complete or satisfactory. In such implementations, the obtaining steps 310 and 312 can be performed iteratively until the calibration error gets lower than a predetermined error value, at which point the providing step 302, identifying step 306 and assigning step 308 can also be stopped… The intrinsic camera parameters describe the internal geometric and optical characteristics of the camera itself. The intrinsic parameters can include, without limitation, the focal length, the coordinates of the principal point, the scale factors and the skew factors. The extrinsic parameters describe the coordinate system transformation (i.e., translation and rotation) between the world coordinate system and the camera coordinate system. The extrinsic parameters can include, without limitation, the distance and relative angle between the world reference frame and the camera reference frame. Knowledge of the intrinsic and extrinsic camera parameters is required to use a camera model and can be achieved by camera calibration. ].
Regarding claim 6, Fukuma and Claveau teach all of the limitations of claim 3, and are analyzed as previously discussed with respect to that claim. Further on, Claveau teaches or suggests wherein the virtual rotating involves multiplying a rotation matrix to coordinates of markers as calculated by the stereo camera using a triangulation method [See Claveau: at least par. 80-81, 98, 105-106, 110 regarding The extrinsic parameters describe the coordinate system transformation (i.e., translation and rotation) between the world coordinate system and the camera coordinate system. Camera calibration can involve acquiring multiple reference images of a calibration target from different distances and angles with respect to the camera to establish a mapping between 3D world points and the corresponding 2D images points, and performing a calibration calculation based on these correspondences... In the present description, the term “viewpoint” refers to a position, describable in a six-parameter space (i.e. three spatial or translational coordinates and three angular or rotational coordinates), where a camera would be to view a scene…].
Regarding claim 7, Fukuma and Claveau teach all of the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Further on, Fukuma and Claveau teach or suggest wherein the determining, calculating, and minimizing steps are performed by a control unit under software control[See Fukuma: at least par. 242-243, 316, 322 regarding the controller 241 may display a software key for the user to select a subsequent process on the display 241. The options may include, for example, any of re-execution of the same searching process, execution of a searching process including movement control of different type, and execution of manual alignment… Computer programs for realizing the above embodiments can be stored in any kind of recording medium that can be read by a computer… See Claveau: at least par. 120, 139, 159 regarding Given the many computational approaches and toolboxes available for performing camera calibration, it will be appreciated by those skilled in the art that various computer-implemented and software-based analytical and/or numerical techniques can be employed for estimating the intrinsic parameters of each camera from the set of reference images…].
Regarding claim 8, Fukuma and Claveau teach all of the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Further on, Claveau teaches or suggests wherein a fast validation is carried out prior to the identifying step, where the method determines if pre-existing calibration files related to pre-calibrated components of the camera device are compatible with the functioning of the camera device[See Claveau: at least par. 44, 48, 128-129, 145-158 regarding In some implementations, the present techniques can automate one or more of the following actions: the continuous capture of target images by the cameras; the target detection in the captured images; the validation of the target detection; the mapping of the current target image with the appropriate volume and angle bins; the formation of sets of reference images and/or validation images; the estimation of intrinsic and extrinsic calibration parameters from the reference images; and the calculation of performance indicators from the validation images… As described below, in some implementations, the qualified target images stored in each multi-camera bin can be used to either obtain the extrinsic camera parameters or validate the obtained extrinsic camera parameters.].
Regarding claim 9, Fukuma and Claveau teach all of the limitations of claim 8, and are analyzed as previously discussed with respect to that claim. Further on, Claveau teaches or suggests wherein the fast validation of the camera devices is provided with errors in pre-calibrated components and if such errors are below a predefined threshold, the camera device is determined to be validated and the camera device deemed ready to use [See Claveau: at least par. 44, 48, 128-129, 145-158 regarding Once the intrinsic and/or extrinsic camera parameters have been obtained, the present techniques can provide a step of validation of the calibration results. Depending on the application or use, different criteria or measures can be used to ascertain the validity or quality of the camera calibration. Non-limiting examples of such criteria or measures can include the reprojection error and the rectification error for the intrinsic parameters; and the reconstruction error and the alignment or registration error for the overall intrinsic and extrinsic calibration…].
Regarding claim 10, Fukuma and Claveau teach all of the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Further on, Fukuma and Claveau teach wherein the variation along a second axis is calculated, wherein the variation along a third axis is calculated, and wherein the variation in movement is determined by moving the movable platform by a first fixed distance along each of the first axis, the second axis and the third axis, and simultaneously capturing images of the multi-planar calibration target using the stereo camera[See Fukuma: at least par. 27-29, 127-130, 146-162, 199-206, 269-270 regarding a first displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing two images substantially simultaneously obtained by the two imaging parts, and the controller is configured to carry out the first control based on the displacement calculated by the first displacement calculator…wherein the imaging part is coaxially arranged with the optical system and obtains a moving image of the anterior eye part, and the information obtaining part is configured to include an alignment optical system configured to project an alignment target on the anterior eye part, and a second displacement calculator configured to calculate a displacement between the eye and the optical system by analyzing the moving image of the anterior eye part on which the alignment target is being projected, and the controller is configured to carry out the first control based on the displacement calculated by the second displacement calculator… See Claveau: par. 80-89, 101 regarding The extrinsic parameters can include, without limitation, the distance and relative angle between the world reference frame and the camera reference frame. Knowledge of the intrinsic and extrinsic camera parameters is required to use a camera model and can be achieved by camera calibration. Camera calibration can involve acquiring multiple reference images of a calibration target from different distances and angles with respect to the camera to establish a mapping between 3D world points and the corresponding 2D images points, and performing a calibration calculation based on these correspondences… In some implementations, orientation markers are present on the target, so that it may be possible to recognize the target in any direction and/or when the target is only partially present in the camera field of view, provided that at least two markers are present in the image… In some implementations, the fiducial markers can also include orientation markers. Orientation markers can allow the calibration target to be recognized in any orientation and/or when the calibration target is only partially present in the field of view of a camera, provided that at least two orientation markers are present in the target image. That is, once the at least two orientations makers are detected, it is possible to define and position a 2D target reference frame on the target image... In some implementations, the calibration target is portable and is intended to be held and moved within the scene by an operator such that the camera gradually captures images of the calibration target from varying distances and/or orientations relative to the camera…].
Regarding claim 11, Fukuma and Claveau teach all of the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Further on, Fukuma and Claveau teach or suggest wherein the one or more captured images have a partial or a complete view of the calibration target [See Fukuma: at least par. 60-65, 80, 135-139, 212, 234-243 regarding the image processor 230 executes various image processes and analysis on images (fundus images, anterior eye images, etc.) obtained by the retinal camera unit 2… Further, the image processor 230 may form a cross sectional image at an arbitrary cross section based on a three-dimensional image data. This process is called MPR (Multi-Planar Reconstruction) and the like, and includes a process of extracting picture elements (voxels) located at a designated cross section and a process of arranging the extracted picture elements…See Claveau: at least Fig. 6, par. 90, 96-101 regarding the method 200 of FIG. 6 includes a step 202 of providing a plurality of target images acquired with the camera, where each target image represents a view of the calibration target in a respective one of a plurality of target poses, each target pose corresponding to a given location and orientation of the calibration target relative to the camera...].
Conclusion
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANA J PICON-FELICIANO whose telephone number is (571)272-5252. The examiner can normally be reached Monday-Friday 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Kelley can be reached at 571 272 7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ana Picon-Feliciano/Examiner, Art Unit 2482
/CHRISTOPHER S KELLEY/Supervisory Patent Examiner, Art Unit 2482
Prosecution Insights