Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7-13, 15-17 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leach (EP 4024346) in view of Shi et al. Fast Uncertainty Quantification for Deep Object Pose Estimation
With respect to claim 1, Leach teaches a method of estimating the uncertainty in a vision based tracking system , the method comprising:
receiving a two dimensional image (see col. 11, line 3) of a portion of a first object (aircraft 110 to be refueled) via a camera (108) on a second object (refueling aircraft 102).
Leach teaches establishing/ predicting a set of key-points (para. 9, line 4) of the first object 110 by means of a 2D key-point (object landmark) detection and a 2D to 3D transformation means that determines the 6 degrees of freedom information for each key-point(s) of a fuel receptacle and a tip of the refueling boom (see para. 11, lines 1-6). It appears that the transformation means and the determining means are performed by neural networks as set forth in para. 11, lines 8-12.
Leach teaches the computing step as claimed which is recited in para. 11 as previously stated. A 2D to 3D transformation is established by the neural networks.
Leach teaches deriving a measure of deviation between 3 D poses. At the bottom of para. 21, Leach teaches using a heatmap pixel values for each key-point, which is a 3D object’s key-point being found at each pixel location of the image.
Furthermore, a Kalman filter is used to measure statistical noise and other inaccuracies (deviations as recited in the claim ), and produce estimates. See paras. 20 and 22.
Leach teaches controlling a process so that the first object (aircraft to be refueled) and the second object (aircraft refuel platform-102) is refueled in response to a decision operation 738 to determine if the boom tip 106 and the fuel receptacle is within a safety parameter 156.
What Leach does not teach is computing a Euclidean norm of the measurement of variation and using it to produce an uncertainty value.
Shi teaches computing a distance between the actual pose and an estimated pose. Shi teaches the computation of uncertainty quantification, based on a distribution of the average differences between ground truth of the pose and an estimated pose. A value that has the least of errors is termed a higher value of certainty (See Problem Statement and Metrics at page 3).
While Leach does not address uncertainty values in the computation of processing the boom tip with the fuel receptacle, the decision operation 738 performs the similar control processing functions.
The Examiner contends that it would have been obvious to one of ordinary skill in the art, to modify the decision operation unit 738 of Leach so that it uses uncertainty values which minimize the errors between estimations, thereby controlling the distance between the boom tip and the fuel receptacle for the purpose of connected the first and second objects.
With respect to claim 2, Leach teaches a 2D to 3D transform 620, see para. 49 at col. 13, lines 13-18. See also para. 27, lines 12-16 and para. 45, lines 33-36. The motivation for this rejection is the same as that to claim 1 above.
With respect to claim 3, Leach teaches a 3D pose of 6 values for six degrees of freedom. See col. 3,lines 50-57. For example, -X, X, -Y, Y, -Z and Z.
Leach teaches deriving a measure of deviation between 3 D poses. At the bottom of para. 21, Leach teaches using a heatmap pixel values for each key-point, which is a 3D object’s key-point being found at each pixel location of the image.
Furthermore, a Kalman filter is used to measure statistical noise and other inaccuracies (deviations as recited in the claim ), and produce estimates. See paras. 20 and 22. The motivation for this rejection is the same as that to claim 1.
With respect to claim 7, Leach teaches a first object (aircraft 110 to be refueled) to be coupled with second object (refueling aircraft 102) by means of a boom tip 104 and fuel receptacle
Leach teaches deriving a measure of deviation between 3 D poses. At the bottom of para. 21, Leach teaches using a heatmap pixel values for each key-point, which is a 3D object’s key-point being found at each pixel location of the image.
Furthermore, a Kalman filter is used to measure statistical noise and other inaccuracies (deviations as recited in the claim ), and produce estimates. See paras. 20 and 22.
Leach teaches controlling a process so that the first object (aircraft to be refueled) and the second object (aircraft refuel platform-102) is refueled in response to a decision operation 738 to determine if the boom tip 106 and the fuel receptacle is within a safety parameter 156. If the safety parameter 156 is within range then engaging will occur and will not be engaged when the safety parameter 156 is outside the threshold value. See para. 49 regarding the engagement of the boom tip 106 and the fuel receptacle is within the safety parameter 156, see para. 49, lines 1-6.
While Leach does not address uncertainty values in the computation of processing the boom tip with the fuel receptacle, the decision operation 738 performs the similar control processing functions.
The Examiner contends that it would have been obvious to one of ordinary skill in the art, to modify the decision operation unit 738 of Leach so that it uses uncertainty values which minimize the errors between estimations, thereby controlling the distance between the boom tip and the fuel receptacle for the purpose of connected the first and second objects.
The motivation for this rejection is the same as that to claim 1.
With respect to claim 8, at para. 49, Leach teaches the control of the engagement between the boom 106 and fuel receptacle. See also the first four lines of para. 50. Leach teaches controlling a process so that the first object (aircraft to be refueled) and the second object (aircraft refuel platform-102) is refueled in response to a decision operation 738 to determine if the boom tip 106 and the fuel receptacle is within a safety parameter 156. See col. 1, lines 45-48 regarding automated control. See also para. 17, last 8 lines.
With respect to claim 9, Leach teaches using manual control of the refueling engagement process. See para. 1, lines 1-9, para. 2 (entirety).
With respect to claim 10, Leach teaches a 2D image of the landmark key- points of the second object (visualization of the boom tip). Leach teaches predicting other key points such as the wingtip para. 21, lines 15-16, and fiducial marks 118 (see para. 18).
With respect to claim 11, Leach teaches the first object is a receiver aircraft 110 and the second object is a tanker aircraft 102 and the process is a refueling process, see paragraph 1, lines 1-5; para. 4, lines 1-6 and para. 9, lines 1-4.
The motivation for the rejection is the same as that to claim 1.
With respect to claim 12, Leach teaches a vision based tracking system comprising:
one or more processors 904; a non-transitory computer readable medium 902 for storing instructions 902a. Leach teaches receiving a two dimensional image (see col. 11, line 3) of a portion of a first object (aircraft 110 to be refueled) via a camera (108) on a second object (refueling aircraft 102).
Leach teaches establishing/ predicting a set of key-points (para. 9, line 4) of the first object 110 by means of a 2D key-point (object landmark) detection and a 2D to 3D transformation means that determines the 6 degrees of freedom information for each key-point(s) of a fuel receptacle and a tip of the refueling boom (see para. 11, lines 1-6). It appears that the transformation means and the determining means are performed by neural networks as set forth in para. 11, lines 8-12.
Leach teaches the computing step as claimed which is recited in para. 11 as previously stated. A 2D to 3D transformation is established by the neural networks.
Leach teaches deriving a measure of deviation between 3 D poses. At the bottom of para. 21, Leach teaches using a heatmap pixel values for each key-point, which is a 3D object’s key-point being found at each pixel location of the image.
Furthermore, a Kalman filter is used to measure statistical noise and other inaccuracies (deviations as recited in the claim ), and produce estimates. See paras. 20 and 22.
Leach teaches controlling a process so that the first object (aircraft to be refueled) and the second object (aircraft refuel platform-102) is refueled in response to a decision operation 738 to determine if the boom tip 106 and the fuel receptacle is within a safety parameter 156.
What Leach does not teach is computing a Euclidean norm of the measurement of variation and using it to produce an uncertainty value.
Shi teaches computing a distance between the actual pose and an estimated pose. Shi teaches the computation of uncertainty quantification, based on a distribution of the average differences between ground truth of the pose and an estimated pose. A value that has the least of errors is termed a higher value of certainty (See Problem Statement and Metrics at page 3).
While Leach does not address uncertainty values in the computation of processing the boom tip with the fuel receptacle, the decision operation 738 performs the similar control processing functions.
The Examiner contends that it would have been obvious to one of ordinary skill in the art, to modify the decision operation unit 738 of Leach so that it uses uncertainty values which minimize the errors between estimations, thereby controlling the distance between the boom tip and the fuel receptacle for the purpose of connected the first and second objects.
With respect to claim 13, Leach teaches a 2D to 3D transform 620, see para. 49 at col. 13, lines 13-18. See also para. 27, lines 12-16 and para. 45, lines 33-36. The motivation for this rejection is the same as that to claim 12 above.
With respect to claim 15, Leach teaches one or more processors 904; a non-transitory computer readable medium 902 for storing instructions 902a. Leach teaches a first object (aircraft 110 to be refueled) to be coupled with second object (refueling aircraft 102) by means of a boom tip 104 and fuel receptacle. The motivation for this rejection is the same as that to claim 12.
Leach teaches deriving a measure of deviation between 3 D poses. At the bottom of para. 21, Leach teaches using a heatmap pixel values for each key-point, which is a 3D object’s key-point being found at each pixel location of the image.
Furthermore, a Kalman filter is used to measure statistical noise and other inaccuracies (deviations as recited in the claim ), and produce estimates. See paras. 20 and 22.
Leach teaches controlling a process so that the first object (aircraft to be refueled) and the second object (aircraft refuel platform-102) is refueled in response to a decision operation 738 to determine if the boom tip 106 and the fuel receptacle is within a safety parameter 156. If the safety parameter 156 is within range then engaging will occur and will not be engaged when the safety parameter 156 is outside the threshold value. See para. 49 regarding the engagement of the boom tip 106 and the fuel receptacle is within the safety parameter 156, see para. 49, lines 1-6.
While Leach does not address uncertainty values in the computation of processing the boom tip with the fuel receptacle, the decision operation 738 performs the similar control processing functions.
The Examiner contends that it would have been obvious to one of ordinary skill in the art, to modify the decision operation unit 738 of Leach so that it uses uncertainty values which minimize the errors between estimations, thereby controlling the distance between the boom tip and the fuel receptacle for the purpose of connected the first and second objects.
With respect to claim 16, Leach teaches a vision based tracking system, comprising: a camera 108 configured to generate a 2D image of a first portion the first object is a receiver aircraft 110 and the second object is a tanker aircraft 102 and the process is a refueling process, see paragraph 1, lines 1-5; para. 4, lines 1-6 and para. 9, lines 1-4.
Leach teaches one or more processors 904; a non-transitory computer readable medium 902 for storing instructions 902a. Leach teaches receiving a two dimensional image (see col. 11, line 3) of a portion of a first object (aircraft 110 to be refueled) via a camera (108) on a second object (refueling aircraft 102).
Leach teaches establishing/ predicting a set of key-points (para. 9, line 4) of the first object 110 by means of a 2D key-point (object landmark) detection and a 2D to 3D transformation means that determines the 6 degrees of freedom information for each key-point(s) of a fuel receptacle and a tip of the refueling boom (see para. 11, lines 1-6). It appears that the transformation means and the determining means are performed by neural networks as set forth in para. 11, lines 8-12.
Leach teaches the computing step as claimed which is recited in para. 11 as previously stated. A 2D to 3D transformation is established by the neural networks.
Leach teaches deriving a measure of deviation between 3D poses. At the bottom of para. 21, Leach teaches using a heatmap pixel values for each key-point, which is a 3D object’s key-point being found at each pixel location of the image.
Furthermore, a Kalman filter is used to measure statistical noise and other inaccuracies (deviations as recited in the claim ), and produce estimates. See paras. 20 and 22.
Leach teaches controlling a process so that the first object (aircraft to be refueled) and the second object (aircraft refuel platform-102) is refueled in response to a decision operation 738 to determine if the boom tip 106 and the fuel receptacle is within a safety parameter 156.
What Leach does not teach is computing a Euclidean norm of the measurement of variation and using it to produce an uncertainty value.
Shi teaches computing a distance between the actual pose and an estimated pose. Shi teaches the computation of uncertainty quantification, based on a distribution of the average differences between ground truth of the pose and an estimated pose. A value that has the least of errors is termed a higher value of certainty (See Problem Statement and Metrics at page 3).
While Leach does not address uncertainty values in the computation of processing the boom tip with the fuel receptacle, the decision operation 738 performs the similar control processing functions.
The Examiner contends that it would have been obvious to one of ordinary skill in the art, to modify the decision operation unit 738 of Leach so that it uses uncertainty values which minimize the errors between estimations, thereby controlling the distance between the boom tip and the fuel receptacle for the purpose of connected the first and second objects.
With respect to claim 17, Leach teaches a 2D to 3D transform 620, see para. 49 at col. 13, lines 13-18. See also para. 27, lines 12-16 and para. 45, lines 33-36. The motivation for this rejection is the same as that to claim 16.
With respect to claim 20, Leach teaches a process of engagement with a first object (aircraft 110 to be refueled) to be coupled with second object (refueling aircraft 102) by means of a boom tip 104 and fuel receptacle
Leach teaches deriving a measure of deviation between 3 D poses. At the bottom of para. 21, Leach teaches using a heatmap pixel values for each key-point, which is a 3D object’s key-point being found at each pixel location of the image.
Furthermore, a Kalman filter is used to measure statistical noise and other inaccuracies (deviations as recited in the claim ), and produce estimates. See paras. 20 and 22.
Leach teaches controlling a process so that the first object (aircraft to be refueled) and the second object (aircraft refuel platform-102) is refueled in response to a decision operation 738 to determine if the boom tip 106 and the fuel receptacle is within a safety parameter 156. If the safety parameter 156 is within range then engaging will occur and will not be engaged when the safety parameter 156 is outside the threshold value. See para. 49 regarding the engagement of the boom tip 106 and the fuel receptacle is within the safety parameter 156, see para. 49, lines 1-6.
While Leach does not address uncertainty values in the computation of processing the boom tip with the fuel receptacle, the decision operation 738 performs the similar control processing functions.
The Examiner contends that it would have been obvious to one of ordinary skill in the art, to modify the decision operation unit 738 of Leach so that it uses uncertainty values which minimize the errors between estimations, thereby controlling the distance between the boom tip and the fuel receptacle for the purpose of connected the first and second objects.
Claim(s) 4, 5, 6, 14, 18 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leach (EP 4024346) in view of Shi et al. Fast Uncertainty Quantification for Deep Object Pose Estimation further in view of KSR v. Teleflex 550 U.S. 398 (2007)
With respect to claim 4, Leach teaches a multiple key point detectors of a 2D key-point (object landmark) detection and a 2D to 3D transformation means which detects multiple key-points. It appears that the multiple means for determining multiple key-points are multi-stage pose estimation pipelines which use real-time deep-learning algorithms such as deep neural networks, see col. 4, lines 5-9. Leach specifically mentions identification of at least three key-points, namely, the boom tip, wing edge and the fiduciary mark 118.
The examiner contends that the detector 310 is comprised of at least one neural network but is structured to identify different structures corresponding with the identification of multiple key-points, such as the wingtip and fuselage.
In the alternative, the Examiner contends that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to try any number of different detectors 310, each for identifying a section of the aircraft t having identical architecture and receiving the same type of training with random valued weights which would have been suggested by one of ordinary skill in the art seeking a specified result to minimize error in estimation.
With respect to claim 5, Leach teaches an aircraft key-point detector 310. Leach also teaches that the detector is trained by a key-point model trainer 352, see para. 6, lines 40-45.
The examiner contends that the detector 310 is comprised of at least one neural network but is structured to identify different structures corresponding with the identification of multiple key-points, such as the wingtip and fuselage.
In the alternative, the Examiner contends that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to try the use of different detectors 310, each for identifying a section of the aircraft to be refueled and each having identical architecture and receiving the same type of training with random valued weights which would have been suggested by one of ordinary skill in the art seeking a specified result to minimize error in estimation. The motivation for this rejection is the same as that to claim 1.
With respect to claim 6, Leach teaches a data pre-processor component 340 that train images. At para. 26 beginning at line 14, the loss that is minimized during training is the pixel-wise mean-squared -error between the heat map and the ground truth heat map. This mean-squared-error is the computation of the standard deviation. The training images occur in 3D as set forth in para. 25, beginning at line 4. The motivation for this rejection is the same as that to claim 5 depending from claim 1.
With respect to claim 14, Leach teaches one or more processors 904; a non-transitory computer readable medium 902 for storing instructions 902a. Leach teaches an aircraft key-point detector 310. Leach also teaches that the detector is trained by a key-point model trainer 352, see para. 6, lines 40-45.
The examiner contends that the detector 310 is comprised of at least one neural network but is structured to identify different structures corresponding with the identification of multiple key-points, such as the wingtip and fuselage.
In the alternative, the Examiner contends that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to try the use of different detectors 310, each for identifying a section of the aircraft to be refueled and each having identical architecture and receiving the same type of training with random valued weights which would have been suggested by one of ordinary skill in the art seeking a specified result to minimize error in estimation.
With respect to claim 18, Leach teaches software code via instructions (902a) implemented by one or more processors 904 for training key-point detectors.
Leach teaches an aircraft key-point detector 310. Leach also teaches that the detector is trained by a key-point model trainer 352, see para. 6, lines 40-45.
The examiner contends that the detector 310 is comprised of at least one neural network but is structured to identify different structures corresponding with the identification of multiple key-points, such as the wingtip and fuselage.
In the alternative, the Examiner contends that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to try the use of different detectors 310, each for identifying a section of the aircraft to be refueled and each having identical architecture and receiving the same type of training with random valued weights which would have been suggested by one of ordinary skill in the art seeking a specified result to minimize error in estimation. The motivation for this rejection is the same as that to claim 16.
With respect to claim 19, Leach teaches a data pre-processor component 340 that train images. At para. 26 beginning at line 14, the loss that is minimized during training is the pixel-wise mean-squared -error between the heat map and the ground truth heat map. This mean-squared-error is the computation of the standard deviation. The training images occur in 3D as set forth in para. 25, beginning at line 4. The motivation for this rejection is the same as that to claim 18.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEROME GRANT II whose telephone number is (571)272-7463. The examiner can normally be reached M-F 9:00 a.m. - 5:00 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nay Maung can be reached at 571-272-7882. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEROME GRANT II/Primary Examiner, Art Unit 2664