DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/27/2026 has been entered.
Status of Claims
This non-final rejection is in response to Applicant’s amended filing of 03/27/2026.
Claims 1-4, 6-12, 14-18, 20-22, and 24 are currently pending and have been examined. Applicant has amended claims 1, 10, 16, and 21-22; cancelled claim 23; and added new claim 24.
Response to Arguments
Applicant’s arguments with respect to claims 1-4, 6-12, 14-18, and 20-23 rejected under 35 USC § 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 10-12, 16-18, and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Cramblitt (US 20190344902 A1) in view of Fu et al. (GB 2591445 A), Fang et al. (US 20240290034 A1), and Xu et al. (CN 115147576 A).
Regarding claim 1, Cramblitt discloses a method (see at least abstract) comprising:
receiving a two-dimensional (2D) image of a device associated with refueling operations between a receiver aircraft and a tanker aircraft (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a receiver aircraft fuel receptacle receiving fuel from a refueling aircraft);
and identifying keypoints on the device in the 2D image (see at least ¶ [0029-0030] disclosing the camera taking images and using the image to identify points on the receiver aircraft to determine its orientation, position, and pose).
Cramblitt does not explicitly disclose projecting the keypoints to three-dimensional (3D) space to produce 3D keypoints that are predicted from the keypoints.
However, Fu suggests projecting the keypoints to three-dimensional (3D) space to produce 3D keypoints that are predicted from the keypoints (see at least ¶ [0046-0047] and [0067] disclosing a UAV capturing images of aircraft features and mapping them to three-dimensional data).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the 2D to 3D mapping process of Fu into the refueling system of Cramblitt with a reasonable expectation of success because both inventions utilize cameras to obtain images and detect features of an aircraft. This would help the refueling (tanker) aircraft to properly determine the receiver aircraft’s orientation, position, and pose in three-dimensional space while flying.
While Fu suggests comparing the 3D keypoints to a previously stored 3D model of the device (see at least ¶ [0019] and [0068] disclosing comparing collected images of aircrafts to stored models) and Cramblitt suggests automatically controlling the refueling operations (see at least ¶ [0036] disclosing the refueling operations being performed by an automated system), the combination of Cramblitt and Fu does not disclose comparing the 3D keypoints to a previously stored 3D model of the device by:
identifying one or more geometric relationships between the 3D keypoints, wherein the one or more geometric relationships comprises at least one of an angle and a distance between a pair of the 3D keypoints;
and comparing the one or more geometric relationships to one or more known geometric relationships corresponding to pairs of keypoints on the previously stored 3D model to produce a confidence value based on a difference between the one or more geometric relationships and the one or more known geometric relationships;
and automatically controlling the refueling operations in response to the confidence value.
However, Fang suggests identifying one or more geometric relationships between the 3D keypoints, wherein the one or more geometric relationships comprises at least one of an angle and a distance between a pair of the 3D keypoints (see at least ¶ [0041], [0051], [0060-0062], [0082-0083], [0095-0097], and [0157] disclosing an image processing method that identifies and connects joints of a pose skeleton, whereby reconstruction of the skeleton verifies the accuracy of the distances between joints using confidence values);
and comparing the one or more geometric relationships to one or more known geometric relationships corresponding to pairs of keypoints on the previously stored 3D model to produce a confidence value based on a difference between the one or more geometric relationships and the one or more known geometric relationships (see at least ¶ [0041], [0051], [0060-0062], [0082-0083], [0095-0097], and [0157] disclosing an image processing method that identifies and connects joints of a pose skeleton, whereby reconstruction of the skeleton verifies the accuracy of the distances between joints using confidence values by comparing the distances to pre-formed joint distance datasets).
Additionally, Xu suggests automatically controlling the refueling operations in response to the confidence value (see at least page 2 of the machine translation starting at “According to the pose of the center of the docking station relative to the robot…” disclosing path planning and guiding the underwater robot to the docking station).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the geometric relationships and confidence values of Fang and the refueling control based on confidence value of Xu into the combination of Cramblitt and Fu with a reasonable expectation of success because all inventions are directed toward using cameras to obtain 2D images and detect features of objects and translate and compare those features to established 3D information, for helping vehicles connect to a refueling or docking location. While Fang is not directed toward modelling keypoints for aerial refueling, the features of associating geometric relationships to 3D points and assigning confidence values to comparisons against a standard are directly applicable to the 3D modelling implementations of Cramblitt and Fu. Additionally, while Xu is directed toward underwater vehicles and the combination of Cramblitt and Fu are directed toward midair refueling, Xu describes computing a confidence value for 2D imaging data and relates it to 3D coordinates of features at a docking station and would be applicable to the imaging systems for connecting a refueling boom to an aircraft. These would help improve the accuracy of visual information transformed into three-dimensional space, and subsequently improve refueling operations.
Regarding claims 10 and 16, Cramblitt discloses a tanker aircraft (claim 10; see at least abstract disclosing a refueling aircraft) and an automated refueling system (claim 16; see at least abstract) comprising:
a camera configured to generate a two-dimensional (2D) image of a device associated with refueling operations between a receiver aircraft and the tanker aircraft see at least abstract and ¶ [0012] disclosing a camera for capturing images of a receiver aircraft fuel receptacle receiving fuel from a refueling aircraft;
a processor (see at least ¶ [0012] disclosing a processor on the refueling aircraft);
and non-transitory computer readable storage media storing code (see at least ¶ [0013] disclosing refueling operations performed from program instructions from non-transitory computer-readable storage media), the code being executable by the processor to perform operations comprising:
identifying keypoints on the device in the 2D image (see at least ¶ [0029-0030] disclosing the camera taking images and using the image to identify points on the receiver aircraft to determine its orientation, position, and pose).
Cramblitt does not explicitly disclose projecting the keypoints to three-dimensional (3D) space to produce 3D keypoints that are predicted from the keypoints.
However, Fu suggests projecting the keypoints to three-dimensional (3D) space to produce 3D keypoints that are predicted from the keypoints (see at least ¶ [0046-0047] and [0067] disclosing a UAV capturing images of aircraft features and mapping them to three-dimensional data).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the 2D to 3D mapping process of Fu into the refueling system of Cramblitt with a reasonable expectation of success because both inventions utilize cameras to obtain images and detect features of an aircraft. This would help the refueling (tanker) aircraft to properly determine the receiver aircraft’s orientation, position, and pose in three-dimensional space while flying.
While Fu suggests comparing the 3D keypoints to a previously stored 3D model of the device (see at least ¶ [0019] and [0068] disclosing comparing collected images of aircrafts to stored models) and Cramblitt suggests automatically controlling the refueling operations (see at least ¶ [0036] disclosing the refueling operations being performed by an automated system), the combination of Cramblitt and Fu does not disclose comparing the 3D keypoints to a previously stored 3D model of the device by:
identifying one or more geometric relationships between the 3D keypoints, wherein the one or more geometric relationships comprises at least one of an angle and a distance between a pair of the 3D keypoints;
and comparing the one or more geometric relationships to one or more known geometric relationships corresponding to pairs of keypoints on the previously stored 3D model to produce a confidence value based on a difference between the one or more geometric relationships and the one or more known geometric relationships;
and automatically controlling the refueling operations in response to the confidence value.
However, Fang suggests identifying one or more geometric relationships between the 3D keypoints, wherein the one or more geometric relationships comprises at least one of an angle and a distance between a pair of the 3D keypoints (see at least ¶ [0041], [0051], [0060-0062], [0082-0083], [0095-0097], and [0157] disclosing an image processing method that identifies and connects joints of a pose skeleton, whereby reconstruction of the skeleton verifies the accuracy of the distances between joints using confidence values);
and comparing the one or more geometric relationships to one or more known geometric relationships corresponding to pairs of keypoints on the previously stored 3D model to produce a confidence value based on a difference between the one or more geometric relationships and the one or more known geometric relationships (see at least ¶ [0041], [0051], [0060-0062], [0082-0083], [0095-0097], and [0157] disclosing an image processing method that identifies and connects joints of a pose skeleton, whereby reconstruction of the skeleton verifies the accuracy of the distances between joints using confidence values by comparing the distances to pre-formed joint distance datasets).
Additionally, Xu suggests automatically controlling the refueling operations in response to the confidence value (see at least page 2 of the machine translation starting at “According to the pose of the center of the docking station relative to the robot…” disclosing path planning and guiding the underwater robot to the docking station).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the geometric relationships and confidence values of Fang and the refueling control based on confidence value of Xu into the combination of Cramblitt and Fu with a reasonable expectation of success because all inventions are directed toward using cameras to obtain 2D images and detect features of objects and translate and compare those features to established 3D information, for helping vehicles connect to a refueling or docking location. While Fang is not directed toward modelling keypoints for aerial refueling, the features of associating geometric relationships to 3D points and assigning confidence values to comparisons against a standard are directly applicable to the 3D modelling implementations of Cramblitt and Fu. Additionally, while Xu is directed toward underwater vehicles and the combination of Cramblitt and Fu are directed toward midair refueling, Xu describes computing a confidence value for 2D imaging data and relates it to 3D coordinates of features at a docking station and would be applicable to the imaging systems for connecting a refueling boom to an aircraft. These would help improve the accuracy of visual information transformed into three-dimensional space, and subsequently improve refueling operations.
Regarding claim 2, Cramblitt discloses the device is the receiver aircraft (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a receiver aircraft fuel receptacle receiving fuel from a refueling aircraft).
Regarding claim 3, Cramblitt discloses the device is a refueling boom (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a refueling boom for refueling a receiver aircraft).
Regarding claims 4, 12, and 18, Cramblitt does not disclose projecting the keypoints to 3D space is based on a 2D to 3D correspondence model.
However, Fu suggests projecting the keypoints to 3D space is based on a 2D to 3D correspondence model (see at least ¶ [0066] disclosing using perspective-n-point methodology to map 2D features to 3D features).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the 2D to 3D mapping process of Fu into the refueling system of Cramblitt with a reasonable expectation of success because both inventions utilize cameras to obtain images and detect features of an aircraft. This would help the refueling (tanker) aircraft to properly determine the receiver aircraft’s orientation, position, and pose in three-dimensional space while flying.
Regarding claims 11, Cramblitt discloses a refueling boom (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a refueling boom for refueling a receiver aircraft), wherein the device is the receiver aircraft or the refueling boom (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a receiver aircraft fuel receptacle receiving fuel from a refueling aircraft).
Regarding claim 17, Cramblitt discloses the device is the receiver aircraft or a refueling boom (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a refueling boom and a receiver aircraft fuel receptacle for refueling the receiver aircraft).
Regarding claims 21 and 22, the combination of Cramblitt and Fu does not explicitly disclose the one or more geometric relationships and the one or more known geometric relationships include a vector from one of the 3D keypoints to another of the 3D keypoints and a corresponding vector from one of the corresponding keypoints of the previously stored 3D model to another of the corresponding keypoints of the previously stored 3D model.
However, Xu suggests the one or more geometric relationships and the one or more known geometric relationships include a vector from one of the 3D keypoints to another of the 3D keypoints and a corresponding vector from one of the corresponding keypoints of the previously stored 3D model to another of the corresponding keypoints of the previously stored 3D model (see at least pages 2-3 of the machine translation starting with “If the total value is greater than the predetermined threshold…” disclosing an underwater robot imaging key points at a docking station and mapping 2D coordinates to 3D coordinates by calculating deviations and confidence values based on key point vector information).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the confidence value of Xu into the combination of Cramblitt and Fu with a reasonable expectation of success because all inventions are directed toward using cameras to obtain 2D images and detect features of objects and translate those features into 3D information for vehicles that connect to a refueling or docking location. While Xu is directed toward underwater vehicles and the combination of Cramblitt and Fu are directed toward midair refueling, Xu describes computing a confidence value for 2D and 3D imaging of features at a docking station and would be applicable to the imaging systems for connecting a refueling boom to an aircraft. This would help improve the accuracy of visual information transformed into three-dimensional space, and subsequently improve refueling operations.
Claims 6-9, 14-15, 20, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Cramblitt in view of Fu et al., Fang et al., and Xu et al., as applied to claims 1, 10, and 16, and in further view of Foo et al. (US 20140346279 A1).
Regarding claims 6 and 14, while Cramblitt discloses automatically controlling the refueling operations (see at least ¶ [0036] disclosing the refueling operations being performed by an automated system) and Xu suggests monitoring the confidence value is below a predefined threshold (see at least pages 10-11 of the machine translation starting with “Among them, Losscls refers to the loss of category confidence…” disclosing the predicted confidence value such that corresponding data is used for key point training if it exceeds a given threshold) the combination of Cramblitt, Fu, Fang, and Xu does not disclose disengaging the refueling operations.
However, Foo teaches automatically controlling the refueling operations comprises disengaging the refueling operations (see at least ¶ [0061-0062] and [0091] disclosing automatically disconnecting and retracting the fueling boom after completing refueling).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the automatic disconnection of Foo into the combination of Cramblitt, Fu, Fang, and Xu with a reasonable expectation of success because both Foo and Cramblitt are directed to aerial refueling systems. Foo demonstrates that automatic disconnection of refueling systems and devices once operations are complete are known and obvious to one of ordinary skill in the art.
Regarding claim 7, the combination of Cramblitt, Fu, Fang, and Xu does not disclose disengaging the refueling operations comprises controlling pilot director lights.
However, Foo suggests disengaging the refueling operations comprises controlling pilot director lights (see at least ¶ [0054] and [0064] disclosing a control module configured to control director lights for following refueling operations).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the automatic disconnection of Foo into the combination of Cramblitt, Fu, Fang, and Xu with a reasonable expectation of success because both Foo and Cramblitt are directed to aerial refueling systems. Foo demonstrates that automatic disconnection of refueling systems and devices once operations are complete are known and obvious to one of ordinary skill in the art.
Regarding claim 8, the combination of Cramblitt, Fu, Fang, and Xu does not disclose disengaging the refueling operations comprises controlling an image presented on a boom operator monitor.
However, Foo suggests disengaging the refueling operations comprises controlling an image presented on a boom operator monitor (see at least ¶ [0058], [0064], [0078], and [0099] disclosing cameras and videos being used to view and control the boom).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the automatic disconnection of Foo into the combination of Cramblitt, Fu, Fang, and Xu with a reasonable expectation of success because both Foo and Cramblitt are directed to aerial refueling systems. Foo demonstrates that systems and devices such as director lights, boom controllers, and automated flight controllers are known and obvious to one of ordinary skill in the art.
Regarding claim 9, Cramblitt discloses the device is a refueling boom (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a refueling boom for refueling a receiver aircraft).
The combination of Cramblitt, Fu, Fang, and Xu does not disclose disengaging the refueling operations comprises moving the refueling boom to a boom disengagement position or navigating the tanker aircraft to an aircraft disengagement position.
However, Foo teaches disengaging the refueling operations comprises moving the refueling boom to a boom disengagement position or navigating the tanker aircraft to an aircraft disengagement position (see at least ¶ [0061-0062] and [0091] disclosing automatically disconnecting and retracting the fueling boom after completing refueling).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the automatic disconnection of Foo into the combination of Cramblitt, Fu, Fang, and Xu with a reasonable expectation of success because both Foo and Cramblitt are directed to aerial refueling systems. Foo demonstrates that automatic disconnection of refueling systems and devices once operations are complete are known and obvious to one of ordinary skill in the art.
Regarding claim 15, Cramblitt discloses a boom operator monitor (see at least abstract and ¶ [0012] disclosing a camera for capturing images of a refueling boom for refueling a receiver aircraft).
The combination of Cramblitt, Fu, Fang, and Xu does not explicitly disclose pilot director lights;
a controller;
and an autopilot system, wherein disengaging the refueling operations comprises:
controlling the pilot director lights;
controlling an image presented on the boom operator monitor;
instructing the controller to move the refueling boom to a boom disengagement position;
or instructing the autopilot system to navigate the aircraft to an aircraft disengagement position.
However, Foo teaches pilot director lights (see at least ¶ [0054] and [0064] disclosing a control module configured to control director lights for following refueling operations);
a controller (see at least ¶ [0052] and [0056] disclosing a flight control module);
and an autopilot system (see at least ¶ [0052] disclosing an automatic flight control module), wherein disengaging the refueling operations comprises:
controlling the pilot director lights (see at least ¶ [0054] and [0064] disclosing a control module configured to control director lights for following refueling operations);
controlling an image presented on the boom operator monitor (see at least ¶ [0058], [0064], [0078], and [0099] disclosing cameras and videos being used to view and control the boom);
instructing the controller to move the refueling boom to a boom disengagement position (see at least ¶ [0061-0062] and [0091] disclosing automatically disconnecting and retracting the fueling boom after completing refueling);
or instructing the autopilot system to navigate the aircraft to an aircraft disengagement position.
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the automatic disconnection of Foo into the combination of Cramblitt, Fu, Fang, and Xu with a reasonable expectation of success because both Foo and Cramblitt are directed to aerial refueling systems. Foo demonstrates that systems and devices such as director lights, boom controllers, and automated flight controllers are known and obvious to one of ordinary skill in the art.
Regarding claim 20, the combination of Cramblitt, Fu, Fang, and Xu does not explicitly disclose automatically controlling the refueling operations comprises:
controlling pilot director lights;
controlling an image presented on a boom operator monitor;
controlling the refueling boom to move to a boom disengagement position;
or controlling the tanker aircraft to navigate to an aircraft disengagement position.
However, Foo discloses controlling pilot director lights (see at least ¶ [0054] and [0064] disclosing a control module configured to control director lights for following refueling operations);
controlling an image presented on a boom operator monitor (see at least ¶ [0058], [0064], [0078], and [0099] disclosing cameras and videos being used to view and control the boom);
controlling the refueling boom to move to a boom disengagement position (see at least ¶ [0061-0062] and [0091] disclosing automatically disconnecting and retracting the fueling boom after completing refueling);
or controlling the tanker aircraft to navigate to an aircraft disengagement position.
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the automatic disconnection of Foo into the combination of Cramblitt, Fu, Fang, and Xu with a reasonable expectation of success because both Foo and Cramblitt are directed to aerial refueling systems. Foo demonstrates that systems and devices such as director lights, boom controllers, and automated flight controllers are known and obvious to one of ordinary skill in the art.
Regarding claim 24, the combination of Cramblitt, Fu, Fang, and Xu does not explicitly disclose presenting a graphic on the image presented on the boom operator monitor, wherein:
the device is a refueling boom;
the graphic comprises one or more directional indicators of a position of the refueling boom relative to an optimum refueling position;
and controlling the image comprises controlling the one or more directional indicators of the graphic.
However, Foo suggests presenting a graphic on the image presented on the boom operator monitor (see at least ¶ [0058], [0064], [0078], and [0099] disclosing cameras and videos being used to display views to an aerial refueling operator (ARO) to help control the boom), wherein:
the device is a refueling boom (see at least ¶ [0058], [0064], [0078], and [0099] disclosing cameras and videos being used to view and control the refueling boom);
the graphic comprises one or more directional indicators of a position of the refueling boom relative to an optimum refueling position (see at least ¶ [0054], [0064], and [0078] disclosing a control module configured to control director lights for following refueling operations);
and controlling the image comprises controlling the one or more directional indicators of the graphic (see at least ¶ [0054], [0064], and [0078] disclosing a control module configured to control director lights for following refueling operations as well as visual laser beams viewable by the ARO to verify alignment ).
It would be obvious to one of ordinary skill in the art before the effective filing date of the present invention to incorporate the automatic disconnection of Foo into the combination of Cramblitt, Fu, Fang, and Xu with a reasonable expectation of success because both Foo and Cramblitt are directed to aerial refueling systems. Foo demonstrates that systems and devices such as director lights, boom controllers, and automated flight controllers are known and obvious to one of ordinary skill in the art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JARED C BEAN whose telephone number is (571)272-5255. The examiner can normally be reached 7:30AM - 5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Navid Z Mehdizadeh can be reached at (571) 272-7691. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.C.B./Examiner, Art Unit 3669
/NAVID Z. MEHDIZADEH/Supervisory Patent Examiner, Art Unit 3669
Prosecution Insights