Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings were received on 5/6/2024. These drawings are accepted.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 5, 9-13, 16 and 18-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kircher et al. (US20230221795, hereinafter “Kircher”)
(Method Claims 1, 2, 5, 9-11, and 19-20 are stated after Claim 12, 13, 16 and 18)
Claim 12. Kirchner teaches A system for visual navigation, the system ([0007] “For a head-worn display unit (HWD; also, head-mounted or helmet-mounted display) to accurately display aircraft or georeferenced symbology and imagery”) comprising:
at least one processor; ([0054] “control processors 400”) and
at least one memory ([0054] “memory/data storage 402.”) storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, ([0058] “control processors 400 of the optical tracker 202 may include processes and/or process flows for optical feature extraction 408, optical pose estimation 410, and optical pose integrity modeling 412.”) the set of operations comprising: receiving a plurality of video frames ([0037] “capture a continuous stream of image data”) from an image sensor disposed on an aircraft; ([0065] “one or more cameras 108 rigidly attached to the aircraft”)
generating an image-based transform based on the plurality of video frames, ([0042] “Optical … devices may provide estimated head pose measurements (“pose estimates”) in six degrees of freedom (6DoF), e.g., translation along x-, y-, and z-axes (pitch, roll, yaw) and rotation relative to these axes.” Is understood to be the same as the claimed image-based transform in light of the instant specifications [0040])
the image-based transform being associated with a movement of one or more image features ([0045] “fiducial markers”) and a movement of the image sensor; ([0045] “image data 210 may be analyzed by the optical tracker 202 to identify image regions corresponding to the fiducial markers 110 and, by comparing the relative locations of the image regions to the known location of each fiducial marker, estimate the optical pose of the camera” pose of the camera is understood to be the same as the claimed movement of the image sensor in light of instant specifications 0066]) determining an image-based motion associated with the aircraft based on the image-based transform; ([0045] “the locations and orientations of each fiducial marker in an inside-out configuration are fixed with respect to the aircraft (100, FIG. 1)). For example, the optical tracker 202 may determine an initial (e.g., coarse) pose estimate based on an initial match of image regions associated with detected fiducial markers 110 with the 3D position and orientation data for each marker.”)
generating a georegistration transform based on at least one video frame of the plurality of video frames and a reference image; ([0048] “georeferenced pose data and corresponding low-rate head pose estimates 214 provided by the optical tracker” georeferencing is a synonym for georegistration https://en.wikipedia.org/wiki/Georeferencing)
determining a georegistration-based geolocation associated with the aircraft based on the georegistration transform; ([0048] “georeferenced pose data…provided by the optical tracker” is understood to be the same as the claimed geolocation in light of instant specifications [0060]) and
determining an aircraft ([0048] “aircraft”) geolocation ([0048] “georeferenced pose data … provided by the optical tracker”) by applying a non-linear Kalman filter ([0048] “determined by the hybrid EKF 208 may combine the extended Kalman filter for relative navigation” is understood to be the same as the claimed non-linear Kalman filter in light of instant specifications [0045] ) to the image-based motion ([0048] “head pose estimates 214 provided by the optical tracker may be the basis of a low-rate high integrity head pose solution” And [0048] “the low-rate high integrity solution may be updated by the hybrid EKF 208 based on the high-rate head-frame and platform-frame pose data”) and the georegistration-based geolocation. ([0048] “georeferenced pose data…provided by the optical tracker may be the basis of a low-rate high integrity head pose solution”) And [0048] “the low-rate high integrity solution may be updated by the hybrid EKF 208 based on the high-rate head-frame and platform-frame pose data”)
Claim 13. Kirchner teaches The system of claim 12, wherein the set of operations further comprises:
receiving metadata associated with a movement of the aircraft, ([0010] “(pIMU) fixed to an aircraft or other mobile platform, the pIMU providing a high-rate platform-referenced position and orientation”)
wherein the metadata includes at least one selected from a group consisting of a speed, ([0008] “velocity”) a heading, ([0048] “heading”) an altitude, ([0048] “altitude”) and an orientation, ([0041] “(pIMU) may track the relative position of the aircraft relative to a last known absolute position as determined by the GNSS receivers, as well as the orientation of the aircraft relative to its pitch, roll, and yaw axes (e.g., x-, y-, and z-axes).” Is understood to be the same as the claimed metadata orientation in light of instant specifications ) and
wherein the determining an aircraft location ([0042] “position and orientation of the aircraft 100”) comprises determining the aircraft location by applying the non-linear Kalman filter ([0042] “extended Kalman filter for relative navigation… pose of an aircraft 100”) to the image-based motion, ([0048] “head pose estimates 214 provided by the optical tracker may be the basis of a low-rate high integrity head pose solution” And [0048] “the low-rate high integrity solution may be updated by the hybrid EKF 208 based on the high-rate head-frame and platform-frame pose data”) the georegistration-based geolocation, ([0048] “georeferenced pose data…provided by the optical tracker may be the basis of a low-rate high integrity head pose solution”) And [0048] “the low-rate high integrity solution may be updated by the hybrid EKF 208 based on the high-rate head-frame and platform-frame pose data”)
and the metadata. ([0048] “updated the hybrid EKF 208 based on the …pIMU 206.”)
Claim 16. Kirchner teaches The system of claim 12, wherein the generating an image-based transform includes:
analyzing a first video frame of the plurality of video frames ([0037] “capture a continuous stream of image data”) to identify one or more first image features in the first video frame; ([0058] “process flows for optical feature extraction” and [])
analyzing a second video frame of the plurality of video frames to identify one or more second image features in the second video frame, each second image features of the one or more second image features matching one first image feature of the one or more first image features, the second video frame being after the first video frame in the plurality of video frames; ([0060] “pose estimate 416 is refined by the detection of fiducial markers 110 from the image data 210 at the current time of validity, even though the fiducial markers detected may not independently support a snapshot head pose estimate based solely on the detection of fiducial markers within a set of image data.” is understood to be the same as the claimed second image after the first because in a video image sequence the current time is after the previous frame which is already captured)
generating one or more motion vectors based on the one or more first image features and the one or more second image features, each motion vector of the one or more motion vectors corresponding to one of the one or more first image features and a matched second image features; and
generating the image-based transform based on the one or more motion vectors. ([0045] “the optical tracker 202 may refine the initial pose estimate using a variety of techniques and/or algorithms, resulting in a refined (e.g., fine) head pose estimate 214.” Tracking the pose from one frame to the next generates the motion vector upon which the new head pose estimate is generated)
Claim 18. Kirchner teaches The system of claim 12, wherein the set of operations further comprises:
determining a movement of the image sensor based on the image-based transform. ([0074] “At a step 712, the optical tracker determines, based on the image locations and corresponding constellation data for each identified marker, an initial optical pose of the camera”)
Claim 1. The method herein has been executed and performed by the system of claim 12 and is likewise rejected
Claim 2. The method herein has been executed and performed by the system of claim 13 and is likewise rejected
Claim 5. The method herein has been executed and performed by the system of claim 16 and is likewise rejected
Claim 9. The method herein has been executed and performed by the system of claim 18 and is likewise rejected
Claim 10. Kirchner teaches The method of claim 1, wherein the plurality of video frames are a first plurality of video frames; ([0037] “capture a continuous stream of image data”) wherein the image sensor is a first image sensor; ([0037] “optical tracker may include one …cameras 108” ) and
wherein the method further comprises: receiving a second plurality of video frames from a second image sensor, ([0037] “capture a continuous stream of image data”) the second image sensor being different from the first image sensor; ([0037] “optical tracker may include one or more cameras 108” and Fig. 5 shows item 108 including more than one camera which encompasses the second claimed camera)
PNG
media_image1.png
740
568
media_image1.png
Greyscale
generating a second image-based transform based on the second plurality of video frames; ([0042] “Optical … devices may provide estimated head pose measurements (“pose estimates”) in six degrees of freedom (6DoF), e.g., translation along x-, y-, and z-axes (pitch, roll, yaw) and rotation relative to these axes.” Is understood to be the same as the claimed image-based transform in light of the instant specifications [0040])
determining a second image-based motion associated with the aircraft based on the second image-based transform; ([0045] “the locations and orientations of each fiducial marker in an inside-out configuration are fixed with respect to the aircraft (100, FIG. 1)). For example, the optical tracker 202 may determine an initial (e.g., coarse) pose estimate based on an initial match of image regions associated with detected fiducial markers 110 with the 3D position and orientation data for each marker.”)
and determining the aircraft ([0048] “aircraft”) geolocation based at least in part on the second image-based motion. ([0048] “georeferenced pose data…provided by the optical tracker” is understood to be the same as the claimed geolocation in light of instant specifications [0060])
Claim 11. Kirchner teaches The method of claim 1, further comprising:
receiving a first aircraft location, ([0042] “position and orientation of the aircraft 100”)
wherein the determining an aircraft geolocation ([0010] “pIMU provides low-rate georeferenced pose data”)comprises determining the aircraft location based at least in part on the first aircraft location. ([0010] “(pIMU) fixed to an aircraft or other mobile platform, the pIMU providing a high-rate platform-referenced position and orientation”)
Claim 19. Kirchner teaches A method for visual navigation, the method comprising:
receiving a plurality of video frames ([0037] “capture a continuous stream of image data”) from an image sensor disposed on an aircraft; ([0065] “one or more cameras 108 rigidly attached to the aircraft”)
generating an image-based transform based on the plurality of video frames, ([0042] “Optical … devices may provide estimated head pose measurements (“pose estimates”) in six degrees of freedom (6DoF), e.g., translation along x-, y-, and z-axes (pitch, roll, yaw) and rotation relative to these axes.” Is understood to be the same as the claimed image-based transform in light of the instant specifications [0040])
the image-based transform being associated with a movement of one or more image features ([0045] “fiducial markers”) and a movement of the image sensor; ([0045] “image data 210 may be analyzed by the optical tracker 202 to identify image regions corresponding to the fiducial markers 110 and, by comparing the relative locations of the image regions to the known location of each fiducial marker, estimate the optical pose of the camera” pose of the camera is understood to be the same as the claimed movement of the image sensor in light of instant specifications 0066])
determining an image-based motion associated with the aircraft based on the image- based transform; ([0045] “the locations and orientations of each fiducial marker in an inside-out configuration are fixed with respect to the aircraft (100, FIG. 1)). For example, the optical tracker 202 may determine an initial (e.g., coarse) pose estimate based on an initial match of image regions associated with detected fiducial markers 110 with the 3D position and orientation data for each marker.”)
generating a georegistration transform based on at least one video frame of the plurality of video frames and a reference image; ([0048] “georeferenced pose data and corresponding low-rate head pose estimates 214 provided by the optical tracker” georeferencing is a synonym for georegistration https://en.wikipedia.org/wiki/Georeferencing)
determining a georegistration-based geolocation associated with the aircraft based on the georegistration transform; ([0048] “georeferenced pose data…provided by the optical tracker” is understood to be the same as the claimed geolocation in light of instant specifications [0060])
receiving metadata associated with the aircraft; ([0010] “(pIMU) fixed to an aircraft or other mobile platform, the pIMU providing a high-rate platform-referenced position and orientation”) and estimating an aircraft ([0048] “aircraft”) geolocation ([0048] “georeferenced pose data … provided by the optical tracker”) by applying a non-linear Kalman filter ([0048] “determined by the hybrid EKF 208 may combine the extended Kalman filter for relative navigation” is understood to be the same as the claimed non-linear Kalman filter in light of instant specifications [0045] ) to the image-based motion, ([0048] “head pose estimates 214 provided by the optical tracker may be the basis of a low-rate high integrity head pose solution” And [0048] “the low-rate high integrity solution may be updated by the hybrid EKF 208 based on the high-rate head-frame and platform-frame pose data”) the georegistration-based geolocation, ([0048] “georeferenced pose data…provided by the optical tracker may be the basis of a low-rate high integrity head pose solution”) And [0048] “the low-rate high integrity solution may be updated by the hybrid EKF 208 based on the high-rate head-frame and platform-frame pose data”) and the metadata, ([0048] “updated the hybrid EKF 208 based on the …pIMU 206.”)
wherein the method is performed using one or more processors. ([0054] “control processors 400”)
Claim 20. Kirchner teaches The method of claim 19, wherein the metadata includes at least one selected from a group consisting of a speed, ([0008] “velocity”) a heading, ([0048] “pIMU 206 may include a georeferenced…heading”) and an altitude ([0048] “pIMU 206 may include a georeferenced…altitude”) of the aircraft. ([0041] “(pIMU) may track the relative position of the aircraft… as well as the orientation of the aircraft relative to its pitch, roll, and yaw axes (e.g., x-, y-, and z-axes).”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Kircher et al. (US20230221795, hereinafter “Kircher”) and in view of Singhal et al (US20220057213, hereinafter “Singhal”)
Claim 3. The method herein has been executed and performed by the system of claim 14 and is likewise rejected
Claim 14. Kirchner teaches The system of claim 13, wherein the set of operations further comprises:
Kirchner does not explicitly teach extracting at least a part of the metadata from at least one video frame of the plurality of video frames.
Singhal teaches extracting at least a part of the metadata ([0041] “location of the aircraft can be determined. The direction, velocity, acceleration, and other aircraft travel datapoints can also be inferred as well…. process 300 can use a photogrammetrical approach to determine changes in aircraft altitude. Altitude can be investigated through photogrammetric successive image scale shifts.”) from at least one video frame of the plurality of video frames. ([0041] “use one or more digital camera(s) 208 to obtain digital images of the landscape beneath the aircraft…. analyze how key points have moved between frames to infer the motion of the aircraft.”)
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the claimed invention to modify Kirchner to have extracting part of the metadata from the plurality of video frames as taught by Singhal to arrive at the claimed invention discussed above. The motivation for the proposed modification would have been to have (Singhal [0004] “improvements to aircraft navigation”)
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kircher et al. (US20230221795, hereinafter “Kircher”) and in view of Beich et al (NPL “Tightly-Coupled Image-Aided Inertial Relative Navigation Using Statistical Predictive Rendering (SPR) Techniques and A Priori World Models”, hereinafter “Beich”)
Claim 4. The method herein has been executed and performed by the system of claim 15 and is likewise rejected
Claim 15. Kirchner teaches The system of claim 12,
Kirchner does not explicitly teach wherein the non-linear Kalman filter is an unscented Kalman filter.
Beich teaches wherein the non-linear Kalman filter is an unscented Kalman filter. (pg8PDF “acceleration (ΔV ) and angle rate (Δθ) measurements for use in the UKF” and pg3PDF “optimal estimation algorithm planned for use in this research is a modified Unscented Kalman Filter (UKF).” And Abstract “images of the ground from the perspective of an overflying air vehicle using an Extended Kalman Filter (EKF) or UKF,”)
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the claimed invention to modify Kirchner to have an Unscented Kalman filter as taught by Beich to arrive at the claimed invention discussed above. The motivation for the proposed modification would have been to have the (Beich Abstract “navigation solution accuracy and robustness is improved”)
Allowable Subject Matter
Claims 6, 7, 8 and 17 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Boggs et al US11914763 teaches determining georeferenced features and estimating a 6DoF pose based on a camera head tracker relative to an aircraft
Greveson et al US20190266396 teaches geographic position based on frame imagery and combining that with sensor measurements including position and orientation of the camera on the drone aircraft
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OWAIS MEMON whose telephone number is (571)272-2168. The examiner can normally be reached M-F (7:00am - 4:00pm) CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OWAIS I MEMON/Examiner, Art Unit 2663
Prosecution Insights