DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-8 and 15-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 5, which recites the limitation “wherein supplementing the extracted metadata comprises supplementing the extracted metadata with simulated metadata”, Examiner notes that claim 4 recites “wherein the extracted metadata supplements metadata received with the motion imagery data”. Therefore, claim 5 is unclear as written because it appears to suggest that the extracted metadata of claim 1 is being further supplemented, whereas claim 4 suggests the opposite (the extracted metadata is the supplemental data). Additionally, claim 5 recites “the simulated metadata being determined from specifications of the UAS platform, sample video data for the UAS platform, and/or one or more machine learning models receiving extracted metadata or metadata derived from the extracted metadata as input”, in which it is unclear if the “extracted metadata” received by the recited one or more machine learning models is the same as that of base claim 1. The same rationale applies to corresponding system claim 15. Appropriate clarification and correction is required. For purposes of examination, the limitations will be given their broadest reasonable interpretation in view of the available prior art and Examiner’s best understanding of Applicant’s specification. Claims 6-8 and 16-18 are additionally rejected for inheriting the deficiencies of their respective base claims 5 and 15.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5 and 11-15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Computer Vision-Based Technologies and Commercial Best Practices for the Advancement of the Motion Imagery Tradecraft” (hereinafter “Phipps”; published 2014).
Regarding claim 1, Phipps discloses a computer-implemented method for synthesizing metadata from motion imagery (Phipps, Introduction, Section 3.9; “Burned-in metadata recovery”), comprising:
receiving, by one or more processors, input motion imagery data comprising video content captured by an Unmanned Aerial System (UAS) platform and including text overlaid over the captured video content (Phipps, Section 3.9; “it is still common practice for many operational platforms to render certain elements of the platform and sensor metadata as a text-based overlay, which is “burned-in” to the video stream prior to compression/transmission”);
extracting, by the one or more processors, at least a portion of the text from the video content (Phipps, Section 3.9, second paragraph; “To leverage this idea, we have developed a system that recognizes text elements in the video image”);
correlating, by the one or more processors, the extracted text data with at least one metadata field to provide extracted metadata; (Phipps, Section 3.9, second paragraph; “To leverage this idea, we have developed a system that recognizes text elements in the video image and parses them to produce a MISP-compliant metadata stream”); and
storing the extracted metadata associated with the motion imagery data (Phipps, Section 2.2; “other intelligence data types (e.g. SIGINT, OSINT, etc.) have been captured, processed, and stored by legacy systems within the DoD/IC. Similarly, metadata and content formats vary across the existing data stores and are anticipated to continue to remain dynamic as new data is added to the mix.”).
Regarding claim 2, claim 1 is incorporated, and Phipps further discloses wherein the extracted metadata comprises geographic coordinates for a position associated with a camera used to capture the video content (Phipps, Section 3, third paragraph; “Fortunately, UAV motion imagery has another facet that can be highly beneficial, namely the presence of platform and sensor metadata. In particular, metadata about camera location, pointing angles and field-of-view proves to be an extremely useful complement to certain vision-based algorithms, as we shall describe”).
Regarding claim 3, claim 2 is incorporated, and Phipps further discloses wherein the geographic coordinates comprise geographic coordinates for a center of a field of view of the camera (Phipps, Section 3, third paragraph; “Fortunately, UAV motion imagery has another facet that can be highly beneficial, namely the presence of platform and sensor metadata. In particular, metadata about camera location, pointing angles and field-of-view proves to be an extremely useful complement to certain vision-based algorithms, as we shall describe”).
Regarding claim 4, claim 1 is incorporated, and Phipps further discloses wherein the extracted metadata supplements metadata received with the motion imagery data (Phipps, Section 3.9, second paragraph; “Even in many newer platforms, which also transmit a MISB-standard metadata stream, the on-screen metadata often exhibits better synchronization and update rates than the encoded KLV metadata…To leverage this idea, we have developed a system that recognizes text elements in the video image and parses them to produce a MISP-compliant metadata stream”).
Regarding claim 5, claim 4 is incorporated, and Phipps further discloses wherein supplementing the extracted metadata comprises supplementing the extracted metadata with simulated metadata, the simulated metadata being determined from specifications of the UAS platform, sample video data for the UAS platform, and/or one or more machine learning models receiving extracted metadata or metadata derived from the extracted metadata as input (Phipps, Section 3.9; “For a given platform feed, the required metadata elements generally lie at several fixed locations within the image. The recognizer is therefore directed to extract characters only at these locations, using a field template defined in a small JavaScript program, which also performs parsing of the recognizer outputs in order to populate the stripped metadata fields. Using JavaScript allows the user to implement sophisticated sanity checking and parsing based on the allowable values in certain fields and/or their variation over time, thus greatly increasing overall robustness”).
Claim 11 recites a system having features corresponding to the elements recited in method claim 1, the rejection of which is applicable here, and Phipps further discloses “at least one communications subsystem; memory; and at least one processor in operative communication with the at least one communications subsystem and memory” (Phipps, Section 3).
Claim 12 recites a system having features corresponding to the elements recited in method claim 2, the rejection of which is applicable here.
Claim 13 recites a system having features corresponding to the elements recited in method claim 3, the rejection of which is applicable here.
Claim 14 recites a system having features corresponding to the elements recited in method claim 4, the rejection of which is applicable here.
Claim 15 recites a system having features corresponding to the elements recited in method claim 5, the rejection of which is applicable here.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Phipps, as applied to claim 1, in view of “LocateUAV: Unmanned Aerial Vehicle Location Estimation via Contextual Analysis in an IoT Environment” (hereinafter “Dilshad”; published 2022).
Regarding claim 10, claim 1 is incorporated, and Dahle further teaches wherein a contrast level between the overlaid text and the captured video content varies over time (Dilshad, p. 4026-4027, Section III.C.-D.; “Owing to the continuous development of scenes, fast-moving objects and the mobility of UAVs cause motion blur, contrast, and brightness problems in captured images.”).
Dilshad is considered analogous art because it pertains to OCR-based text extraction from aerial images. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method taught by Phipps to accept and accommodate for changes in contrast level of overlaid text in UAV images, as taught by Dilshad, in order to ensure accurate OCR recognition (Dilshad, p. 4026-4027, Section III.C.-D.).
Claim 20 recites a system having features corresponding to the elements recited in method claim 10, the rejection of which is applicable here.
Allowable Subject Matter
Claims 6-8 and 16-18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claims 9 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
The closest prior art of record, either alone or in combination, does not expressly teach or render obvious the limitations of claim 6 and claim 9.
Regarding claims 6 and 16, neither Phipps nor Dilshad expressly teaches or suggests “wherein the extracted metadata comprises sensor frame coordinates or frame sensor coordinates, timestamp, altitude, and heading angle, and metadata derived from the extracted metadata comprises one or more of a speed, a rate of change of altitude, or a rate of change of heading angle”. In particular, Phipps (as cited above) does not expressly disclose or suggest deriving metadata comprising a speed, a rate of change of altitude, or a rate of change of heading angle from the extracted text-based metadata. Claims 7-8 and 17-18 are considered allowable by virtue of their dependency from dependent claim 6 and 16, respectively.
Regarding claims 9 and 19, neither Phipps nor Dilshad as cited above expressly teaches or suggests “computing a camera footprint using at least the extracted metadata, wherein computing the camera footprint comprises deriving camera footprint coordinates for at least one frame or timestamp of the motion imagery data.” At most, Phipps discloses determining camera position coordinates from the extracted text, and Dilshad discloses determining map-based location information from extracted text information.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The additionally provided references pertain generally to aerial image processing.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMAH A BEG whose telephone number is (571)270-7912. The examiner can normally be reached M-F 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HENOK SHIFERAW can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAMAH A BEG/Primary Examiner, Art Unit 2676
Prosecution Insights