DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN202210265427.9, filed on 03/17/2022.
Acknowledgment is made of applicant' s submission for Domestic Benefit/National State Information under 35 U.S.C. 371 for PCT/CN2022/141709 with filing date 12/24/2022.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 09/16/2024 and 07/17/2025 are in compliance with the provisions of 35 CFR 1.97. Accordingly, the IDS have been considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
The Specification throughout uses the terms “image processing module,” “data processing module,” “image fusion module,” “image compression module,” and “platform control and driver module” (Spec. [0004-0014], [0023-0049], [0064-0090), while the claims use “processor” for each corresponding element. The Specification at [0035] introduces a “geographical tracking unit,” a “tracking module,” and a “moving target detection module” — which the claims rename as “geographical tracking processor,” “tracking processor,” and “moving target detection processor,” respectively. The Applicant should harmonize the terminology between the specification, images and the claims so that each claim element has clearly corresponding antecedent disclosure under the same name.
Appropriate correction is required.
Drawings
The drawings are objected to because: FIG. 2 and FIG. 4 of the application use the term “image processing module,” “data processing module,” “image fusion module,” “image compression module,” and “platform control and driver module,” whereas the claims recite the corresponding “…processor” elements. Examiner notes that there are drawing-claim terminology mismatches and recommends amending the drawings (or claims) to use uniform nomenclature.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 1-4, 8 objected to because of the following informalities:
Claim 1 first introduces “an electronics cabin” and thereafter refers to “the electronics cabin,” but the third clause recites “the load cabin is electrically coupled to the electronic cabin” (singular “electronic”, omitting the trailing “s”). It is unclear whether “the electronic cabin” is intended to refer back to “the electronics cabin” or to a different element.
Claim 1 introduces “a platform control and driver processor” (with “driver”), but subsequent recitations in the same claim refer to “the platform drive processor” (in the data-processor clause) and “the platform control and drive processor” (in the next clause). Claim 2 likewise uses “the platform control and drive processor.” Three different names are used for what appears to be the same element.
Claim 2 introduces “a geographical tracking processor” in the wherein clause, but later in the same claim refers to “the geographic tracking processor” (omitting the “al” suffix), and still later refers to “the geographical tracking processor.”
Claim 1 recites “the infrared thermal imager, performs image sampling, obtain an infrared image signal” — “obtain” should be “obtains.”
Claim 1 recites “performs image registration and fusion processing on the corrected optical image and the corrected radar image in sequence, obtain an original image, and send the original image” — “obtain” should be “obtains” and “send” should be “sends.”
Claim 2 recites “measures a target offset and send the target offset” — “send” should be “sends.”
Claim 2 recites “parsing the target offset to obtain the target miss angle” — “parsing” should be “parses” for consistency with the surrounding finite verbs (“receives… sends”).
Claim 3 recites “performs echo data sampling and sending echo data,” “positioning the visible light video signal and infrared video signal… to obtain an optical video,” and “completing moving target detection and heterosource video fusion reconnaissance” — the “‐ing” forms are inconsistent with the finite-verb construction used in the rest of the same paragraph; “sending” should be “sends,” “positioning” should be “positions,” and “completing” should be “completes” or similar.
Claim 4 and 8 recite “the multi-source imaging reconnaissance mode,” but Claim 1, from which Claim 4 depends, introduces only “a radar multi-source reconnaissance mode.” The terms differ in two respects: Claim 1 uses “radar” as the modifier whereas Claim 4 uses “imaging”; and Claim 1 uses “reconnaissance” as the noun whereas Claim 4 uses “imaging reconnaissance.” It is unclear whether “the multi-source imaging reconnaissance mode” of Claim 4 is intended to refer back to “a radar multi-source reconnaissance mode” of Claim 1 or to a different operational mode.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“platform control and driver processor” / “platform control and drive processor” / “platform drive processor” in claims 1 and 2. Function: “receives the load cabin adjustment angle… and drives, according to the load cabin adjustment angle, the cantilever to rotate the load cabin to aim at the target.” Corresponding structure: Spec. [0012], [0032] disclose a “platform control and drive module” that performs the function but do not disclose the specific motor-control structure or actuator driver circuit that effects the rotation.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites “the platform drive processor” in the data-processor clause. There is no antecedent basis for this limitation in the claim. Claim 1 introduces “a platform control and driver processor” as one of five named processors in the electronics cabin, and later refers to this element as “the platform control and drive processor,” which a person having ordinary skills in the art (PHOSITA) would recognize as the same element under a trivial grammatical variation. However, the earlier reference to “the platform drive processor” — which omits “control and” entirely — does not correspond to any introduced element. Because claim 1 expressly recites five distinct named processors, a PHOSITA cannot determine with reasonable certainty whether “the platform drive processor” identifies the same element as “a platform control and driver processor” or constitutes a separate, unintroduced element. The metes and bounds of claim 1 are therefore indeterminate under MPEP § 2173.05(e). Claims 2–9, which depend from claim 1 and incorporate this defect through dependency, are rejected for the same reason. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites “a visual axis of the optical sensor” in the parallelism limitation. There is no antecedent basis for “the optical sensor” in the claim. Neither claim 8 nor its parent claims — claim 7 and claim 1 — introduce an element designated as “an optical sensor.” Claim 1 introduces four sensors: a visible light camera, an infrared thermal imager, a laser rangefinder, and a radar. Each of the visible light camera, the infrared thermal imager, and the laser rangefinder is an optical sensor in the sense understood by a PHOSITA. Because claim 8 recites the parallelism between a beam direction of the radar and a visual axis of the optical sensor without identifying which of these three introduced sensors constitutes the optical sensor, and because the visual axes of those three sensors are potentially distinct measurement references for the recited 0.5° requirement, a PHOSITA cannot determine with reasonable certainty which element defines the visual axis of the optical sensor as used in claim 8. The metes and bounds of claim 8 are therefore unclear under MPEP § 2173.05(e).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–6 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over De et al. (CN104808558A) in view of Yang et al. (CN113960591A).
Regarding Claim 1, De et al. (’558) in view of Yang et al. (’591) teaches:
De et al. (’558) teaches: A system for photoelectric radar comprehensive reconnaissance ([0009]: “The object of the present invention is to provide a multi-task load system suitable for a special general-purpose aircraft, which can set a plurality of task loads on a special general-purpose aircraft, fuse the output results of various task loads, improve target recognition accuracy”).
De et al. (’558) teaches: comprising an electronics cabin ([0025-0026]: “The integrated processor specifically includes: Load data interface module, data fusion processing module, task management module, general data interface module”; [0073], [0074]: “The integrated processor 15 specifically includes: Load data interface module 21, data fusion processing module 22, task management module 23, general data interface module 24”). The integrated processor 15, together with the data storage device, onboard data terminal, and position and attitude measuring system, constitutes the airborne electronics processing and communication subsystem of the multi-task load system, corresponding under the broadest reasonable interpretation to the electronics cabin.
De et al. (’558) teaches: a cantilever and a load cabin ([0057]: “The multitasking load system suitable for special general purpose aircraft includes: aerial camera 11, photoelectric pod 12, multifunction radar 13, mission planning map display 14, integrated processor 15”; [0059]: “The photoelectric pod 12 is configured to perform a monitoring task on the ground target according to a control command sent by the integrated processor, and send the monitoring information to the integrated processor”). De et al.’s photoelectric pod 12 constitutes a self-contained sensor payload unit (load cabin) that is physically separated from the aircraft’s avionics body and mounted thereon via a structural support arm (cantilever), an arrangement inherent in the term “pod” as used in the airborne reconnaissance context.
De et al. (’558) teaches: the load cabin being installed on a side wall of the electronics cabin through the cantilever, and the load cabin is electrically coupled to the electronic cabin ([0075]: “The load data interface module 21 is connected to the aerial camera 11, the photoelectric pod 12, the multi-function radar 13, and the mission planning map display 14, respectively, to the aerial camera 11, the photoelectric cabin 12, the multi-function radar 13, and the mission planning map display 14 perform data interaction”). The load data interface module electrically couples the sensor payload (load cabin) to the integrated processor (electronics cabin) through the data interface connections.
De et al. (’558) teaches: the load cabin is provided with a visible light camera ([0022], [0067]: “visible light imaging…performed on ground targets in accordance with control commands sent by the integrated processor”);
De et al. (’558) teaches: an infrared thermal imager ([0022], [0067]: “Infrared imaging…performed on ground targets in accordance with control commands sent by the integrated processor”);
De et al. (’558) teaches: a laser rangefinder ([0022], [0067]: “laser ranging…performed on ground targets in accordance with control commands sent by the integrated processor”);
De et al. (’558) teaches: and a radar therein ([0060], [0068]: “The multi-function radar 13 is configured to complete an imaging task according to a control instruction of the integrated processor, and send target image information and target location information to the integrated processor”; [0024], [0069]: “And performing a wide area search imaging task, an active target indicator imaging task, and a synthetic aperture radar imaging task according to the control instructions of the integrated processor”).
De et al. (’558) teaches: the electronics cabin is provided with an image processor ([0028]: “The data fusion processing module is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image”; [0076]: “The data fusion processing module 22 is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image according to a control instruction sent by the universal data interface module 24”). De et al.’s data fusion processing module 22 receives and processes the visible light and infrared image data from the photoelectric pod, corresponding under BRI to the image processor configured to receive and process optical image signals from the visible light camera and infrared thermal imager.
De et al. (’558) teaches: a data processor ([0015]: “The multifunctional radar is configured to complete an imaging task according to a control instruction of the integrated processor, and send target image information and target location information to the integrated processor”; [0028], [0076]: “perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image”). De et al.’s integrated processor receives and processes radar echo data and target image information from the multi-function radar, corresponding under BRI to the data processor configured to receive radar echo data and convert it into a radar image.
De et al. (’558) teaches: an image fusion processor ([0028]: “The data fusion processing module is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image”; [0095]: “It is especially important to synthesize the image information obtained by SAR and photoelectric sensors and to take advantage of their respective advantages. On the one hand, the advantages of the signals received by each sensor can be utilized at the same time, and on the other hand, the defects of each sensor can be compensated”). De et al.’s data fusion processing module directly teaches the image fusion processor configured to perform image registration and fusion on corrected optical and radar images.
De et al. (’558) teaches: an image compression processor ([0018]: “the merged processed data is packaged and delivered to the onboard data terminal”; [0019], [0064]: “The onboard data terminal 16 sends the packaged data sent by the integrated processor to the ground data terminal by using a wireless signal”). De et al. teaches that fused reconnaissance imagery is packaged and wirelessly transmitted to a ground data terminal. Image compression is an inherent and known component of the packaging step for wireless transmission of high-resolution imagery over a bandwidth-constrained airborne link, and De et al.’s data packaging function corresponds under BRI to the image compression processor.
De et al. does not explicitly teach and a platform control and driver processor therein — De et al. does not explicitly teach a dedicated platform control and driver processor that derives a load cabin adjustment angle from processed radar data and drives a cantilever mechanism to rotate the load cabin to aim at a detected target. However, Yang et al. (’591) teaches: a platform control and driver processor ([0036], [0084]: “extracts the radar target position information of the target to be identified and the current position and attitude information of the unmanned ship, calculates the photoelectric eye-pointing angle, and controls the photoelectric pointing device to locate the eye-pointing angle”; [0070]: “Servo control includes a variety of servo mode switching, such as manual search, eye finger, tracking mode switching and control…When performing eye-finger control, in order to ensure that the photoelectric energy can accurately reach the target detected by the radar, the attitude and heading information of the inertial navigation are required. At the same time, the photoelectric azimuth and pitch angle are used for coordinate system conversion, from the deck coordinate system to the earth coordinate system, calculate the optical axis angle in the geodetic coordinate system, and then reversely convert it to the deck coordinate system, calculate the angle that the photoelectric needs to rotate, and perform precise eye-finger control”). Yang et al.’s integrated control system receives radar-derived target position data, computes the required sensor pointing angle (load cabin adjustment angle), and drives the servo mechanism to rotate the photoelectric pointer (load cabin) to aim at the radar-detected target, directly teaching the platform control and driver processor.
De et al. (’558) in view of Yang et al. (’591) further teaches, when the system is in a radar multi-source reconnaissance mode:
De et al. (’558) teaches: the visible light camera, performs image sampling, obtains a visible light image signal, and sends the visible light image signal to the image processor ([0022], [0067]: “visible light imaging…performed on ground targets in accordance with control commands sent by the integrated processor”; [0013]: “The aerial camera is configured to send digital photo information to the integrated processor after taking a picture of the ground target”).
De et al. (’558) teaches: the infrared thermal imager, performs image sampling, obtain an infrared image signal, and sends the infrared image signal to the image processor ([0022], [0067]: “Infrared imaging…performed on ground targets in accordance with control commands sent by the integrated processor”).
De et al. (’558) teaches: the laser rangefinder, measures a distance value between the load cabin and a target, and sends the distance value to the image processor ([0022], [0067]: “laser ranging…performed on ground targets in accordance with control commands sent by the integrated processor”; [0017]: “The position and attitude measuring system is coupled to the integrated processor for providing position and attitude measurement data for the aerial camera, the photoelectric pod, and the multi-function radar”).
De et al. (’558) teaches: the image processor, receives the distance value from the laser rangefinder, receives the visible light image signal from the visible light camera, receives the infrared image signal from the infrared thermal imager, and performs geometric correction on the visible light image signal and the infrared image signal to generate a corrected optical image, and sends the corrected optical image to the image fusion processor ([0028], [0076]: “The data fusion processing module is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image”; [0017]: “The position and attitude measuring system is coupled to the integrated processor for providing position and attitude measurement data for the aerial camera, the photoelectric pod, and the multi-function radar”). De et al.’s data fusion processing module receives visible light and infrared image data together with position and attitude measurement data (inclusive of laser rangefinder distance data), and performs data fusion that inherently requires geometric correction as a prerequisite preprocessing step for registering multi-sensor images from differing viewing geometries. Performing geometric correction on optical imagery using position, attitude, and range data prior to multi-sensor fusion is a standard and well-known technique in the image processing art.
De et al. (’558) teaches: the radar, performs echo data sampling and sends echo data to the data processor ([0060], [0068]: “The multi-function radar 13 is configured to complete an imaging task according to a control instruction of the integrated processor, and send target image information and target location information to the integrated processor”; [0024], [0069]: “performing a wide area search imaging task, an active target indicator imaging task, and a synthetic aperture radar imaging task”).
De et al. (’558) teaches: the data processor, receives the echo data from the radar, converts the echo data into a radar image, performs geometric correction on the radar image to generate a corrected radar image, and sends the corrected radar image to the image fusion processor ([0028]: “perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image”; [0076]: same teaching in embodiment description). De et al.’s integrated processor receives and processes SAR/MTI radar imagery from the multi-function radar. Geometric correction of radar imagery is an inherent and standard preprocessing step necessary to produce a georeferenced SAR image suitable for spatial registration with optical imagery, and is well known in the SAR image processing art.
De et al. (’558) teaches: and obtains a load cabin adjustment angle according to the corrected radar image, and sends the load cabin adjustment angle to the platform drive processor
De et al. does not explicitly teach the data processor deriving a load cabin adjustment angle from the processed radar image and sending it to a platform drive processor. However, Yang et al. (’591) teaches this element ([0036], [0084]: “extracts the radar target position information of the target to be identified and the current position and attitude information of the unmanned ship, calculates the photoelectric eye-pointing angle, and controls the photoelectric pointing device to locate the eye-pointing angle”). Yang et al.’s integrated control system extracts radar-derived target position information, combines it with platform pose data, and calculates the required sensor pointing angle (load cabin adjustment angle), directly teaching this element.
Yang et al. (’591) teaches: the platform control and drive processor, receives the load cabin adjustment angle from the data processor, and drives, according to the load cabin adjustment angle, the cantilever to rotate the load cabin to aim at the target ([0036]: “controls the photoelectric pointing device to locate the eye-pointing angle”; [0070]: “When performing eye-finger control, in order to ensure that the photoelectric energy can accurately reach the target detected by the radar…calculate the angle that the photoelectric needs to rotate, and perform precise eye-finger control”). Yang et al.’s integrated control system receives the calculated pointing angle and drives the servo mechanism to rotate the optical sensor platform (load cabin) to the required angle, precisely aiming it at the radar-detected target.
De et al. (’558) teaches: the image fusion processor, receives the corrected optical image from the image processor, receives the corrected radar image from the data processor, performs image registration and fusion processing on the corrected optical image and the corrected radar image in sequence, obtain an original image, and send the original image to the image compression processor ([0028], [0076]: “The data fusion processing module is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image according to a control instruction sent by the universal data interface module”; [0095]: “It is especially important to synthesize the image information obtained by SAR and photoelectric sensors and to take advantage of their respective advantages”). De et al.’s data fusion processing module 22 directly teaches the image fusion processor performing image registration and fusion on the corrected optical and radar images.
De et al. (’558) teaches: the image compression processor, receives the original image from the image fusion processor, compresses the original image, generates a compressed image, and downloads the compressed image to an external ground station ([0018], [0063]: “the merged processed data is packaged and delivered to the onboard data terminal”; [0019]: “The onboard data terminal sends the packaged data sent by the integrated processor to the ground data terminal by using a wireless signal”; [0064]: “The onboard data terminal 16 sends the packaged data sent by the integrated processor to the ground data terminal by using a wireless signal”). De et al. packages and wirelessly transmits the fused imagery to a ground data terminal, directly teaching the image compression processor and external ground station download function.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to incorporate the radar-guided servo platform control mechanism of Yang et al. (’591) into the multi-task load system of De et al. (’558). De et al. teaches a multi-sensor airborne reconnaissance system that integrates a photoelectric pod (visible light, infrared, and laser ranging sensors) with a multi-function radar, and an integrated processor that receives target image information and target location information from the radar together with position and attitude measurement data. However, De et al. does not teach a mechanism by which processed radar data is used to derive a pointing angle that drives a servo to rotate the load cabin toward the detected target. Yang et al. teaches, in the analogous context of a mobile-platform multi-sensor reconnaissance system combining a search radar and a photoelectric sensor assembly, that the integrated control system extracts radar target position information, calculates the required optical sensor pointing angle, and drives the servo to rotate the photoelectric pointer to aim precisely at the radar-detected target. One would have been motivated to apply Yang et al.’s servo platform control concept to De et al.’s load system because De et al.’s integrated processor already receives the identical inputs that Yang et al. demonstrates are needed for the servo control calculation — radar-derived target location and platform position/attitude data — and because the core objective of De et al. (accurate multi-source fusion of radar and optical data) is directly served by ensuring the optical load cabin is automatically aimed at the same target being imaged by the radar. Incorporating Yang et al.’s radar-guided servo platform control would predictably enable the load cabin to be automatically rotated to aim at radar-detected targets, making the multi-source image fusion described in De et al. both more accurate and more efficient. There is a reasonable expectation of success because Yang et al. demonstrates that radar-guided optical sensor servo control operates successfully in an operationally analogous multi-sensor reconnaissance system on a mobile platform, and the underlying calculation and servo control principles apply equally to the airborne load cabin configuration of De et al.
Regarding Claim 2, De et al. (‘558) in view of Yang et al. (‘591) teaches the system according to Claim 1.
De et al. (‘558) teaches that the electronics cabin houses multiple distinct functional processor modules within its integrated processor architecture ([0025-0026]: “The integrated processor specifically includes: Load data interface module, data fusion processing module, task management module, general data interface module”; [0073-0074]: “The integrated processor 15 specifically includes: Load data interface module 21, data fusion processing module 22, task management module 23, general data interface module 24”). De et al. does not explicitly teach that the electronics cabin is further provided with a geographical tracking processor, a tracking processor and a moving target detection processor as distinct processor modules within that architecture. However, Yang et al. (‘591) teaches ([0014]: “Integrated control system, the integrated control system is used for servo control and sensor manipulation control of the photoelectric pointer, the servo control includes manual search, eye finger, tracking mode switching and control”; [0017]: “Intelligent identification system: The intelligent identification system is used for target identification. According to the model training of the identification system, the target to be paid attention to is preset, and the identification result is fed back to the photoelectric display and control terminal as a record, and the result is fed back to the ship control system”; [0070]: “the integrated control system is often used with the photoelectric pointer as the overall control and execution mechanism, and the main task is to do servo control and sensor manipulation control. Servo control includes a variety of servo mode switching, such as manual search, eye finger, tracking mode switching and control”; [0073]: “The intelligent identification system needs to communicate with the integrated control system, and also needs to access the image of the photoelectric system. The intelligent identification system is based on the hardware architecture of DSP+FPGA, and uses the neural network deep learning algorithm”). Yang et al. teaches a multi-processor reconnaissance architecture in which the integrated control system constitutes both the geographical tracking processor (computing pointing angles from radar-derived target position and platform pose) and the tracking processor (converting target positional deviation into a miss angle for closed-loop tracking), and the intelligent identification system constitutes the moving target detection processor (detecting and measuring target offset within the optical sensor field of view upon receipt of a turning-on indication), directly teaching three distinct processor modules within the electronics processing architecture of a combined radar-optical reconnaissance system corresponding to the three recited processors. It would have been obvious to a PHOSITA to implement these three functional units as distinct processor modules within the electronics cabin of De et al.’s multi-module integrated processor architecture, given that De et al. already implements its own functional units as distinct named modules ([0025-0028]) and Yang et al. demonstrates that such a three-module architecture is a workable and effective design for a combined radar-optical reconnaissance system.
De et al. (‘558) teaches that the multi-task load system operates in multiple distinct reconnaissance modes, including a wide area search mode, an active target indicator mode, and a synthetic aperture radar imaging mode ([0024], [0069]: “And performing a wide area search imaging task, an active target indicator imaging task, and a synthetic aperture radar imaging task according to the control instructions of the integrated processor”), establishing that the system is configurable to enter and execute operationally distinct reconnaissance modes.
De et al. does not explicitly teach when the reconnaissance system is in a collaborative search reconnaissance mode as a named distinct mode in which the radar and optical sensors cooperate through the described processor sequence to search for, acquire, and track a target area. Yang et al. (‘591) teaches: ([0019-0020]: “Target search: When the unmanned ship is working, it will cruise according to the pre-planned navigation route in the target sea area, and search for the target on the sea surface. Display control terminal and integrated control system”; [0021]: “Priority sorting: After preliminary screening of the targets sent by the radar, the optoelectronic display control terminal performs priority sorting, and sends the batch number of the target with the highest priority to the integrated control system”; [0022]: “Eye finger movement: After receiving the current new target batch number, the integrated control system calculates the photoelectric eye finger movement angle according to the ship attitude angle sensed by inertial navigation and the geographical location of the target, and controls the photoelectric pointer to perform eye finger movement”; [0024]: “Target tracking: After the integrated control system receives the identification information, it controls the photoelectric pointer, switches to the tracking mode according to the image position of the target, starts to track the target continuously”). Yang et al. teaches a complete and distinct collaborative search reconnaissance mode in which the radar and optical sensors operate cooperatively through a defined sequence — radar search, target selection, radar-cued optical pointing, and continuous optical tracking — directly teaching the system operating in a collaborative search reconnaissance mode.
De et al. (‘558) teaches: the radar, performs echo data sampling and sends echo data to the data processor ([0060]: “The multi-function radar 13 is configured to complete an imaging task according to a control instruction of the integrated processor, and send target image information and target location information to the integrated processor”; [0069]: “And performing a wide area search imaging task, an active target indicator imaging task, and a synthetic aperture radar imaging task according to the control instructions of the integrated processor”). De et al.’s multi-function radar in its active target indicator (MTI/GMTI) mode performs echo data sampling and transmits the resulting target information to the integrated processor, directly teaching this element.
De et al. (‘558) teaches: the data processor, receives the echo data from the radar, performs parsing processing on the echo data, obtains plots and tracks data of a target ([0076]: “integrating the dot/track data generated by the multi-function radar 13 with the track/track data generated by the photoelectric pod 12”). De et al.’s integrated processor receives and parses the radar MTI echo data to generate dot/track data (plots and tracks) of detected moving targets.
De et al. (‘558) teaches that the data processor routes its processed output to the data fusion processing module within the integrated processor ([0029]: “The task management module is configured to perform packet processing on the load status information sent by the load data interface module, send the data to the universal data interface module”; [0028]: “The data fusion processing module is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image according to a control instruction sent by the universal data interface module”; [0076]: “Or integrating the dot/track data generated by the multi-function radar 13 with the track/track data generated by the photoelectric pod 12”). De et al. does not explicitly teach and sends the plots and tracks data of the target to the geographical tracking processor, as De et al. routes the radar-derived plots and tracks data to the data fusion processing module rather than to a distinct geographical tracking processor. Yang et al. (‘591) teaches that the plots and tracks data is received by and acted upon by the geographical tracking processor ([0022]: “the integrated control system calculates the photoelectric eye finger movement angle according to the ship attitude angle sensed by inertial navigation and the geographical location of the target, and controls the photoelectric pointer to perform eye finger movement”; [0036]: “extracts the radar target position information of the target to be identified and the current position and attitude information of the unmanned ship, calculates the photoelectric eye-pointing angle, and controls the photoelectric pointing device to locate the eye-pointing angle”). Yang et al.’s integrated control system receives the radar-derived target position data (plots and tracks) as its direct input for computing the pointing angle, directly teaching the plots and tracks data being sent to and received by the geographical tracking processor.
De et al. (‘558) does not explicitly teach the geographic tracking processor, receives the plots and tracks data of the target from the data processor, obtains a load cabin adjustment angle by parsing based on the plots and tracks data of the target and its own pose information, and sends the load cabin adjustment angle to the platform control and drive processor. Yang et al. (‘591) teaches: ([0022]: “the integrated control system calculates the photoelectric eye finger movement angle according to the ship attitude angle sensed by inertial navigation and the geographical location of the target, and controls the photoelectric pointer to perform eye finger movement”; [0033]: “The inertial navigation system transmits the current speed, attitude and position information of the unmanned ship in the navigation coordinate system to the integrated control system”; [0036]: “extracts the radar target position information of the target to be identified and the current position and attitude information of the unmanned ship, calculates the photoelectric eye-pointing angle, and controls the photoelectric pointing device to locate the eye-pointing angle”). Yang et al.’s integrated control system receives radar-derived target plots and tracks data, combines it with the platform’s own pose information supplied by inertial navigation, computes the required sensor pointing angle (load cabin adjustment angle), and transmits that angle to drive the platform servo, directly and completely teaching the geographical tracking processor and its described data flow.
De et al. (‘558) does not explicitly teach the platform control and drive processor, receives the load cabin adjustment angle from the geographical tracking processor, drives, according to the load cabin adjustment angle, the cantilever to rotate the load cabin to aim at the target, and receives a target miss angle from the tracking processor, and drives, according to the target miss angle, the cantilever to rotate the load cabin, to aim at and track a target area, thus achieving collaborative search. Yang et al. (‘591) teaches: ([0036]: “controls the photoelectric pointing device to locate the eye-pointing angle”; [0037]: “send the in-position information to the intelligent identification system, and switch to a self-stabilizing state, while waiting for the feedback from the intelligent identification system”; [0089]: “the integrated control system controls the photoelectric pointer to switch to the tracking state and continues to track the target. The system sends the recognition result, and the photoelectric pointer adjusts the focal length of the photoelectric pointer according to the proportion of the target in the screen”; [0070]: “Servo control includes a variety of servo mode switching, such as manual search, eye finger, tracking mode switching and control”). Yang et al.’s integrated control system first receives the computed load cabin adjustment angle and drives the servo to rotate the optical sensor to aim at the target, then upon confirmation of the in-position state transitions to a closed-loop tracking mode in which it continuously adjusts the sensor direction using the target miss angle received from the tracking processor, directly teaching the platform control and drive processor operating in both the initial aiming and continuous tracking sub-modes described in Claim 2.
De et al. (‘558) does not explicitly teach the moving target detection processor, after receiving a moving target detection turning-on indication, measures a target offset and send the target offset to the tracking processor. Yang et al. (‘591) teaches: ([0037]: “send the in-position information to the intelligent identification system, and switch to a self-stabilizing state, while waiting for the feedback from the intelligent identification system”; [0039-0040]: “the intelligent recognition system receives the image captured by the photoelectric pointer, and judges whether there is a target to be recognized in the image…The deviation information is sent to the integrated control system, otherwise no message will be sent to the integrated control system”; [0023]: “If there is a target in the image, the upper left corner and lower right corner of the screen where the target is located or the relative position of the center of the field of view is fed back. Deviation information, inform the integrated control system”). Yang et al.’s intelligent identification system is activated upon receipt of an in-position signal from the integrated control system (equivalent to the moving target detection turning-on indication), measures the positional deviation of the detected target within the optical sensor field of view (target offset), and transmits that deviation to the integrated control system, directly teaching the moving target detection processor and its described operation.
De et al. (‘558) does not explicitly teach the tracking processor, receives the target offset from the moving target detection processor, parsing the target offset to obtain the target miss angle, and sends the target miss angle to the platform control and drive processor. Yang et al. (‘591) teaches: ([0041-0043]: “The comprehensive control system judges whether it has received the position deviation information of the target to be identified sent by the intelligent identification system…the integrated control system controls the photoelectric pointer to switch to the tracking state and continues to track the target”; [0044]: “Determine whether the duration of the tracking state exceeds the tracking preset time, if so, continue to track the target”). Yang et al.’s integrated control system receives the target positional deviation (target offset) from the intelligent identification system, processes it into the required servo correction signal (target miss angle), and drives the servo for continuous closed-loop tracking, directly teaching the tracking processor.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to incorporate the radar-guided collaborative tracking architecture of Yang et al. (‘591) — comprising a geographical tracking processor, a tracking processor, and a moving target detection processor operating in the described sequence — into the multi-task load system of De et al. (‘558). De et al. teaches a multi-function radar already operating in an active MTI/GMTI mode generating target plots and tracks data and a photoelectric pod already performing monitoring and tracking tasks, but provides no mechanism by which the radar-derived plots and tracks are used to drive an initial target-aiming sequence followed by closed-loop optical tracking. Yang et al. addresses precisely this coordination problem in an analogous mobile-platform multi-sensor reconnaissance context by implementing a three-processor architecture in which plots and tracks data together with platform pose information drive an initial pointing command, detected target deviation subsequently drives a continuous tracking loop, and the platform control and drive processor transitions between these two sub-modes in response to inputs from the two downstream processors. One would have been motivated to apply this architecture to De et al.’s system because the data streams required by each stage — radar-derived target plots and tracks data and platform pose data — are already produced and available within De et al.’s system, presenting no integration barrier; because both references address the same underlying technical problem of coordinating a radar sensor and an optical sensor on a mobile reconnaissance platform for automated target acquisition and tracking; and because Yang et al. demonstrates that this architecture enables fully automated collaborative search without manual operator intervention, which is a recognized objective for airborne multi-sensor reconnaissance systems of the type described by De et al. There is a reasonable expectation of success because Yang et al. demonstrates the complete three-processor architecture operating successfully in an analogous multi-sensor mobile-platform reconnaissance system; because the mathematical principles underlying the pointing-angle computation from radar-derived geographic target position and platform pose, and the deviation-based tracking feedback, are not specific to any platform type and apply equally to the airborne load cabin configuration of De et al.; and because De et al. already generates and makes available all the data inputs that each of the three processors in Yang et al.’s architecture requires, confirming that no additional sensing or data infrastructure would be needed to implement the combination.
Claim 3 depends from Claim 2 and describes the full element-by-element data flow when the reconnaissance system is in a moving target detection and heterosource video fusion reconnaissance mode, in which the image processor generates an optical video with positioning information by using laser rangefinder distance and pose data, and the image fusion processor fuses that positioned optical video with radar-derived plots and tracks data to produce a video superimposed with moving target information.
Regarding Claim 3, De et al. (‘558) in view of Yang et al. (‘591) teaches the system according to Claim 2.
De et al. (‘558) teaches: the visible light camera, performs image sampling, obtains a visible light video signal, and sends the visible light video signal to the image processor ([0022, [0067]: “visible light imaging…performed on ground targets in accordance with control commands sent by the integrated processor”). De et al.’s photoelectric pod performs continuous visible light imaging of ground targets as a monitoring task and transmits the resulting imagery to the integrated processor.
De et al. (‘558) teaches: the infrared thermal imager, performs image sampling, obtains an infrared video signal, and sends the infrared video signal to the image processor ([0022], [0067]: “Infrared imaging…performed on ground targets in accordance with control commands sent by the integrated processor”). De et al.’s photoelectric pod performs continuous infrared imaging and transmits the resulting imagery to the integrated processor.
De et al. (‘558) teaches: the laser rangefinder, measures a distance value between the load cabin and a target, and sends the distance value to the image processor ([0022], [0067]: “laser ranging…performed on ground targets in accordance with control commands sent by the integrated processor”; [0017]: “The position and attitude measuring system is coupled to the integrated processor for providing position and attitude measurement data for the aerial camera, the photoelectric pod, and the multi-function radar”). De et al.’s photoelectric pod performs laser ranging and transmits the distance value to the integrated processor alongside the position and attitude measurement data.
De et al. (‘558) teaches: the image processor, receives the distance value from the laser rangefinder, receives the visible light video signal from the visible light camera, receives the infrared video signal from the infrared thermal imager ([0059]: “The photoelectric pod 12 is configured to perform a monitoring task on the ground target according to a control command sent by the integrated processor, and send the monitoring information to the integrated processor. The monitoring task may include tasks such as infrared imaging, visible light imaging, laser ranging, and tracking”; [0067]: “Infrared imaging, visible light imaging, laser ranging, and tracking tasks are performed on ground targets in accordance with control commands sent by the integrated processor”; [0063]: “the integrated processor 15 receives the digital photo information, the monitoring information, the target image information, and the target location information”; [0075]: “The load data interface module 21 is connected to the aerial camera 11, the photoelectric pod 12, the multi-function radar 13, and the mission planning map display 14, respectively, to the aerial camera 11, the photoelectric cabin 12, the multi-function radar 13, and the mission planning map display 14 perform data interaction”). De et al.’s photoelectric pod performs visible light imaging, infrared imaging, and laser ranging simultaneously as components of a single monitoring task and transmits all resulting monitoring information — including the visible light imagery, infrared imagery, and laser rangefinder distance value — to the integrated processor via the load data interface module, directly and completely teaching the image processor receiving all three inputs.
De et al. (‘558) does not explicitly teach positioning the visible light video signal and infrared video signal according to the distance value and its own pose information to obtain an optical video with positioning information, and send the optical video with positioning information to the image fusion processor — specifically, the step of using the laser rangefinder distance value together with the platform’s own pose information to compute geographic positioning data for the optical video prior to sending it to the image fusion processor. Yang et al. (‘591) teaches this element ([0070]: “When performing eye-finger control, in order to ensure that the photoelectric energy can accurately reach the target detected by the radar, the attitude and heading information of the inertial navigation are required. At the same time, the photoelectric azimuth and pitch angle are used for coordinate system conversion, from the deck coordinate system to the earth coordinate system, calculate the optical axis angle in the geodetic coordinate system, and then reversely convert it to the deck coordinate system”; [0033]: “The inertial navigation system transmits the current speed, attitude and position information of the unmanned ship in the navigation coordinate system to the integrated control system”; [0036]: “extracts the radar target position information of the target to be identified and the current position and attitude information of the unmanned ship, calculates the photoelectric eye-pointing angle”). Yang et al.’s integrated control system uses platform pose information from inertial navigation together with the optical sensor’s azimuth and pitch angles to perform coordinate system conversion and establish the geographic position of the optical sensor’s line of sight, directly teaching the use of the platform’s own pose information to compute geographic positioning data for the optical video. Combined with the laser rangefinder distance value already available in De et al.’s system, this coordinate conversion approach generates an optical video output with geographic positioning information, directly teaching the claimed element.
De et al. (‘558) teaches: the radar, performs echo data sampling and sending echo data to the data processor ([0060]: “The multi-function radar 13 is configured to complete an imaging task according to a control instruction of the integrated processor, and send target image information and target location information to the integrated processor”; [0069]: “And performing a wide area search imaging task, an active target indicator imaging task, and a synthetic aperture radar imaging task according to the control instructions of the integrated processor”).
De et al. (‘558) teaches: the data processor, receives the echo data from the radar, performs echo data processing on the echo data, generates plots and tracks data of the target, and sends the plots and tracks data of the target to the image fusion processor ([0076]: “or integrating the dot/track data generated by the multi-function radar 13 with the track/track data generated by the photoelectric pod 12”). De et al.’s data fusion processing module explicitly teaches that the data processor generates dot/track data (plots and tracks) from the multi-function radar and passes this data to the fusion module (image fusion processor) for integration with the optical data.
De et al. (‘558) teaches: the image fusion processor, receives the optical video with positioning information from the image processor, receives the plots and tracks data from the data processor, performs registration and fusion processing on the optical video with positioning information with the plots and tracks data, to obtain a video superimposed with moving target information, completing moving target detection and heterosource video fusion reconnaissance ([0028]: “The data fusion processing module is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image according to a control instruction sent by the universal data interface module”; [0028], [0076]: “Or integrating the dot/track data generated by the multi-function radar 13 with the track/track data generated by the photoelectric pod 12”; [0095]: “It is especially important to synthesize the image information obtained by SAR and photoelectric sensors and to take advantage of their respective advantages. On the one hand, the advantages of the signals received by each sensor can be utilized at the same time, and on the other hand, the defects of each sensor can be compensated”). De et al.’s data fusion processing module explicitly teaches performing registration and fusion of radar-derived dot/track data with photoelectric optical data to produce a composite output that combines moving target information from the radar with imagery from the optical sensors, directly teaching the image fusion processor receiving the two inputs and producing a video superimposed with moving target information.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to apply Yang et al.’s (‘591) pose-based coordinate system conversion technique to the image processor of De et al.’s (‘558) system so as to generate an optical video with geographic positioning information for use in the moving target detection and heterosource video fusion mode. De et al. teaches a system in which the image fusion processor must register and fuse optical video with radar-derived plots and tracks data, and already supplies both the position and attitude measurement data ([0017]) and the laser rangefinder distance value ([0067]) that Yang et al. demonstrates are the required inputs for computing the geographic position of the optical line of sight. However, De et al. does not describe using those inputs in the image processor to embed geographic positioning information into the optical video prior to fusion. Yang et al. demonstrates, in an analogous multi-sensor reconnaissance context, that performing coordinate system conversion using platform pose and optical sensor orientation data is both necessary and sufficient to establish the geographic position of the optical sensor’s line of sight, enabling the optical imagery to be meaningfully registered with geographically referenced radar data ([0036], [0070]). One would have been motivated to apply this technique within De et al.’s image processor because the fusion of optical video with radar plots and tracks data — which De et al. explicitly describes as a design objective ([0028], [0076]) — requires that the optical video and the radar data share a common geographic reference frame; without geo-positioning the optical video, accurate spatial registration with the radar plots and tracks data would not be achievable, and the moving target detection and heterosource video fusion mode described in Claim 3 could not function as designed. Applying Yang et al.’s coordinate conversion approach to De et al.’s image processor, using the laser distance and pose data already available in the system, would predictably produce an optical video with geographic positioning information suitable for registration with the radar plots and tracks data in the image fusion processor. There is a reasonable expectation of success because Yang et al. demonstrates that pose-based coordinate system conversion successfully produces geographically positioned optical data in an operationally analogous mobile-platform multi-sensor reconnaissance system; because the mathematical principles of the coordinate conversion are not specific to any platform type and apply equally to the airborne configuration of De et al.; and because all the input data required by the conversion — platform pose from the position and attitude measurement system and laser range from the laser rangefinder — are already generated and available within De et al.’s system, confirming that no additional hardware or sensing capability would be required to implement the combination.
Regarding Claim 4, De et al. (‘558) in view of Yang et al. (‘591) teaches the system according to Claim 1.
De et al. (‘558) does not explicitly teach, but Yang et al. (‘591) teaches: wherein the multi-source imaging reconnaissance mode refers to that a servo action of the cantilever is controlled by the radar ([0036]: “extracts the radar target position information of the target to be identified and the current position and attitude information of the unmanned ship, calculates the photoelectric eye-pointing angle, and controls the photoelectric pointing device to locate the eye-pointing angle”; [0070]: “When performing eye-finger control, in order to ensure that the photoelectric energy can accurately reach the target detected by the radar, the attitude and heading information of the inertial navigation are required. At the same time, the photoelectric azimuth and pitch angle are used for coordinate system conversion, from the deck coordinate system to the earth coordinate system, calculate the optical axis angle in the geodetic coordinate system, and then reversely convert it to the deck coordinate system, calculate the angle that the photoelectric needs to rotate, and perform precise eye-finger control”). Yang et al.’s integrated control system derives a servo pointing command directly from radar-detected target position information and drives the optical sensor platform to aim at the radar-designated target area, directly teaching the servo action of the cantilever being controlled by the radar.
De et al. (‘558) teaches: the radar is in a strip mode ([0024], [0069]: “And performing a wide area search imaging task, an active target indicator imaging task, and a synthetic aperture radar imaging task according to the control instructions of the integrated processor”). De et al. (‘558) teaches a multifunction radar configured to perform a wide area search imaging task and synthetic aperture radar imaging task, which correspond to the claimed radar operating in a strip/search imaging mode for obtaining an image of a target area for fusion under the broadest reasonable interpretation.
De et al. (‘558) does not explicitly teach, but Yang et al. (‘591) teaches: adjusts an optical field of view ([0043]: “the photoelectric pointer adjusts the focal length of the photoelectric pointer according to the proportion of the target in the screen, more detailed”; [0014], [0070]: “the sensor control includes zoom control, fog penetration, black and white switching”; [0070]: “when the photoelectric is tracking the target, it performs zoom control according to the proportion of the target in the screen, so that the target is in a well-recognized outline in the image, which is convenient for the recognition system to perform intelligent recognition”). Yang et al.’s integrated control system adjusts the optical sensor’s focal length (field of view) based on the target’s proportion within the image frame, directly teaching adjustment of the optical field of view.
De et al. (‘558) teaches: obtains an image of a target area for fusion ([0028]: “The data fusion processing module is configured to perform data fusion processing on two or more images in a digital photo, a visible light image, an infrared image, and a synthetic aperture radar image according to a control instruction sent by the universal data interface module”; [0076]: same teaching in embodiment description; [0095]: “It is especially important to synthesize the image information obtained by SAR and photoelectric sensors and to take advantage of their respective advantages”). De et al. explicitly teaches that the multi-function radar obtains a SAR image of the target area and that this image is passed to the data fusion processing module for fusion with optical imagery, directly teaching this element.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to incorporate Yang et al.’s (‘591) radar-driven servo control and optical field-of-view adjustment into the multi-source imaging reconnaissance mode of De et al. (‘558). De et al. teaches that the core purpose of the multi-source imaging mode is to fuse SAR strip imagery with optical imagery of the same target area ([0028], [0095]), and that the photoelectric pod’s visible light imaging, infrared imaging, and laser ranging are all performed on the same ground targets as the radar ([0067]). However, De et al. does not describe any mechanism to ensure that the optical sensors are pointed at and properly framed on the target area being imaged by the SAR radar in strip mode. Yang et al. addresses precisely this co-coverage problem by demonstrating that using radar-derived target position data to drive the optical sensor servo and adjusting the optical field of view based on the target’s apparent size in the image ensures that the optical sensors cover the same target area as the radar ([0036], [0070], [0024]). One would have been motivated to apply this approach to De et al.’s multi-source imaging mode because the image fusion described in De et al. ([0028], [0076]) is only meaningful when the optical and radar sensors are imaging overlapping portions of the target area; without the optical sensors being pointed at the strip-mode radar target area and having an appropriately adjusted field of view, the corrected optical image and corrected radar image would not spatially correspond, making image registration and fusion unreliable. Applying Yang et al.’s radar-driven servo control and field-of-view adjustment would predictably ensure that the optical sensors are continuously aimed at and properly framed on the same target area being swept by the SAR in strip mode, directly enabling the image registration and fusion that De et al. identifies as the objective of the multi-source imaging reconnaissance mode. There is a reasonable expectation of success because Yang et al. demonstrates that radar-driven servo control and optical field-of-view adjustment operate successfully in an analogous mobile-platform multi-sensor reconnaissance system combining radar and optical sensors; because the underlying control principles apply equally to the airborne cantilever and load cabin configuration of De et al.; and because De et al. already provides the radar target location data and position and attitude measurement data ([0015], [0017]) that Yang et al. demonstrates are sufficient inputs to implement both the servo pointing control and the field-of-view adjustment.
Regarding Claim 5, De et al. (‘558) in view of Yang et al. (‘591) teaches the system according to Claims 2.
De et al. (‘558) teaches: a load is first in a GMTI (ground moving target indication) operation mode ([0060]: “The multi-function radar 13 is configured to complete an imaging task according to a control instruction of the integrated processor, and send target image information and target location information to the integrated processor”; [0069]: “And performing a wide area search imaging task, an active target indicator imaging task, and a synthetic aperture radar imaging task according to the control instructions of the integrated processor”; [0076]: “integrating the dot/track data generated by the multi-function radar 13 with the track/track data generated by the photoelectric pod 12”). De et al.’s multi-function radar explicitly includes an active target indicator (MTI/GMTI) imaging mode that detects moving ground targets and generates the dot/track (plots and tracks) data characteristic of GMTI operation, directly teaching the load operating first in a GMTI mode.
De et al. (‘558) does not explicitly teach, but Yang et al. (‘591) teaches: selects the target ([0021]: “Priority sorting: After preliminary screening of the targets sent by the radar, the optoelectronic display control terminal performs priority sorting, and sends the batch number of the target with the highest priority to the integrated control system”; [0026-0030]: “After receiving the radar data, the photoelectric display control terminal judges whether the target distance is less than the preset distance according to the radar data…If the target is not in the identified target linked list, judge whether the target id is in the to-be-identified target linked list, if so, update the priority of the to-be-identified target linked list…Send the updated header data of the target linked list to be identified to the integrated control system”). Yang et al. teaches a priority-based target selection process in which radar-detected targets are screened and ranked, and the highest-priority target is selected and passed to the control system for subsequent pointing and tracking, directly teaching the target selection step;
De et al. (‘558) does not explicitly teach, but Yang et al. (‘591) teaches: guides photoelectric to point to the target ([0022]: “Eye finger movement: After receiving the current new target batch number, the integrated control system calculates the photoelectric eye finger movement angle according to the ship attitude angle sensed by inertial navigation and the geographical location of the target, and controls the photoelectric pointer to perform eye finger movement”; [0036]: “extracts the radar target position information of the target to be identified and the current position and attitude information of the unmanned ship, calculates the photoelectric eye-pointing angle, and controls the photoelectric pointing device to locate the eye-pointing angle”; [0037]: “Judging whether the photoelectric pointer is in place, if so, continue to control the photoelectric pointer to locate the angle of the eye, if so, send the information of the in-position to the intelligent identification system, and switch to a self-stabilizing state”). Yang et al. teaches that following target selection, the integrated control system computes the required optical sensor pointing angle from the selected target’s geographic position and the platform’s attitude information, and drives the servo to rotate the optical sensor until it is aimed at the selected target, directly teaching guiding the photoelectric sensor to point to the target;
De et al. (‘558) does not explicitly teach, but Yang et al. (‘591) teaches: detects and tracks the target ([0023]: “The photoelectric pointer transmits the captured image to the intelligent recognition system, and the intelligent recognition system starts to recognize the target. If there is a target in the image, the upper left corner and lower right corner of the screen where the target is located or the relative position of the center of the field of view is fed back”; [0024]: “Target tracking: After the integrated control system receives the identification information, it controls the photoelectric pointer, switches to the tracking mode according to the image position of the target, starts to track the target continuously, and adjusts the focal length of the photoelectric pointer according to the proportion of the target in the screen”; [0044]: “Determine whether the duration of the tracking state exceeds the tracking preset time, if so, continue to track the target”). Yang et al. teaches that once the photoelectric sensor is pointed at the selected target, the intelligent identification system detects the target within the optical image and reports its position, after which the integrated control system switches to tracking mode and continuously tracks the target, directly teaching the detection and tracking step.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to incorporate Yang et al.’s (‘591) target selection, optical pointing, and optical tracking sequence into the cooperative search reconnaissance mode of De et al. (‘558). De et al. teaches a multi-function radar operating in GMTI mode that generates plots and tracks data of multiple moving ground targets and a photoelectric pod capable of monitoring, tracking, and imaging those targets, but De et al. does not describe any mechanism by which a specific target is selected from the GMTI output, the photoelectric sensor is directed toward that target, or the target is subsequently kept in the optical sensor’s field of view. Yang et al. addresses precisely this gap in an analogous mobile-platform multi-sensor reconnaissance context by demonstrating that priority-based target selection from radar data, followed by radar-cued optical pointing and closed-loop optical tracking, constitutes a complete and operationally effective cooperative search workflow. One would have been motivated to apply this sequence to De et al.’s system because the cooperative value of having both a GMTI radar and an optical tracking sensor on the same platform — which De et al. explicitly recognizes by combining these sensors and fusing their outputs — is only fully realized when the optical sensor can be directed to, and made to track, specific targets that the GMTI radar has detected; without such a selection-pointing-tracking sequence, the optical sensor cannot contribute meaningful co-registered imagery of radar-detected targets, defeating the purpose of the combined sensor suite. There is a reasonable expectation of success because Yang et al. demonstrates the complete selection-pointing-tracking sequence operating successfully in an operationally analogous multi-sensor reconnaissance system on a mobile platform; because the priority-sorting, pointing-angle computation, and deviation-based tracking principles taught by Yang et al. are mathematically general and not specific to any platform type, applying equally to the airborne configuration of De et al.; and because De et al. already generates all the data inputs required by Yang et al.’s sequence — GMTI target position data and platform position and attitude data — confirming that no additional hardware or sensing infrastructure would be needed to implement the combination.
Regarding Claim 6, De et al. (’558) in view of Yang et al. (’591) teaches the system according to Claims 3.
De et al. (’558) teaches: wherein moving target detection and heterosource video fusion reconnaissance mode is to label plots and tracks of a radar GMTI on the optical video to improve readability of the moving target ([0076]: “Or integrating the dot/track data generated by the multi-function radar 13 with the track/track data generated by the photoelectric pod 12”; [095]: “the measurement angle information is not accurate enough and the active sensor is easily found by the target. Photoelectric sensors of photoelectric pods have the advantage of high accuracy of measuring angle information…It is especially important to synthesize the image information obtained by SAR and photoelectric sensors”). De et al. explicitly teaches integrating (overlaying) the dot/track (plots and tracks) data generated by the multi-function radar’s MTI/GMTI mode with the optical tracking data from the photoelectric pod to produce a composite output with moving target information, directly teaching labeling radar GMTI plots and tracks on the optical video to improve target readability.
Regarding Claim 9, De et al. (’558) in view of Yang et al. (’591) teaches the system according to Claim 1.
De et al. (’558) teaches digital photo capture and wireless transmission of packaged image data to a ground station ([0013]: “The aerial camera is configured to send digital photo information to the integrated processor after taking a picture of the ground target”; [0018]: “the merged processed data is packaged and delivered to the onboard data terminal”; [0019]: “The onboard data terminal sends the packaged data sent by the integrated processor to the ground data terminal by using a wireless signal”). De et al. does not explicitly specify JPEG as the compression format. However, the compressed image is in a format of JPEG — JPEG was, at the time of the invention, the universally recognized and dominant standard format for lossy compression of digital photography and reconnaissance imagery. Selecting JPEG as the image compression format for the aerial camera’s digital photo output and the fused imagery transmitted to the ground station would have been an obvious design choice well within the ordinary skill of the art. PHOSITA would have selected JPEG compression as the standard interoperable format for digital reconnaissance imagery, providing a well-understood balance of compression efficiency and image quality for wireless transmission and ground station display. No invention is involved in selecting the JPEG format in this context.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over De et al. (CN104808558A) in view of Yang et al. (CN113960591A) and further in view of Guch (US 6,211,951 B1).
Regarding Claim 7, De et al. (’558) in view of Yang et al. (’591) teaches the system according to Claim 1.
De et al. (’558) teaches the visible light camera, infrared thermal imager, and laser rangefinder as components of the load cabin ([0067]: “Infrared imaging, visible light imaging, laser ranging, and tracking tasks are performed on ground targets in accordance with control commands sent by the integrated processor”). Neither De et al. nor Yang et al. teaches calibrating and maintaining the optical axis parallelism of these three sensors within a specific numerical tolerance of 0.2 mrad.
De et al. (’558) does not explicitly teach, but Guch (US 6,211,951 B1) teaches: wherein an optical axis parallelism of the visible light camera, the infrared thermal imager, and the laser rangefinder is calibrated and maintained within 0.2 mrad (Abstract: “A boresight alignment method is used for aligning a laser with an optical sighting system in a laser designator or the like…An optical sighting system can be aligned to a laser with high accuracy under rugged military and industrial field operating conditions”; Col. 1, lines 13-22: “Modern military vehicles employ electro-optical fire control systems using multiple sensors to detect and track desired targets. The multiple sensors may include a visible sensor, a forward-looking infrared sensor, and an intensified night vision sensor, all of which may be disposed within the same instrument as the laser. In order to meet laser targeting performance goals, the boresight accuracy along the various sensors is required to be of high accuracy, e.g. 100 microradians.”; Col. 3, lines 18-21: “An optical sighting system using this technique can be aligned to a laser with high accuracy (e.g. less than 100 microradions) under rugged military and industrial field operating conditions”). Guch teaches a boresight alignment method for co-aligning a visible light sensor, an infrared sensor, and a laser, with the system requiring accuracy of 188 microradians and achieving accuracy of less than 100 microradians (0.1 mrad) — well within the claimed 0.2 mrad (200 microradians) tolerance. The sensors aligned by Guch — visible sensor, forward-looking infrared sensor, and laser — correspond directly to the visible light camera, infrared thermal imager, and laser rangefinder of Claim 7.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to apply the boresight alignment method of Guch (’951) to calibrate and maintain the optical axis parallelism of the visible light camera, infrared thermal imager, and laser rangefinder of the combined De et al./Yang et al. system within 0.2 mrad. One would have been motivated to do so because Guch specifically identifies that in multi-sensor systems integrating a laser, visible sensor, and infrared sensor — precisely the sensor combination of De et al.’s photoelectric pod — achieving co-alignment of the sensor optical axes is essential for accurate multi-sensor targeting and data correlation (Background section, Col. 1). The geometric correction and multi-source image fusion described in Claim 1 depend on the optical sensor axes being co-aligned; without such calibration, the visible light and infrared images would not be properly registered to each other or to the laser rangefinder distance measurements, degrading the quality of the corrected optical image and the subsequent fusion output. There is a reasonable expectation of success because Guch demonstrates that its boresight alignment method achieves co-alignment accuracy of less than 100 microradians for the identical sensor combination (visible sensor, infrared sensor, laser) under rugged field operating conditions directly comparable to those of the airborne reconnaissance system of De et al., confirming operability well within the claimed 0.2 mrad tolerance.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over De et al. (CN104808558A) in view of Yang et al. (CN113960591A) and further in view of Guch (US 6,211,951 B1) and Linting et al. (CN108956099A).
Regarding Claim 8, De et al. (‘558) in view of Yang et al. (‘591) and further in view of Guch (‘951) and Linting et al. (CN108956099A) teaches the system according to Claim 7.
De et al. (’558) teaches the combination of radar and optical sensors for multi-mode reconnaissance imaging and fusion ([0057-0070]). Yang et al. (’591) teaches that the optical sensor assembly is servo-controlled to aim at radar-detected targets ([0036], [0070]), which functionally establishes co-alignment between the radar beam direction and the optical sensor visual axis during operation. However, neither De et al. nor Yang et al. explicitly teaches maintaining a specific parallelism between the beam direction of the radar and the visual axis of the optical sensor within a defined numerical tolerance (0.5°) as a pre-designed structural requirement of the system.
Linting et al. (CN108956099A) teaches: a parallelism between a beam direction of the radar and a visual axis of the optical sensor is less than 0.5°, enabling acquisition of information in the same target area and performing the multi-source imaging reconnaissance, the collaborative search reconnaissance and the moving target detection and heterosource video fusion reconnaissance mode ([003]: “The method relates to the field of precision measurement of optical axis spatial pointing of multi-band optical systems. This method is suitable for accurate measurement of optical axis consistency of large-pitch, multi-band optical systems”; [005]: “According to the design requirements of the photoelectric detection system, the optical axes of the multiple optical systems should be parallel, and the deviation of the optical axis of each optical system from the reference optical axis is generally not more than 0.1 mard”; [005]: “The consistency deviation between the optical axes affects the detection accuracy of the photoelectric detection system to the target position”; [008]: “In order to meet the requirement of using the unit to quantitatively detect the multi-optical axis consistency deviation of the photodetection system, a method of measuring the uniformity deviation of the large-pitch optical axis by a double theodolite is studied by using the characteristics of the universal self-collimation theodolite”). Linting et al. teaches that in multi-band photoelectric detection systems integrating multiple sensors on a common tracking platform, all sensor optical axes must be maintained in parallel alignment with a reference axis within a defined tolerance, and that consistency deviation between sensor beam directions and visual axes directly affects detection accuracy. The general principle of maintaining co-alignment between a radar beam direction and an optical sensor visual axis within a specified tolerance to ensure that all sensors acquire information in the same target area — as claimed — follows directly from the axis co-alignment teachings of Linting et al.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to apply the axis co-alignment principles of Linting et al. (’099) to the combined De et al./Yang et al./Guch system to establish and maintain a parallelism between the radar beam direction and the optical sensor visual axis within 0.5°. One would have been motivated to do so because Linting et al. teaches that axis consistency deviation between sensors on a multi-sensor platform directly affects detection accuracy ([005]), and in the combined system of De et al. and Yang et al., the radar and optical sensors must image the same target area for multi-source fusion to function correctly. If the radar beam direction and the optical sensor visual axis are not co-aligned within an adequate tolerance, the radar and optical sensors will not image overlapping portions of the target area, making the multi-source imaging, collaborative search, and moving target detection fusion modes of Claims 1–3 unreliable. Maintaining co-alignment within 0.5° ensures sufficient spatial overlap for all three reconnaissance modes to operate as designed. There is a reasonable expectation of success because Linting et al. demonstrates that optical axis co-alignment deviation can be quantitatively measured and controlled to tolerances far tighter than 0.5° (0.1 mrad ≈ 0.006°) using established double-theodolite measurement methods, confirming that achieving the claimed 0.5° radar-optical axis parallelism is well within the capabilities of PHOSITA applying routine calibration techniques.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REMASH R GUYAH whose telephone number is (571)270-0115. The examiner can normally be reached M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Resha H Desai can be reached at (571) 270-7792. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REMASH R GUYAH/Examiner, Art Unit 3648
Prosecution Insights