Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. KR10-2023-0111608, filed on 08/24/2023.
Acknowledgment is made of applicant' s submission for Domestic Benefit/National State Information under 35 U.S.C. 371 for PCT/KR2024/012580 with filing date 08/23/2024.
Claim Objections
Claim 1 is objected to because the transition between limitations iii) and the wherein clause reads: “iii) calculate information of the distinguished artificial objects. wherein the processing module includes …” — the period after “objects” is a punctuation error and should be a semicolon (or a comma with restructured clause) so that the wherein clause reads as a limitation of the same claim rather than a sentence fragment.
Claim 12 is objected to for the recitation “filters among the extract shadows“, which is grammatically incorrect and lacks proper antecedent basis. The antecedent is “shadows” as introduced in Claim 1(i) (“extract shadows defined as shaded areas …”). Applicant should amend to “filters among the extracted shadows“. The same claim recites “a shadow extraction unit that filters among the extract shadows so as to detect and extract a shadow formed by a natural object” — the phrase mixes the verb “extract” with the participial “extracted” inconsistently and should be corrected for clarity.
Claim 16 is objected to for the recitation “selecting moving an object with velocity“, which is a grammatical inversion. Correct usage is “selecting a moving object with velocity“. Appropriate correction is required.
Claim 16 is further objected to for the recitation “matching the extract shadow located within the set boundary“. The word “extract” should read “extracted” for grammatical correctness and to maintain antecedent basis with the earlier recitation “extracting shadows.”
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“boundary setting unit configured to set a boundary based on a maximum velocity of the moving object …” (Claim 2 and by reference Claims 5, 10, 11). Prong 1: “unit” is a generic placeholder. Prong 2: modified by functional language (“configured to set a boundary …”). Prong 3: no structure recited in the claim. Corresponding structure disclosed in the specification is a processor executing the algorithm of [0148-0155], applying Mathematical Formula 1 Δ = Rv/V ([0149]) with per-object retrieval of maximum velocity by object type ([0151-0152]) and reflecting per-axis pixel spatial resolution ([0155]).
“target-shadow matching unit configured to find and match a shadow of the moving object among the extracted shadows within the set boundary, using a machine learning model” (Claim 2 and by reference Claims 7, 10). Prong 1: “unit” is a generic placeholder. Prong 2: modified by functional language. Prong 3: no structure. Corresponding structure disclosed in the specification is a processor executing the algorithm of [0157-0181] and Fig. 6, including a machine learning model generation unit (131), a matching candidate verification unit (132), and a matching result acquisition unit (133), operating on shadows located within the boundary set per Formula 1.
“target information calculation unit” (Claims 8, 10). Prong 1: generic placeholder “unit”. Prong 2: functional. Prong 3: no structure. Corresponding structure disclosed is a processor executing Mathematical Formula 2 of [0183] and the surrounding computations at [0181-0187], including calculation of ΔD reflecting range and azimuth spatial resolutions.
“target setting unit configured to filter out stationary objects with no velocity …” (Claim 10). Prong 1: generic placeholder. Prong 2: functional. Prong 3: no structure. Corresponding structure disclosed is a processor performing deep-learning object extraction ([0136]) or backscattering-coefficient-threshold filtering ([0139]).
“machine learning model generation unit configured to generate shadow learning data …” (Claims 7 and 14). Prong 1: generic placeholder. Prong 2: functional. Prong 3: no structure. Corresponding structure disclosed is a processor executing the pipeline of Fig. 4 (steps S100–S140): input 3D models by category, simulate virtual shadow data under SAR environmental variables, mix virtual and actual shadow data at a maximum ratio ([0223]: “maximum of 8:2”), and iteratively learn until “F1-score of 0.8 or higher”.
“matching result acquisition unit” (Claims 7, 14). Prong 1: generic placeholder. Prong 2: functional. Prong 3: no structure. Corresponding structure disclosed for the moving-target embodiment (Claim 7) is a processor executing [0175-0180] (retrieving beta angle, forward/rear determination). Claim 14 recites this unit narrowly — “acquires the natural object identified from the extracted shadows through the machine learning model” — and corresponding structure for that narrower function is disclosed at [0284-0289] and [0306] (result acquisition unit (233) retrieving the natural object inferred/identified by the machine learning model of Fig. 4). Claim 14 does not recite acquisition of shadow-size- or shape-based object information; that function is separately recited in Claim 15’s “object information calculation unit”.
“shadow extraction unit” (Claims 12, 13, 15). Prong 1: generic placeholder. Prong 2: functional. Prong 3: no structure. Corresponding structure disclosed is a processor executing [0267-0280]: filtering artificial-object shadows out; searching for shadows without a visible source using a variable-size window (starting at 3×3, sequentially increasing/decreasing); and deep-learning classification of visible/non-visible sources.
“shadow identification unit” (Claims 12, 14, 15). Prong 1: generic placeholder. Prong 2: functional. Prong 3: no structure. Corresponding structure disclosed is a processor executing [0283-0311]: machine learning model of Fig. 4 applied to natural-object categories, inference or identification of natural objects, cross-verification against public data (SAR from other orbits, RGB satellite, aerial, GIS), and iterative feedback re-learning.
“shadow enhancement unit” (Claim 15). Prong 1: generic placeholder. Prong 2: functional. Prong 3: no structure. Corresponding structure disclosed is a processor executing [0263-0265]: filtering techniques including multi-look with median filter and shortening the synthesis interval to increase shadow resolution.
“object information calculation unit that calculates information about the natural object identified by the shadow identification unit from the extracted shadow” (Claim 15). Prong 1: generic placeholder “unit”. Prong 2: functional. Prong 3: no structure recited in the claim. The specification does not disclose adequate corresponding structure for this limitation. The only supporting disclosure is [0311-0312]: “the object information calculation unit (240) can calculate information about the natural object based on the natural object identified by the shadow identification unit (230) and/or the shadow” and, by way of example, “can calculate the height (or elevation), size, etc. of the identified natural object based on information such as the length, size, occupied area, etc. of the shadow“. This is a restatement of the claimed function rather than a disclosure of the structure, formula, or model that performs it. By contrast, the parallel moving-target-side limitation (item 3 above, “target information calculation unit”) is supported by an explicit mathematical relationship (Formula 2); no analogous relationship, geometric or otherwise, is disclosed for the natural-object side. Under Biomedino, LLC v. Waters Tech. Corp., 490 F.3d 946, 952 (Fed. Cir. 2007), “a bare statement that known techniques or methods can be used does not disclose structure” for purposes of 112(f); the same reasoning applies where, as here, the specification asserts only that a calculation “can” be performed without disclosing how. This limitation is accordingly indefinite under 112(b).
“correction module” (Claim 8). Prong 1: “module” is a generic placeholder. Prong 2: functional (“reprocesses the synthetic aperture radar image by reflecting …”). Prong 3: no structure. Corresponding structure disclosed is a processor executing [0188-0192], including geometric/radiometric correction and repositioning the shifted object at its actual position.
“moving target object processing module” and “natural object processing module” (Claims 1, 12, and elsewhere). The generic placeholder, “module,” is a nonce term. Prong 1 satisfied. Prong 2: modified by functional language. Prong 3: no recited structure. Corresponding structure disclosed for the moving target object processing module is a processor implementing the assembly of sub-units 110/120/130/140 (Fig. 2), executing the algorithms described in [0130-0187]. Corresponding structure disclosed for the natural object processing module is a processor implementing sub-units 210/220/230/240 (Fig. 9), executing the algorithms described in [0258-0312].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 15, the claim recites “an object information calculation unit that calculates information about the natural object identified by the shadow identification unit from the extracted shadow.” As discussed previously, this limitation invokes 112(f), and the specification’s only supporting disclosure ([0051, [0311-0312]) restates the claimed function — calculating height, size, etc. based on information such as the length, size, occupied area, etc. of the shadow — without disclosing any formula, geometric relationship, model, or other structure by which the calculation is performed. This stands in contrast to the moving-target side’s parallel limitation (“target information calculation unit,” Claim 8), which is supported by an explicit mathematical relationship (Formula 2, [0183]
PNG
media_image1.png
68
380
media_image1.png
Greyscale
). Because no corresponding structure, material, or act is disclosed for the natural-object-side calculation, the metes and bounds of Claim 15 cannot be determined, and the claim is indefinite under 112(b) per MPEP § 2181(II)(B). Applicant is invited to amend Claim 15 to recite, or to point to specification support for, the specific relationship (e.g., a shadow-length-to-height ratio referenced against a feature of known height, analogous to the boundary and velocity formulas disclosed for the moving-target embodiment) by which the object information calculation unit performs its function or similar.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 8, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (“Ground Moving Target Tracking and Refocusing Using Shadow in Video-SAR,” Remote Sens. 2020, 12, 3083 published September 20, 2020).
Regarding Claim 1, Yang et al. teaches:
Yang et al. teaches a system for detecting object information based on a synthetic aperture radar image (Abstract: “Stable and efficient ground moving target tracking and refocusing is a hard task in synthetic aperture radar (SAR) data processing”; Yang et al.’s framework detects, tracks, refocuses, and classifies multiple moving targets from SAR image data), comprising:
Yang et al. teaches a receiver module configured to receive an image of a synthetic aperture radar (pg. 10: “The transmitter radiates LFM pulses into the observation area with a fixed pulse repetition frequency (PRF), and the electromagnetic waves inspire the scattering electromagnetic fields that arrive at the receiver with some delays. The receiver acquires the echoes corresponding to the different pulse repetition indices (PRIs) after a specific delay”). Yang et al.’s SAR system includes a receiver that acquires radar echoes and processes them into SAR images via the video-SAR back-projection (v-BP) algorithm.
Yang et al. teaches a processing module configured to: i) extract shadows defined as shaded areas where electromagnetic waves cannot reach in the received image (pg. 6: “Because of the shielding effect of the target, the scattering point on the ground cannot interact with the radar electromagnetic wave, which leads to shadowing”; Yang et al., p. 8, Eq. 31
PNG
media_image2.png
200
400
media_image2.png
Greyscale
: the echo at the shadow position is 0 when the scattering point is completely sheltered by the target). Yang et al.’s system defines and extracts shadows as areas where the target blocks electromagnetic waves from reaching ground scattering points, which corresponds to the claimed shaded areas where electromagnetic waves cannot reach.
Yang et al. teaches ii) distinguish artificial objects that reflect electromagnetic waves in the received image (pg. 13, Section 5: “roads and vehicles are considered as background and moving targets, respectively”; Yang et al., pg. 6: “When the target is moving (i.e., range and azimuth velocity are both not zero), its shadow will separate form its defocused image significantly due to the deviation phenomenon caused by the Doppler frequency”). Yang et al.’s vehicles are man-made targets that produce strong radar returns (Doppler-shifted echoes) allowing them to be distinguished from the natural terrain background in the SAR image.
Yang et al. teaches iii) calculate information of the distinguished artificial objects (pg. 10: “the shadow motion has a clear geometric meaning, which is convenient for estimating the position and velocity of moving targets and provides necessary information for moving target focusing”; Yang et al., pg. 6, Eq. 25:
PNG
media_image3.png
200
400
media_image3.png
Greyscale
, giving the positional offset of the moving target; Yang et al., pg. 23: “the velocity error is less than 0.1 m/s in our numerical experiments”). Yang et al. calculates position and velocity information for the distinguished moving targets.
Yang et al. teaches wherein the processing module includes a moving target object processing module configured to determine an actual position of a moving object moving with velocity among the artificial objects based on a position of the extracted shadow (pg. 13: “No matter how the target speed varies, its shadow position is fixed relative to the road, and it can thus be applied to locate tracking”; Yang et al., pg. 1, Abstract: “shadows in video-SAR indicate the actual positions of moving targets at different moments without any displacement”). Yang et al.’s framework uses the shadow’s fixed position to determine the actual position of the moving vehicle, because the shadow does not suffer from Doppler-induced displacement.
Regarding Claim 8, Yang et al. teaches the system of Claim 1 as set forth above.
Yang et al. teaches a target information calculation unit configured to acquire a moving velocity or moving direction of the moving object based on the determined actual position of the moving object (pg. 10: “the shadow motion has a clear geometric meaning, which is convenient for estimating the position and velocity of moving targets”; Yang et al., p. 6, Eq. 25
PNG
media_image3.png
200
400
media_image3.png
Greyscale
, from which the moving velocity vr can be derived when the offset ∆y, range R, and platform velocity vp are known; Yang et al., p. 23: “the velocity error is less than 0.1 m/s in our numerical experiments”). Yang et al. calculates velocity from the spatial relationship between the target’s actual position (shadow) and its displaced image position.
Yang et al. teaches a correction module configured to reprocess the synthetic aperture radar image by reflecting the moving velocity or moving direction or shifted position information of the moving object (pg. 19, Section 5.3: Yang et al.’s moving target back-projection (m-BP, pg. 22) algorithm refocuses the moving target image by using the estimated trajectory; Yang et al., pg. 20: “the imaging result is able to present the detailed features of the moving target”). Yang et al.’s m-BP algorithm reprocesses the SAR data to produce a corrected, refocused image of the moving target based on its estimated velocity and trajectory, which constitutes reprocessing the SAR image by reflecting the moving velocity and shifted position information.
Regarding Claim 9, Yang et al. teaches the system of Claim 1 as set forth above.
Yang et al. teaches the moving object appears at a position separated from its own shadow in the image received by the receiver module (pg. 6: “When the target is moving (i.e., range and azimuth velocity are both not zero), its shadow will separate form its defocused image significantly due to the deviation phenomenon caused by the Doppler frequency”; pg. 13: “the larger the range speed is, the greater the target offset is”). Yang et al. expressly teaches that the moving target’s image position is separated from its shadow due to Doppler-induced displacement.
Yang et al. teaches the moving target object processing module is configured to filter out objects connected to shadows in the received image as not being the moving object (pg. 15: “offset will not exist in the imaging result when target radial velocity is 0 and available shadow can not be obtained”). Yang et al. teaches that when a target has no radial velocity, its image position is not offset from its shadow — i.e., the object and shadow remain connected. Yang et al.’s system processes only targets whose images are separated from their shadows, effectively filtering out stationary objects (those connected to their shadows) as not being moving objects.
Claims 2, 5, 7, 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. in view of Bao et al. (“ShadowDeNet: A Moving Target Shadow Detection Network for Video SAR,” Remote Sensing, vol. 14, no. 2, 2022, pp. 320).
Regarding Claim 2, Yang et al. teaches the system of Claim 1 as set forth above.
Yang et al. does not explicitly teach a boundary setting unit configured to set a boundary based on a maximum velocity of the moving object to match the extracted shadow with the moving object; and a target-shadow matching unit configured to find and match a shadow of the moving object among the extracted shadows within the set boundary, using a machine learning model. However, Bao et al. teaches both a boundary-based shadow search and machine-learning-based shadow matching. Bao et al.’s ShadowDeNet employs a region proposal network to generate candidate shadow regions within the SAR image and then uses a deep neural network to detect and match moving target shadows (pg. 5: “a region proposal network (RPN), and a Fast R-CNN”; pg. 2: “the shadow reflects the real position and motion state information of the moving target”). The region proposal network defines spatial boundaries within which shadow candidates are evaluated, and the deep network performs the matching function.
It would have been obvious to a person having ordinary skills in the art (PHOSITA) before the effective filing date of the claimed invention to modify Yang’s shadow-tracking framework to incorporate Bao et al.’s region-proposal-based shadow detection with boundary-constrained candidate generation and deep-network matching, and to set the search boundary based on the maximum velocity of the moving object. One would have been motivated to do so because Bao teaches that shadow detection “can also greatly expand the detectable velocity range of moving targets and improve the robustness of trackers further” (Bao et al. pg. 2), expressly motivating a system designed to accommodate targets across a wide range of velocities. A PHOSITA seeking to expand the detectable velocity range would size the search boundary to the maximum expected target velocity, since the spatial separation between a target and its shadow increases with speed — a well-known property of SAR moving target imaging — and a boundary sized to the maximum velocity ensures that the matching shadow falls within the search area for all targets up to that speed. There is a reasonable expectation of success because both Yang et al. and Bao et al. operate on video-SAR shadow data for ground moving target indication, use deep learning architectures for shadow processing, and address the same fundamental problem of detecting and locating moving target shadows in SAR imagery.
Regarding Claim 5, Yang et al. in view of Bao et al. teaches the system of Claim 2 as set forth above.
Yang et al. does not explicitly teach the boundary setting unit is configured to vary a size of the boundary based on the maximum velocity of the moving object. However, Bao et al. teaches the motivation for varying the boundary size based on target velocity. Bao teaches that shadow detection (pg. 2: “can also greatly expand the detectable velocity range of moving targets and improve the robustness of trackers further”). Bao et al.’s ShadowDeNet employs a region proposal network that generates candidate regions of varying sizes within the SAR image (pg. 5). Accommodating a wide velocity range, as Bao et al. expressly motivates, requires varying the size of the search boundary because the spatial separation between a target and its shadow increases with target speed.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to configure the boundary setting unit of the combined Yang et al./Bao et al. system to vary the size of the boundary based on the maximum velocity of the moving object. One would have been motivated to do so because Bao et al. expressly seeks to expand the detectable velocity range, and a PHOSITA would recognize that a faster-moving target produces a larger displacement from its shadow, requiring a correspondingly larger search boundary. Varying the boundary size based on the maximum velocity of the target class ensures that the search area is neither too small (missing high-velocity target shadows) nor unnecessarily large (increasing false alarms for low-velocity targets). There is a reasonable expectation of success because Bao’s region proposal network already generates variable-sized candidate regions, and scaling the region based on expected velocity is a straightforward parameterization of that existing mechanism.
Regarding Claim 7, Yang et al. in view of Bao et al. teaches the system of Claim 2 as set forth above.
Yang et al. does not explicitly teach a machine learning model generation unit configured to generate shadow learning data of synthetic aperture radar images for each moving object by distinguishing moving objects by category and generate a machine learning model for shadow detection and matching. However, Bao et al. teaches a machine learning model that generates shadow detection capability from SAR image training data, where the training data includes shadow features from different target categories. Bao et al.’s ShadowDeNet uses a backbone network (ResNet-50) trained on annotated video-SAR shadow data to generate a detection model for shadow identification and matching (pg. 5: “a backbone network is used to extract shadow features. In ShadowDeNet, without losing generality, we select the commonly-used ResNet-50”). The training data distinguishes shadow characteristics by target type as shadows vary with target geometry.
PNG
media_image4.png
200
400
media_image4.png
Greyscale
Yang et al. does not explicitly teach a matching result acquisition unit configured to match the moving object and shadow through the machine learning model generated by the machine learning model generation unit and acquire information of the moving object based on a position of the shadow, wherein the matching result acquisition unit is configured to calculate a beta angle defined as an angle between a direction in which the synthetic aperture radar views the moving object and a direction in which the moving object faces forward, from the position of the matched shadow. However, Bao et al. teaches matching moving targets with their shadows and acquiring position information from the matched shadow. Bao et al.’s detection network outputs bounding boxes that locate the shadow positions (pg. 2: “the shadow reflects the real position and motion state information of the moving target”).
Yang et al. does not explicitly teach the beta angle, however Korchev et al. (‘974) teaches calculating the aspect angle of a target relative to the radar viewing direction ([0116]: “the renderer 1216 uses aspect angles relative to the radar 1202”), which corresponds to the claimed beta angle — the angle between the SAR viewing direction and the target’s forward-facing direction.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Yang et al.’s shadow-tracking system to incorporate Bao et al.’s deep-learning-based shadow detection model with category-distinguished training data, and to further calculate the aspect angle (beta angle) of the target relative to the SAR viewing direction from the matched shadow position. One would have been motivated to do so because Bao et al. demonstrates superior shadow detection performance over traditional methods evaluated in Yang et al., and calculating the target’s orientation angle relative to the radar is a standard SAR parameter used in target characterization and recognition (as shown by Korchev et al. (‘974)’s use of aspect angle). There is a reasonable expectation of success because both Yang et al. and Bao et al. process SAR shadow data using deep learning, and aspect angle computation from target-shadow geometry is a well-established SAR technique.
Regarding Claim 10, Yang et al. in view of Bao et al. teaches the system of Claim 2 as set forth above.
Yang et al. teaches the receiver module is configured to receive a level 1 image of a single synthetic aperture radar (pg. 3: Yang et al.’s SAR system processes echoes from a single radar platform through range compression and coherent accumulation via the back-projection algorithm to produce SAR images; Figure 4, pg. 10: the v-BP algorithm produces individual SAR image frames). The application defines a level 1 image as a two-dimensional image produced from raw SAR data through range compression and azimuth compression. Yang et al.’s BP-processed images are such level 1 images from a single SAR sensor.
PNG
media_image5.png
200
400
media_image5.png
Greyscale
Yang et al. teaches wherein the moving target object processing module further includes a target setting unit configured to filter out stationary objects with no velocity in the received level 1 image and target moving objects that are a subject of information detection (pg. 13: “roads and vehicles are considered as background and moving targets, respectively”; pg. 15: “offset will not exist in the imaging result when target radial velocity is 0”). Yang et al.’s framework distinguishes stationary background (roads, terrain) from moving targets (vehicles) and processes only the moving targets for information detection.
The boundary setting unit and target-shadow matching unit limitations are taught by the combination of Yang et al. and Bao et al. as set forth in the rejection of Claim 2 above. Bao et al. teaches constraining the shadow search region using a region proposal network and expressly motivates expanding the detectable velocity range (Bao et al. pg. 2), which drives sizing the boundary to the maximum velocity of the target class. Bao’s ShadowDeNet generates candidate shadow regions within bounded areas and uses deep-network matching to identify shadows within those regions (Bao et al. pg. 5), as set forth in the rejection of Claim 2.
Yang et al. teaches wherein the moving target object processing module further includes a target information calculation unit configured to calculate velocity and direction information of the moving target object based on the determined shadow (Yang et al., p. 10: “convenient for estimating the position and velocity of moving targets”; pg. 5, Eq. 12:
PNG
media_image6.png
200
400
media_image6.png
Greyscale
the velocity vector v = [vx, vy, vz] resolves the target’s motion into directional components; pg. 6, Eq. 25:
PNG
media_image7.png
200
400
media_image7.png
Greyscale
the offset formula allows velocity calculation from shadow position). Yang et al. calculates velocity from the shadow-target displacement, and the velocity vector components provide directional information.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to combine Yang et al.’s target setting and velocity/direction calculation with Bao’s deep-learning-based shadow detection and matching within velocity-bounded search regions, for the same reasons and with the same reasonable expectation of success as set forth in the rejection of Claim 2.
Regarding Claim 11, Yang et al. in view of Bao et al. teaches the system of Claim 10 as set forth above.
Yang et al. teaches the mathematical relationship pg. 6, Eq. 25: ∆y = vrR/vp, where vr is the target’s radial velocity, R is the range, and vp is the platform velocity). Yang et al.’s formula is algebraically identical to the claimed formula Δ=Rv/V. The claim variable Δ corresponds to Yang et al.’s ∆y (displacement), R corresponds to Yang et al.’s R (range), v corresponds to Yang et al.’s vr (target velocity), and V corresponds to Yang et al.’s vp (platform velocity).
PNG
media_image7.png
200
400
media_image7.png
Greyscale
=
PNG
media_image8.png
23
78
media_image8.png
Greyscale
. The claim specifies that v is the maximum velocity of the moving target object, making the resulting Δ a maximum position displacement that defines the boundary. Substituting the maximum velocity into Yang’s formula to obtain the maximum displacement for boundary setting is the direct application of Yang’s formula to Bao’s goal of expanding the detectable velocity range (Bao et al. pg. 2: “the shadow detection can also greatly expand the detectable velocity range of moving targets”). A PHOSITA combining Yang’s displacement formula with Bao’s velocity-range objective would substitute the maximum expected velocity to compute the maximum displacement.
Yang et al. teaches wherein an actual position of the moving target object in the received level 1 image is determined as a position connected to the matched shadow (pg. 1, Abstract: “shadows in video-SAR indicate the actual positions of moving targets at different moments without any displacement”; pg. 13: “its shadow position is fixed relative to the road, and it can thus be applied to locate tracking”). Yang et al. determines the actual target position as the position where the shadow appears, since the shadow is unaffected by Doppler displacement.
Claims 3, 4, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. in view of Bao et al. and further in view of Korchev et al. (US 2017/0350974 A1).
Regarding Claim 3, Yang et al. in view of Bao et al. teaches the system of Claim 2,
Yang et al. in view of Bao et al. does not explicitly teach the machine learning model is configured to repeatedly generate normalizing data by mixing virtual shadow learning data that simulates changes in synthetic aperture radar environment variables and actual shadow data acquired through actual synthetic aperture radar images. However, Korchev et al. (‘974) teaches generating synthetic SAR shadow data from 3D CAD models and combining it with real SAR image data for training a classifier. Korchev et al. (‘974) teaches rendering synthetic SAR shadows from CAD models at varying aspect, grazing, and tilt angles — which correspond to SAR environment variables — and comparing these synthetic shadows with real SAR image data ([0103]: “a database 1214 containing at least one CAD model is provided… a “simple” SAR renderer 1216 is used to render a synthetic SAR shadow of the at least one CAD model in the database, optionally using the metadata 1208”; [0114]: “The renderer module 1216 generates a variety of orthographic projections of the CAD model for the range of grazing, aspect and tilt angles of the CAD model relative to the radar 1202 location… Each combination of these angles will produce an image and corresponding far edge of the shadow”; [0102]: the metadata 1208 includes “grazing angle, aspect angle, geolocation”). Korchev et al. (‘974)’s regression renderer is also trained on both real SAR data and CAD models ([0135]: “the regression that was trained on known targets with real (i.e. known) SAR data and CAD models of these targets”), teaching the mixing of synthetic and actual SAR data as normalizing training data.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the machine learning model of Yang et al. in view of Bao et al. to incorporate Korchev et al. (‘974)’s technique of generating synthetic shadow training data from CAD models across varying SAR environment variables and mixing that synthetic data with actual SAR shadow data for training. One would have been motivated to do so because Korchev et al. (‘974) expressly addresses the difficulty of acquiring sufficient real SAR training data (Korchev et al. (‘974), [0002]: the severe hardware requirements of SAR data acquisition), and demonstrates that synthetic shadow data generated from CAD models at varied angles can supplement limited real SAR data to improve classifier robustness. There is a reasonable expectation of success because Korchev et al. (‘974) demonstrates a working system that trains on mixed synthetic-and-real SAR data for target recognition, and both Yang et al./Bao et al. and Korchev et al. (‘974) operate on SAR imagery with shadow analysis as a central feature.
Regarding Claim 4, Yang et al. in view of Bao et al. and Korchev et al. (‘974) teaches the system of Claim 3 as set forth above.
Yang et al. does not explicitly teach the machine learning model is configured to perform repeated learning based on the normalizing data so that an F1-score defined as a harmonic mean of precision and recall is acquired to be 0.8 or higher. However, Bao et al. teaches the use of F1-score as a performance metric for shadow detection. Bao et al. reports F1-scores for its ShadowDeNet shadow detection network across multiple experimental configurations (pg. 17, Table 3: f1 scores reported for various configurations including ShadowDeNet achieving approximately 66.01% f1; pg. 22, Table 5: f1 scores for ablation studies). The F1-score is the harmonic mean of precision and recall, exactly as defined by the claim. Setting a target F1-score threshold of 0.8 or higher as a training objective is a design choice representing optimization of a result-effective variable — the training threshold — which a person of ordinary skill would routinely select based on the desired detection reliability for the particular application.
PNG
media_image9.png
200
400
media_image9.png
Greyscale
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to configure the machine learning model of the combined Yang et al./Bao et al./Korchev et al. (‘974) system to perform repeated learning until an F1-score of 0.8 or higher is achieved. One would have been motivated to do so because Bao et al. already evaluates its shadow detection model using F1-score as a key metric, and setting a minimum F1-score threshold is a routine optimization of a result-effective variable (see MPEP § 2144.05(II)) that ensures the model achieves adequate precision and recall before deployment. There is a reasonable expectation of success because Bao et al. demonstrates that iterative training with architectural improvements progressively increases F1-score, confirming that repeated learning to meet a target threshold is achievable within the video-SAR shadow detection domain.
Regarding Claim 16, Yang et al. in view of Bao et al. and Korchev et al. (‘974) teaches:
Yang et al. teaches a method for detecting object information based on a synthetic aperture radar image, (Abstract) comprising:
Yang et al. teaches receiving an image of a synthetic aperture radar (pg. 10: the v-BP algorithm receives and processes SAR echoes into image frames, as cited for Claim 1).
Yang et al. teaches extracting shadows defined as shaded areas where electromagnetic waves cannot reach in the received image (pg. 6: “Because of the shielding effect of the target, the scattering point on the ground cannot interact with the radar electromagnetic wave, which leads to shadowing”, as cited for Claim 1).
Yang et al. teaches selecting a moving object with velocity among artificial objects that reflect electromagnetic waves in the received image, excluding a stationary object with no velocity and a shadow connected to the stationary object, to set moving target object (pg. 13: “roads and vehicles are considered as background and moving targets, respectively”; pg. 15: “offset will not exist in the imaging result when target radial velocity is 0”). Yang et al. selects only moving vehicles (artificial objects with velocity) and excludes stationary background, as stationary objects show no displacement from their shadows.
Yang et al. does not explicitly teach setting a boundary for the moving target object based on the maximum velocity of the moving target object. Bao et al. teaches constraining the shadow search region using a region proposal network and expressly motivates expanding the detectable velocity range (pg. 2: “the shadow detection can also greatly expand the detectable velocity range of moving targets”), which drives sizing the boundary to the maximum velocity of the target, as set forth in the rejection of Claim 2.
Yang et al. does not explicitly teach matching the extracted shadow located within the set boundary with the moving target object using a machine learning model for the moving target object. Bao et al. teaches shadow matching via deep neural network within constrained candidate regions, as cited for Claim 2.
Yang et al. does not explicitly teach loading beta angle information based on the matched moving target object and shadow in a matching result acquisition step. Korchev et al. (‘974) teaches calculating the aspect angle of a target relative to the radar viewing direction from SAR shadow data ([0116]: “the renderer 1216 uses aspect angles relative to the radar 1202”), which corresponds to the beta angle.
Yang et al. teaches calculating a moving velocity of the moving target object using a mathematical formula Vmovingvelocity = VsΔD / (R·cosβ). Yang et al.’s displacement formula ∆y = vrR/vp (Yang et al., p. 6, Eq. 25) provides the velocity calculation from the displacement between the target and its shadow. The claimed formula Vmovingvelocity = Vs·ΔD/(R·cosβ) adds a cosine beta correction for the target’s aspect angle, which accounts for the angular relationship between the SAR viewing direction and the target’s forward direction.
Yang et al. does not explicitly teach, but Korchev et al. (‘974) teaches aspect angle calculation as set forth above, and incorporating a cosine correction for the aspect angle into Yang et al.’s velocity formula is a straightforward geometric refinement that a PHOSITA would apply.
It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to modify Yang et al.’s shadow-based velocity calculation method to incorporate Bao et al.’s deep-learning-based shadow detection and matching within boundary-constrained search regions, and to add Korchev’s aspect angle (beta angle) computation and a corresponding cosine correction to the velocity formula. One would have been motivated to do so because Bao’s deep-learning approach provides superior shadow detection performance, and incorporating the aspect angle correction per Korchev et al. (‘974) improves velocity estimation accuracy by accounting for the target’s orientation relative to the radar. There is a reasonable expectation of success because all three references operate on SAR imagery for target detection and characterization, and the cosine correction is a standard trigonometric adjustment in radar signal processing.
Regarding Claim 17, Yang et al. in view of Bao et al. and Korchev et al. (‘974) teaches the method of Claim 16 as set forth above.
Yang et al. teaches feeding back the matched shadow and moving target object result to a step of the setting a boundary by storing it (pg. 12: SiamFc tracker updates the target position from frame to frame, using the tracking result from the current frame as the search center for the next frame; Yang et al., p. 19, Fig. 17: trajectory reconstruction results show that tracking results are stored and accumulated across frames). This frame-to-frame feedback of matched shadow-target results to refine subsequent tracking constitutes feeding back the result to the boundary setting step.
PNG
media_image10.png
200
400
media_image10.png
Greyscale
Yang et al. Page 12
PNG
media_image11.png
200
400
media_image11.png
Greyscale
Yang et al. Page 17
Claims 6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. in view of Bao et al. and Korchev et al. (‘974), and further in view of Medasani et al. (US 2013/0004017 A1).
Regarding Claim 6, Yang et al. in view of Bao et al. and Korchev et al. (‘974) teaches the system of Claim 3 as set forth above.
Korchev et al. (‘974), already applied in the rejection of Claim 3, further teaches: i) simulate and store virtual shadow data according to changes in synthetic aperture radar environment variables based on 3D models of artificial objects with high electromagnetic wave reflectivity to generate virtual shadow learning data ([0103]: “a database 1214 containing at least one CAD model is provided… a “simple” SAR renderer 1216 is used to render a synthetic SAR shadow of the at least one CAD model in the database”; [0114]: “The renderer module 1216 generates a variety of orthographic projections of the CAD model for the range of grazing, aspect and tilt angles of the CAD model relative to the radar 1202 location… Each combination of these angles will produce an image and corresponding far edge of the shadow”; [0002]: the CAD models include “accurate settings of electromagnetic properties of all parts of the CAD models”). The CAD models are three-dimensional models of man-made targets (ships, vehicles) with high electromagnetic wave reflectivity, the aspect, grazing, and tilt angles are SAR environment variables, and the generated synthetic shadow projections constitute virtual shadow learning data.
Korchev et al. (‘974) further teaches mixing the generated virtual shadow data with actual SAR data for training ([0135]: “the regression that was trained on known targets with real (i.e. known) SAR data and CAD models of these targets”).
However, the combination of Yang et al., Bao et al., and Korchev et al. (‘974) does not explicitly teach, but Medasani et al. (‘017) teaches: ii) learn by mixing the virtual shadow data and actual shadow data at a predetermined maximum ratio ([0096]: “shadow detector 308 applies number of thresholds 326 to processed image 316”). Medasani et al. (‘017) demonstrates that predetermined parameter settings are routinely employed in SAR image classification systems. Setting a predetermined maximum ratio between virtual and actual shadow data is an analogous parameter choice that controls the relative contribution of synthetic versus real data during training.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the machine learning model of Yang et al. in view of Bao et al. and Korchev et al. (‘974) to further set a predetermined maximum ratio for mixing virtual and actual shadow data per Medasani et al. (‘017)’s teaching. One would have been motivated to do so because setting a maximum mixing ratio prevents synthetic data from dominating the training distribution and degrading generalization to real-world SAR data, which is a standard concern in machine learning when combining synthetic and real training data. There is a reasonable expectation of success because Korchev et al. (‘974) demonstrates a working system trained on mixed real-and-synthetic SAR data, and Medasani et al. (‘017) confirms that parameterized thresholds are standard practice in SAR classification.
Regarding Claim 18, Yang et al. in view of Bao et al. and Korchev et al. (‘974) teaches the method of Claim 16 as set forth above.
Yang et al. does not explicitly teach, however, Korchev et al. (‘974), already applied in the rejection of Claim 16, further teaches: inputting 3D models of artificial objects with high electromagnetic wave reflectivity by category to generate virtual shadow data ([0103]: “a database 1214 containing at least one CAD model is provided… a “simple” SAR renderer 1216 is used to render a synthetic SAR shadow of the at least one CAD model in the database”; [0002]: the CAD models include “accurate settings of electromagnetic properties of all parts of the CAD models”). The CAD models are three-dimensional models of man-made targets (ships, vehicles) with high electromagnetic wave reflectivity, organized by target type, and the generated output is virtual shadow data.
Korchev et al. (‘974) further teaches: generating and storing the virtual shadow data according to changes in synthetic aperture radar environment variables based on the input 3D models ([0114]: “The renderer module 1216 generates a variety of orthographic projections of the CAD model for the range of grazing, aspect and tilt angles of the CAD model relative to the radar 1202 location… Each combination of these angles will produce an image and corresponding far edge of the shadow”; [0102]: the associated metadata 1208 includes “grazing angle, aspect angle, geolocation”). The aspect, grazing, and tilt angles are SAR environment variables, and Korchev et al. (‘974)’s renderer generates and stores the synthetic shadow projections across the range of these variables.
Korchev et al. (‘974) further teaches mixing the generated virtual shadow data with actual SAR data for training ([0135]: “the regression that was trained on known targets with real (i.e. known) SAR data and CAD models of these targets”).
However, Yang et al. in view of Bao et al. and Korchev et al. (‘974) does not explicitly teach, but Medasani et al. (‘017) teaches: normalizing by mixing the virtual shadow data generated by combinations of changes in synthetic aperture radar environment variables and an actual shadow data acquired by actual synthetic aperture radar images at a predetermined maximum ratio ([0096]: “shadow detector 308 applies number of thresholds 326 to processed image 316”). Medasani et al. (‘017) demonstrates that predetermined parameter settings are routine in SAR classification systems. Setting a predetermined maximum ratio between virtual and actual shadow data is an analogous parameter choice that controls the relative contribution of synthetic versus real data during training.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Yang et al. in view of Bao et al. and Korchev et al. (‘974) to further set a predetermined maximum ratio for mixing virtual and actual shadow data per Medasani et al. (‘017)’s teaching. One would have been motivated to do so because setting a maximum mixing ratio prevents synthetic data from dominating the training distribution and degrading generalization to real-world SAR data, which is a standard concern in machine learning when combining synthetic and real training data. There is a reasonable expectation of success because Korchev et al. (‘974) demonstrates a working system trained on mixed real-and-synthetic SAR data, and Medasani et al. (‘017) confirms that parameterized thresholds are standard practice in SAR classification.
Yang et al. does not explicitly teach, however, Bao et al., already applied in the rejection of Claim 16, further teaches: repeatedly learning so that an F1-score is satisfied (pg. 17, Table 3: F1-scores reported for various configurations; Bao et al., p. 23, Table 5: F1-scores for ablation studies). The F1-score is the harmonic mean of precision and recall, exactly as defined by the claim. Setting a target F1-score threshold of 0.8 or higher as a training objective is a design choice representing optimization of a result-effective variable (see MPEP § 2144.05(II)), which a person of ordinary skill would routinely select based on the desired detection reliability for the particular application.
Claims 12, 13, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. in view of Long et al. (US 5,424,742) and Medasani et al. (‘017).
Regarding Claim 12, Yang et al. teaches the system of Claim 1 as set forth above.
Yang et al. does not explicitly teach, but Long et al. (‘742) teaches: a natural object processing module that distinguishes natural objects that absorb at least a part of electromagnetic waves. Long et al. (‘742)’s system classifies features into two categories: natural terrain features (including grass, trees, dirt/gravel, and water) and man-made structures (col. 47, Table II). Long et al. (‘742) teaches that “Natural features are classified based on texture, gray level and context” (col. 48, lines 1-23). Long et al. (‘742)’s natural terrain features — trees, grass, water — are precisely natural objects, as their radar return characteristics are determined by surface roughness and volumetric scattering rather than specular metallic reflection.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Yang et al.’s SAR processing system to include Long et al. (‘742)’s natural object classification. One would have been motivated to do so because a SAR system that processes shadows of artificial objects, as Yang et al. teaches, necessarily captures image data containing natural terrain features. Long et al. (‘742) demonstrates that comprehensive terrain classification — both natural and man-made — improves scene understanding and target recognition accuracy in SAR imagery. There is a reasonable expectation of success because both Yang et al. and Long et al. (‘742) operate on SAR imagery and address object classification using shadow and return characteristics.
Yang et al. in view of Long et al. (‘742) does not explicitly teach, but Medasani et al. (‘017) teaches: a shadow extraction unit that filters among the extracted shadows so as to detect and extract a shadow formed by a natural object in the image received from the receiver module ([0096]: “shadow detector 308 is configured to identify set of shadows 336”; [0096–0097]: the shadow detector applies intensity thresholds to identify shadow regions in the SAR image). Medasani et al. (‘017)’s shadow detector identifies all shadows in the image, extracting them as a set for subsequent classification.
Medasani et al. (‘017) further teaches: a shadow identification unit that identifies the natural object forming the shadow using a machine learning model from the extracted shadows ([0127]: “Vegetation detector 1400 applies texture features filter 1402 to image 228”; [0138]: “Boosted classifiers 1406 are texture classifiers that have been identified using boosting algorithms, such as, for example, without limitation, adaptive boosting”). Medasani et al. (‘017)’s boosted classifiers constitute a machine learning model that identifies vegetation — a natural object — from SAR image features including the extracted shadows.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to further modify the Yang et al./Long et al. (‘742) system to incorporate Medasani et al. (‘017)’s shadow detection and machine-learning-based vegetation identification. One would have been motivated to do so because Medasani et al. (‘017) demonstrates that automated shadow detection using intensity thresholds, combined with boosted classifiers for vegetation identification, provides a complete processing pipeline for identifying natural objects in SAR imagery — directly complementing Long et al. (‘742)’s natural terrain classification framework. There is a reasonable expectation of success because Medasani et al. (‘017) demonstrates a working shadow-detection-to-classification pipeline for SAR images.
Medasani et al. (‘017) further teaches: wherein the shadow extraction unit is configured to detect the shadow formed by the natural object by selecting a shadow whose visible shape of a source object is unclear in the received image. Medasani et al. (‘017)’s classification module processes both bright areas (high-intensity radar returns) and shadows together to determine which shadows are produced by buildings — man-made structures that produce distinct bright areas as visible source objects ([0098]: “classification module 310 is configured to process set of lines 318, set of bright areas 324, and set of shadows 336 to identify which of the items in these sets are produced by buildings”). Shadows that lack corresponding bright areas — that is, shadows whose visible source objects are not clearly identifiable as man-made structures — are the residual set not classified as building-produced. Selecting shadows that lack clear visible source objects (no corresponding bright area) is functionally equivalent to selecting shadows whose visible shape of a source object is unclear, because natural objects’ low reflectivity produces indistinct or absent visible returns in SAR imagery.
Regarding Claim 13, Yang et al. in view of Long et al. (‘742) and Medasani et al. (‘017) teaches the system of Claim 12 as set forth above.
Yang et al. does not explicitly teach the shadow extraction unit is configured to search for the shadow with unclear visible shapes by applying a window with variable size within the received image. However, Medasani et al. (‘017) teaches applying variable-sized processing windows to SAR images. Medasani et al. (‘017)’s vegetation detector applies texture features filters with texton layout filters to the image (Fig. 2, [0128]: “vegetation detector 1400 applies texture features filter 1402 to image 228”; [0137]: “Vegetation detector 1400 generates number of feature vectors 1424 in response to applying texton layout filter 1418. A feature vector in number of feature vectors 1424 is an n-dimensional vector of numerical features that represent some object or texture in image 228.”). The application of multiple filter configurations across the image constitutes searching with variable window sizes to accommodate different object scales.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to configure the shadow extraction unit of the combined Yang et al./Long et al. (‘742)/Medasani et al. (‘017) system to search for shadows with unclear visible shapes using a window with variable size, as Medasani et al. (‘017)’s multi-scale filtering approach demonstrates. One would have been motivated to do so because natural objects such as trees and vegetation vary significantly in size, and applying variable-size windows ensures that shadows from both small and large natural objects can be detected. There is a reasonable expectation of success because multi-scale windowed filtering is a standard image processing technique well-established in SAR imagery analysis, as demonstrated by Medasani et al. (‘017)’s texture-based vegetation detection.
Regarding Claim 14, Yang et al. in view of Long et al. (‘742) and Medasani et al. (‘017) teaches the system of Claim 12 as set forth above.
Yang et al. does not explicitly teach a machine learning model generation unit that generates shadow learning data of synthetic aperture radar level 1 images for natural objects classified by category and repeatedly learns to generate a machine learning model.
Medasani et al. (‘017) teaches generating a machine learning model for SAR image classification using category-based training. Medasani et al. (‘017)’s vegetation detector uses boosted classifiers trained with boosting algorithms ([0138]: “Boosted classifiers 1406 are texture classifiers that have been identified using boosting algorithms, such as, for example, without limitation, adaptive boosting”). The classifiers are trained on SAR image data to distinguish vegetation textures from other feature types, and the boosting algorithm performs repeated learning iterations to generate the classification model.
Yang et al. does not explicitly teach a matching result acquisition unit that acquires the natural object identified from the extracted shadows through the machine learning model generated by the machine learning model generation unit, wherein the machine learning model repeatedly learns about a shadow of a certain natural object characteristic or a characteristic of a shadow that a certain natural object has. However, Medasani et al. (‘017) teaches acquiring classification results from the machine learning model. Medasani et al. (‘017)’s classification module processes shadow data along with other features to identify objects ([0098]), and the vegetation detector outputs a texture mask identifying vegetation regions ([0138]: “Number of feature vectors 1424 is processed using boosted classifiers 1406 to identify texture mask 1426”). The repeated boosting process learns characteristics that distinguish vegetation shadows and textures from other features.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the combined Yang et al./Long et al. (‘742)/Medasani et al. (‘017) system to include a machine learning model generation unit that generates category-specific shadow learning data and repeatedly learns to build a classification model for natural objects, and a matching result acquisition unit that identifies natural objects from extracted shadows using that model. One would have been motivated to do so because Medasani et al. (‘017) demonstrates that boosted classifiers trained through iterative learning can successfully distinguish vegetation and other terrain features in SAR imagery, and Long et al. (‘742) provides the categorical framework (grass, trees, water, etc.) for organizing natural object training data. There is a reasonable expectation of success because Medasani et al. (‘017)’s boosted classifier approach is a proven machine learning technique for SAR image classification, and combining it with Long et al. (‘742)’s terrain categorization provides a structured training pipeline.
Regarding Claim 15, Yang et al. in view of Long et al. (‘742) and Medasani et al. (‘017) teaches the system of Claim 12 as set forth above.
Yang et al. does not explicitly teach a shadow enhancement unit that enhances a signal or an image of the shadow that natural objects can form through filtering techniques. However, Medasani et al. (‘017) teaches shadow enhancement through image filtering. Medasani et al. (‘017) applies noise filtering to reduce noise in the SAR image before shadow detection ([0084]: “Noise filter 302 is configured to reduce noise 312 that may be present in image 228”), and applies thresholds to enhance and distinguish shadow features ([0096]: “shadow detector 308 applies number of thresholds 326 to processed image 316”). These filtering operations enhance shadow visibility for subsequent processing.
Yang et al. does not explicitly teach an object information calculation unit that calculates information about the natural object identified by the shadow identification unit from the extracted shadow. However, Long et al. (‘742) teaches calculating information about terrain features from shadow data. Long et al. (‘742) teaches a shadow forecasting process that calculates shadow dimensions based on feature height and viewing geometry (Long et al. (‘742), col. 48, lines 65-67: “the height classification process 314b is completed” followed by col. 49, lines 1-18“the radar shadow forecasting process 314c is completed in which a radar shadow forecasting scene is predicted”). Long et al. (‘742)’s system uses shadow geometry to derive feature height, size, and location information. This calculation of terrain feature information from shadow data corresponds to the claimed object information calculation unit.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the combined Yang et al./Long et al. (‘742)/Medasani et al. (‘017) system to include a shadow enhancement unit per Medasani et al. (‘017)’s noise filtering and threshold application, and an object information calculation unit per Long et al. (‘742)’s shadow-based height and size calculation for terrain features. One would have been motivated to do so because Medasani et al. (‘017) demonstrates that pre-processing the SAR image to enhance shadow characteristics improves subsequent detection accuracy, and Long et al. (‘742) demonstrates that shadow geometry can be used to derive physical dimensions of terrain features. There is a reasonable expectation of success because noise filtering and shadow-based geometric calculation are standard, well-established SAR processing techniques.
Claims 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. in view of Long et al. (‘742), Peregrim et al. (US 5,430,445), and Medasani et al. (‘017).
Regarding Claim 19, Yang et al. in view of Long et al. (‘742), Peregrim et al. (‘445), and Medasani et al. (‘017) teaches:
Yang et al. teaches the preamble structural limitations: a method for detecting object information in an object information detection system including a receiver module for receiving a synthetic aperture radar image (pg. 10: Yang et al.’s SAR system receives and processes radar echoes into images), and a processing module for extracting shadows defined as shaded areas where electromagnetic waves cannot reach in the received image (pg. 6: “the scattering point on the ground cannot interact with the radar electromagnetic wave, which leads to shadowing”).
Yang et al. does not explicitly teach calculating information of natural objects that absorb at least a part of electromagnetic waves based on the extracted shadows. Yang et al.’s system processes artificial (vehicle) targets, not natural objects. However, Long et al. (‘742) teaches classifying natural terrain features in SAR imagery, including calculating information about those features. Long et al. (‘742) classifies features into natural terrain (grass, trees, dirt/gravel, water, concrete, asphalt) and man-made structures (col. 47, Table II). Long et al. (‘742) calculates radar cross section values and heights for these features (col. 48, lines 65-57: “the height classification process 314b is completed”; Long et al. (‘742), cols. 49-50: terrain RCS values are calculated from scattering coefficients using the Ohio State Handbook).
Yang et al. in view of Long et al. (‘742) does not explicitly teach a filtering step of selecting shadows whose visible source object is not identified among the extracted shadows. However, Peregrim et al. (‘445) teaches a shadow forecasting and template matching process that distinguishes shadows from their source objects in SAR data. Peregrim et al. (‘445)’s system generates predicted shadow patterns based on target geometry and radar viewing parameters (col. 48, lines 50-63: “the radar shadow forecasting process 314c is completed in which a radar shadow forecasting scene is predicted”). Peregrim et al. (‘445)’s shadow forecasting identifies which image regions are shadows versus source objects, enabling the system to select shadows independently of their source objects. In the context of natural objects with unclear visible shapes, selecting shadows whose source cannot be identified corresponds to filtering for shadows without clear visible sources.
Yang et al. in view of Long et al. (‘742) and Peregrim et al. (‘445) does not explicitly teach a step of inferring or identifying natural objects from the shadows that passed through the filtering step using a machine learning model. However, Medasani et al. (‘017) teaches using machine learning to identify objects from SAR image features including shadows. Medasani et al. (‘017)’s vegetation detector uses boosted classifiers ([0138]: “Boosted classifiers 1406 are texture classifiers that have been identified using boosting algorithms”) to identify vegetation (natural objects) from SAR image data, and Medasani et al. (‘017)’s shadow detector identifies and processes shadows as classification features ([0096-0098]).
Yang et al. does not explicitly teach a step of calculating information of the inferred or identified natural objects based on a size or a shape of the extracted shadows, wherein the information of the inferred or identified natural objects includes the position, length, size, and occupied area of the natural objects. However, Long et al. (‘742) teaches calculating physical dimensions from shadow geometry (Long et al. (‘742), col. 48: the height classification process determines feature heights, and the shadow forecasting process uses feature height and geometry to derive shadow dimensions). From the size and shape of a shadow, the position, length, size, and occupied area of the source object can be geometrically derived — as Long et al. (‘742) demonstrates for terrain features whose physical dimensions are calculated from shadow data and viewing geometry.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Yang et al.’s SAR shadow processing system to incorporate Long et al. (‘742)’s terrain feature classification, Peregrim et al. (‘445)’s shadow forecasting and source-object identification, and Medasani et al. (‘017)’s machine-learning-based vegetation detection to identify and calculate information about natural objects from their shadows. A SAR system that processes shadows of artificial objects, as Yang et al. teaches, inherently captures image data containing natural terrain features. Extending the system to classify and extract information from those natural features is an obvious application of known techniques to the remaining image content. One would have been motivated to do so because Long et al. (‘742) demonstrates that classifying both natural and man-made features improves target scene understanding, Peregrim et al. (‘445) demonstrates shadow-based geometric analysis for SAR targets, and Medasani et al. (‘017) demonstrates automated vegetation detection using machine learning. There is a reasonable expectation of success because all four references operate on SAR imagery and address object detection or classification using shadow and return characteristics, and combining their respective techniques applies each to a compatible SAR processing pipeline.
Regarding Claim 20, Yang et al. in view of Long et al. (‘742), Peregrim et al. (‘445), and Medasani et al. (‘017) teaches the method of Claim 19 as set forth above.
Yang et al. does not explicitly teach the machine learning model repeatedly learns about a shadow of a certain natural object characteristic or a characteristic of a shadow that a certain natural object has. However, Medasani et al. (‘017) teaches repeated machine learning for natural object shadow characteristics. Medasani et al. (‘017)’s boosted classifiers are trained through iterative boosting algorithms ([0138]: “Boosted classifiers 1406 are texture classifiers that have been identified using boosting algorithms, such as, for example, without limitation, adaptive boosting”). Adaptive boosting (AdaBoost) is an iterative algorithm that repeatedly trains weak classifiers on the data, weighting misclassified examples more heavily in each iteration, thereby repeatedly learning about shadow and texture characteristics that distinguish natural objects. Each iteration of the boosting algorithm learns about the characteristics of shadows and textures that particular natural objects produce.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to configure the machine learning model of the combined Yang et al./Long et al. (‘742)/Peregrim et al. (‘445)/Medasani et al. (‘017) system to repeatedly learn about natural object shadow characteristics through Medasani et al. (‘017)’s iterative boosting approach. One would have been motivated to do so because iterative boosting progressively refines the classifier’s ability to distinguish among natural object categories based on their shadow characteristics, improving classification accuracy with each learning iteration. There is a reasonable expectation of success because Medasani et al. (‘017) demonstrates that adaptive boosting successfully identifies vegetation textures in SAR imagery, confirming that iterative learning on natural object characteristics is effective in this domain.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REMASH R GUYAH whose telephone number is (571)270-0115. The examiner can normally be reached M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Resha H Desai can be reached at (571) 270-7792. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REMASH R GUYAH/Examiner, Art Unit 3648
Prosecution Insights