DETAILED ACTION
Claim Interpretations - 35 USC § 112(f)
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as "configured to" or "so that"; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: a first feature extractor, a second feature extractor, a third feature extractor in claims 1-6.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth para.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-4, 6-10 , 12-16, 18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Liu (“Lending Orientation to Neural Networks for Cross-view Geo-localization”, cited from IDS).
Regarding claim 1, Liu teaches a training apparatus comprising:
at least one memory that is configured to store instructions; and
at least one processor that is configured to execute the instructions( Page 4320, right col., GTX1080Ti GPU) to:
acquire a training data including a first ground captured image, a first aerial captured image( Figure 2), and a first map image(Page 4324, left col., orientation maps);
input the first ground captured image to a first feature extractor to extract features of the first ground captured image( CNN in Figure 2);
input the first aerial captured image to a second feature extractor to extract features of the first aerial captured image( CNN in Figure 2);
input the first map image to a third feature extractor to extract features of the first map image( Page 4324, right col., injecting orientation information to the Siamese net at different layers);
compute a combined loss based on the features of the first ground captured image, the features of the first aerial captured image, and the features of the first map image ( Page 4325, left column, 4.3 Triplet loss); and
update the first feature extractor, the second feature extractor, and the third feature extractor based on the combined loss( Page 4325, right col., We train our 7-layer Siamese network…to maximize the number of triplets within each batch).
Regarding claim 2, Liu teaches the training apparatus according to claim 1,
wherein the computation of the combined loss includes:
computing a first loss based on the features of the first ground captured image and the features of the first aerial captured image( Page 4325, left col.,
f
g
-
f
s
2);
computing a second loss based on the features of the first ground captured image and the features of the first map image( Page 4325, left col.,
f
g
-
f
s
*
2 );
computing a third loss based on the features of the first aerial captured image and the features of the first map image( Page 4325, left col.,
f
g
-
f
s
2 -
f
g
-
f
s
*
2 ) and
combining the first loss, the second loss, and the third loss into the combined loss (Page 4325, left col.,
log
{
1
+
e
x
p
[
α
(
f
g
-
f
s
2 -
f
g
-
f
s
*
2
)
}
).
Regarding claim 3, Liu teaches the training apparatus according to claim 2, wherein the combined loss is a weighted sum of the first loss, the second loss, and the third loss(Page 4325, left col.,
log
{
1
+
e
x
p
[
α
(
f
g
-
f
s
2 -
f
g
-
f
s
*
2
)
}
).
Regarding claim 4, Liu teaches the training apparatus according to claim 1,
wherein the at least one process is further configured to:
generate an augmented training data based on the training data( Page 4325, we adopt ‘data augmentation’ strategy), the augmented training data including a second ground captured image, a second aerial captured image, and a second map image,
wherein the generation of the augmented training data includes:
blending a part of the first aerial captured image and a part of the first map image to generate an augmented image( page 4325, right col., circularly shifting the input ground-level panorama by a random relative angle); and
replacing the part of the first aerial captured image with the augmented image to obtain the second aerial captured image, replacing the part of the first map image with the augmented image to obtain the second map image, or doing both (page 4325, right col., resulting in a random rotation of the ground-level image along the azimuth direction ).
[Claim 5] (Original) The training apparatus according to claim 4, wherein the generation of the augmented training data further includes:
determining one or more partial image pairs each of which is a pair of a part of the first aerial captured image and a part of the first map image; and
generating the augmented image for each partial image pair, wherein the part of the first aerial captured image and the part of the first map image that are included in a partial image pair have a same position, a same shape, and a same size as each other, and
the determination of the partial image pair includes determining the position, the shape, and the size for the partial image pair.
Regarding claim 6, Liu teaches the training apparatus according to claim 4,
wherein the at least one processor is further configured to:
input the second ground captured image to a first feature extractor to extract features of the second ground captured image;
input the second aerial captured image to a second feature extractor to extract features of the second aerial captured image;
input the second map image to a third feature extractor to extract features of the second map image;
compute a second combined loss based on the features of the second ground captured image, the features of the second aerial captured image, and the features of the second map image; and
update the first feature extractor, the second feature extractor, and the third feature extractor based on the second combined loss. ( Repeating the steps in claim 1 for the augmented data in claim 4).
Claims 7-10 , 12 and 13-16, 18 recite the method and medium in the apparatus of claim 1-4, 6, and are also rejected.
Allowable Subject Matter
Claims 5, 11 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANGENG SUN whose telephone number is (571)272-3712. The examiner can normally be reached 8am to 5pm, EST, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Randolph Vincent can be reached at 571 272 8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIANGENG SUN
Examiner
Art Unit 2661
/Jiangeng Sun/Examiner, Art Unit 2671