DETAILED ACTION
Notice of AIA Status
The present application is being examined under the AIA the first inventor to file provisions.
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 04/17/2024 and 11/07/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawing objections
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the common device and image acquisition device must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 1, 10, 12, and 20 recites limitation that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claim 1; recites the limitation, “obtaining a target image acquired by an image acquisition device” [Line 2]
Claim 10; recites the limitation, “obtaining a standard definition image acquired by a common device” [Line 2]
Claim 10; recites the limitation, “determining, by using an epipolar line search technology” [Line 5]
Claim 12; recites the limitation, “obtain a target image acquired by an image acquisition device” [Line 5]
Claim 20; recites the limitation, “obtain a target image acquired by an image acquisition device” [Line 3]
Because this claim limitation is being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claims 1, 10, 12, and 20:
(i) “image acquisition device” (Paragraph [0039]- The image acquisition device in this embodiment of this application is a to-be-positioned device, and may be, for example, a to-be-positioned vehicle or a mobile terminal. For the reference position of the image acquisition device, the server may currently only obtain position information having low positioning precision, and may identify a current position of the image acquisition device. The reference position may generally be a positioning result determined by a satellite navigation system (such as the global positioning system (GPS) or the BeiDou system) or another positioning method. The first image is an image acquired by the image acquisition device at the reference position. The first image may be, for example, an image acquired by an event data recorder mounted on a to-be-positioned vehicle, or an image acquired by a camera of a to-be-positioned mobile terminal. (wherein the image acquisition device is a camera.).).
(ii) “common device” (Paragraph [0135]- The common device herein may be a device having a positioning capability but with low positioning precision, for example, a vehicle provided with an event data recorder. In the embodiments of this application, the vehicle may upload standard definition images acquired by the event data recorder mounted on the vehicle to the server by a vehicle-mounted computer mounted on the vehicle. (wherein the common device is a camera with low positioning precision.).).
(iii) “epipolar line search technology” (Paragraph [0136]- the server may use the epipolar line search technology to determine elements existing both in both the standard definition images and the to-be-updated high definition images, and use the elements as associated elements. FIG. 15 is a schematic diagram showing determining associated elements in standard definition images and to-be-updated high definition images according to an embodiment of this application. As shown in FIG. 15, the server may perform data differential based on the standard definition images and the to-be-updated high definition images by using the epipolar line search technology, and elements 1501 may be determined as associated elements existing both in the standard definition images and the to-be-updated high definition images. (wherein the epipolar line search technology does not have sufficient structure associated with it.).).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 along with its dependent claims are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 10 recites limitations:
Claim 10; recites the limitation, “determining, by using an epipolar line search technology” [Line 5]
Claim 10 respectively invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification is devoid of adequate structure to perform the claimed functions. The specification does not provide sufficient details such that one of the ordinary skill in the art would understand which structure performed(s) the claimed function.
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 10 along with its dependent claims are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention. As described above, the disclosure does not provide adequate structure to perform the claimed function in the recited limitation Claim 10 recites limitations
Claim 10; recites the limitation, “determining, by using an epipolar line search technology” [Line 5]
The specification does not demonstrate that applicant has made an invention that achieves the claimed function because the invention is not described with sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor had possession of the claimed invention.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 12-13, and 20 are rejected under 35 U.S.C 103 as being unpatentable over Kawashima et al. (US 20230222686 A1) hereafter referenced as Kawashima in view of Kwon et al. (US 20210072405 A1) hereafter referenced as Kwon and Cai et al. (US 20230063099 A1) hereafter referenced as Cai.
Regarding claim 1, Kawashima teaches a vision positioning method (Fig. 1, Paragraph [0005]- Kawashima discloses the present disclosure proposes an information processing apparatus, an information processing method, and a program capable of improving the position/orientation estimation accuracy),
performed by a computer device (Fig. 1, Paragraph [0006]- Kawashima discloses according to the present disclosure, an information processing method in which an information process of the information processing system is executed by a computer, and a program for causing the computer to execute the information process of the information processing system, are provided.),
determining one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other (Fig. 1, Paragraph [0048]- Kawashima discloses the feature matching unit 15 extracts a plurality of corresponding point pairs from the current image IMA and the keyframe image 41 based on image information regarding the extracted keyframe image 41 (information regarding the local feature of the feature point RFP) and the image information regarding the current image IMA (information regarding the local feature of the feature point FP).);
and determining a positioning result corresponding to the image acquisition device according to position information of the reference feature point and position information of the target feature point in each of the one or more target matching feature point pairs (Fig. 2, Paragraph [0049]- Kawashima discloses the position/orientation estimation unit 16 compares the current image IMA with the information regarding the keyframe image 41 based on the calculation result reflecting the weight w of each feature point FP in the current image IMA, and estimates the position and orientation of the device that has captured the current image IMA based on the comparison result.).
Kawashima fails to explicitly teach comprising: obtaining a target image acquired by an image acquisition device at a reference position; determining, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image.
However, Kwon explicitly teaches comprising: obtaining a target image acquired by an image acquisition device at a reference position (Fig. 1, Paragraph [0052]- Kwon discloses the camera unit 110 may generate image information including images for at least one or more facility objects at the positioning time of the GNSS receiver 120 while the vehicle is driving.);
determining, from one or more pre-stored high definition images corresponding to the reference position (Fig. 1, Paragraph [0070]- Kwon discloses the storage unit 140 stores a high-definition map including information for properties and feature point spatial coordinates for each facility object. The high-definition map may mean, or include, a database storing the respective properties (or attributes) of all the facility objects and absolute spatial coordinates of the feature points of the facility objects.), a reference high definition image matching the target image (Fig. 1, Paragraph [0087]- Kwon discloses the map searching unit 171 searches for or identifies a unique object present within an error radius of the GNSS receiver from the point corresponding to the GNSS positioning information of the GNSS information on the high-definition map. Fig. 1, Paragraph [0093]- Kwon discloses thus, the precise positioning information may mean precise location information for the capturing position obtained by matching the object on the high-definition map with the facility object.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima of a vision positioning method, performed by a computer device, determining one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Kwon comprising: obtaining a target image acquired by an image acquisition device at a reference position; determining, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image.
Wherein having Kawashima’s system for position and orientation estimation wherein comprising: obtaining a target image acquired by an image acquisition device at a reference position; determining, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image.
The motivation behind the modification would have been to allow for a more accuracy with a lower computational load, since both Kawashima and Kwon are systems that use images to determine a precise position. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Kwon’s system wherein further improved accuracy while reducing computational loads. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Kwon et al. (US 20210072405 A1) Paragraph [0122].
Kawashima in view of Kwon is silent to explicitly teach positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
However, Cai explicitly teaches positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image (Fig. 1, Paragraph [0031]- Cai discloses Since the candidate reference images are collected by the high-precision measurement device, the accuracy of the positioning information of each candidate reference image is higher than the accuracy of the positioning information of the to-be-corrected image.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon of a vision positioning method, performed by a computer device, determining one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Cai positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
Wherein having Kawashima’s system for position and orientation estimation wherein positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
The motivation behind the modification would have been to allow for greater accuracy, since both Kawashima and Cai are systems that use images to determine a precise position. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Cai’s system wherein further improved accuracy. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Cai et al. (US 20230063099 A1) Paragraph [0033].
Regarding claim 2, Kawashima in view of Kwon and Cai teaches the method according to claim 1,
Kawashima further teaches wherein determining one or more target matching feature point pairs includes: constructing one or more candidate matching feature point pairs, each including a candidate target feature point in the target image and a candidate reference feature point in the reference high definition image that match each other (Fig. 2, Paragraph [0048]- Kawashima discloses the feature matching unit 15 extracts a plurality of corresponding point pairs from the current image IMA and the keyframe image 41 based on image information regarding the extracted keyframe image 41 (information regarding the local feature of the feature point RFP) and the image information regarding the current image IMA (information regarding the local feature of the feature point FP).);
performing a plurality of first-level outlier removal operations based on the one or more candidate matching feature point pairs, each of the plurality of first-level outlier removal operations including: selecting one or more basic matching feature point pairs from the one or more candidate matching feature point pairs (Fig. 2, Paragraph [0051]- Kawashima discloses the outlier removal unit 161 performs hypothesis verification based on robust estimation using the information acquired from the local feature extraction unit 11. Using hypothesis verification, the outlier removal unit 161 obtains a pair having the most coherent positional relationship between each feature point FP of the current image IMA and each feature point RFP of the keyframe image 41.);
determining a predicted pose of the image acquisition device according to the one or more basic matching feature point pairs (Fig. 1, Paragraph [0052]- Kawashima discloses in the P3P algorithm, the tentative relative position and orientation is obtained by three corresponding point pairs selected from all the corresponding point pairs.);
and determining a removal result (Fig 1, Paragraph [0051]- Kawashima discloses using hypothesis verification, the outlier removal unit 161 obtains a pair having the most coherent positional relationship between each feature point FP of the current image IMA and each feature point RFP of the keyframe image 41. The outlier removal unit 161 removes an incoherent corresponding point pair (for example, one feature point of the corresponding point pair is a point on a moving subject, or one feature point is hidden) with respect to the positional relationship obtained by hypothesis verification. This results in extraction of a plurality of inlier pairs having high reliability as corresponding points.) and a removal effect of the first-level outlier removal operation according to the predicted pose and the candidate matching feature point pairs (Fig 1, Paragraph [0050]- Kawashima discloses this results in extraction of a plurality of inlier pairs having high reliability as corresponding points. It is possible to obtain, from the positional relationship obtained by the hypothesis verification, a tentative relative position and orientation between the imaging position and orientation of the current image IMA and the imaging position and orientation of the keyframe image 41 (wherein reliability is the removal effect).);
determining, from the plurality of first-level outlier removal operations, a target first-level outlier removal operation having an optimal removal effect (Fig 1, Paragraph [0050]- Kawashima discloses this results in extraction of a plurality of inlier pairs having high reliability as corresponding points. It is possible to obtain, from the positional relationship obtained by the hypothesis verification, a tentative relative position and orientation between the imaging position and orientation of the current image IMA and the imaging position and orientation of the keyframe image 41.);
and determining the one or more target matching feature point pairs according to a removal result of the target first-level outlier removal operation (Fig 1, Paragraph [0050]- Kawashima discloses the outlier removal unit 161 extracts a plurality of inlier pairs by robust estimation from a plurality of corresponding point pairs which has been prioritized according to the weight w among a plurality of corresponding point pairs extracted by the feature matching unit 15.).
Regarding claim 12, Kawashima teaches a computer device comprising (Fig. 8, Paragraph [0006]- Kawashima discloses according to the present disclosure, an information processing method in which an information process of the information processing system is executed by a computer, and a program for causing the computer to execute the information process of the information processing system, are provided.):
one or more processors (Fig. 8, Paragraph [0006]- Kawashima discloses in the computer IPS, a central processing unit (CPU) PR, read only memory (ROM) M1, and random access memory (RAM) M2 are mutually connected by a bus BU.);
and one or more memories storing one or more computer programs that, when executed by the one or more processors (Fig. 8, Paragraph [0102]- Kawashima discloses in the computer IPS configured as described above, for example, the CPU PR loads a program stored in the storage unit ST into the RAM M2 via the input/output interface IF and the bus BU and executes the program, whereby the above-described series of processing is performed.),
determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other (Fig. 1, Paragraph [0048]- Kawashima discloses the feature matching unit 15 extracts a plurality of corresponding point pairs from the current image IMA and the keyframe image 41 based on image information regarding the extracted keyframe image 41 (information regarding the local feature of the feature point RFP) and the image information regarding the current image IMA (information regarding the local feature of the feature point FP).);
and determine a positioning result corresponding to the image acquisition device according to position information of the reference feature point and position information of the target feature point in each of the one or more target matching feature point pairs (Fig. 2, Paragraph [0049]- Kawashima discloses the position/orientation estimation unit 16 compares the current image IMA with the information regarding the keyframe image 41 based on the calculation result reflecting the weight w of each feature point FP in the current image IMA, and estimates the position and orientation of the device that has captured the current image IMA based on the comparison result.).
Kawashima fails to explicitly teach cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position; determine, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image.
However, Kwon explicitly teaches cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position (Fig. 1, Paragraph [0052]- Kwon discloses the camera unit 110 may generate image information including images for at least one or more facility objects at the positioning time of the GNSS receiver 120 while the vehicle is driving.);
determine, from one or more pre-stored high definition images corresponding to the reference position (Fig. 1, Paragraph [0070]- Kwon discloses the storage unit 140 stores a high-definition map including information for properties and feature point spatial coordinates for each facility object. The high-definition map may mean, or include, a database storing the respective properties (or attributes) of all the facility objects and absolute spatial coordinates of the feature points of the facility objects.),
a reference high definition image matching the target image (Fig. 1, Paragraph [0087]- Kwon discloses the map searching unit 171 searches for or identifies a unique object present within an error radius of the GNSS receiver from the point corresponding to the GNSS positioning information of the GNSS information on the high-definition map. Fig. 1, Paragraph [0093]- Kwon discloses thus, the precise positioning information may mean precise location information for the capturing position obtained by matching the object on the high-definition map with the facility object.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima of a computer device comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Kwon cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position; determine, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image.
Wherein having Kawashima’s system for position and orientation estimation wherein cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position; determine, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image.
The motivation behind the modification would have been to allow for a more accuracy with a lower computational load, since both Kawashima and Kwon are systems that use images to determine a precise position. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Kwon’s system wherein further improved accuracy while reducing computational loads. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Kwon et al. (US 20210072405 A1) Paragraph [0122].
Kawashima in view of Kwon is silent to explicitly teach positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image
However, Cai explicitly teaches positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image (Fig. 1, Paragraph [0031]- Cai discloses Since the candidate reference images are collected by the high-precision measurement device, the accuracy of the positioning information of each candidate reference image is higher than the accuracy of the positioning information of the to-be-corrected image.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon of a computer device comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Cai positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
Wherein having Kawashima’s system for position and orientation estimation wherein positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
The motivation behind the modification would have been to allow for greater accuracy, since both Kawashima and Cai are systems that use images to determine a precise position. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Cai’s system wherein further improved accuracy. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Cai et al. (US 20230063099 A1) Paragraph [0033].
Regarding claim 13, Kawashima in view of Kwon and Cai teaches the computer device according to claim 12,
Kawashima further teaches, wherein the one or more computer programs further cause the one or more processors to: construct one or more candidate matching feature point pairs, each including a candidate target feature point in the target image and a candidate reference feature point in the reference high definition image that match each other (Fig. 2, Paragraph [0048]- Kawashima discloses the feature matching unit 15 extracts a plurality of corresponding point pairs from the current image IMA and the keyframe image 41 based on image information regarding the extracted keyframe image 41 (information regarding the local feature of the feature point RFP) and the image information regarding the current image IMA (information regarding the local feature of the feature point FP).);
perform a plurality of first-level outlier removal operations based on the one or more candidate matching feature point pairs, each of the plurality of first-level outlier removal operations including: selecting one or more basic matching feature point pairs from the one or more candidate matching feature point pairs (Fig. 2, Paragraph [0051]- Kawashima discloses the outlier removal unit 161 performs hypothesis verification based on robust estimation using the information acquired from the local feature extraction unit 11. Using hypothesis verification, the outlier removal unit 161 obtains a pair having the most coherent positional relationship between each feature point FP of the current image IMA and each feature point RFP of the keyframe image 41.);
determining a predicted pose of the image acquisition device according to the one or more basic matching feature point pairs (Fig. 1, Paragraph [0052]- Kawashima discloses in the P3P algorithm, the tentative relative position and orientation is obtained by three corresponding point pairs selected from all the corresponding point pairs.);
and determining a removal result (Fig 1, Paragraph [0051]- Kawashima discloses using hypothesis verification, the outlier removal unit 161 obtains a pair having the most coherent positional relationship between each feature point FP of the current image IMA and each feature point RFP of the keyframe image 41. The outlier removal unit 161 removes an incoherent corresponding point pair (for example, one feature point of the corresponding point pair is a point on a moving subject, or one feature point is hidden) with respect to the positional relationship obtained by hypothesis verification. This results in extraction of a plurality of inlier pairs having high reliability as corresponding points.) and a removal effect of the first-level outlier removal operation according to the predicted pose and the candidate matching feature point pairs (Fig 1, Paragraph [0050]- Kawashima discloses this results in extraction of a plurality of inlier pairs having high reliability as corresponding points. It is possible to obtain, from the positional relationship obtained by the hypothesis verification, a tentative relative position and orientation between the imaging position and orientation of the current image IMA and the imaging position and orientation of the keyframe image 41 (wherein reliability is the removal effect).);
determine, from the plurality of first-level outlier removal operations, a target first-level outlier removal operation having an optimal removal effect (Fig 1, Paragraph [0050]- Kawashima discloses this results in extraction of a plurality of inlier pairs having high reliability as corresponding points. It is possible to obtain, from the positional relationship obtained by the hypothesis verification, a tentative relative position and orientation between the imaging position and orientation of the current image IMA and the imaging position and orientation of the keyframe image 41.);
and determine the one or more target matching feature point pairs according to a removal result of the target first-level outlier removal operation (Fig 1, Paragraph [0050]- Kawashima discloses the outlier removal unit 161 extracts a plurality of inlier pairs by robust estimation from a plurality of corresponding point pairs which has been prioritized according to the weight w among a plurality of corresponding point pairs extracted by the feature matching unit 15.).
Regarding claim 20, Kawashima teaches a non-transitory computer-readable storage medium storing one or more computer programs that (Fig. 8, Paragraph [0104]- Kawashima discloses in the computer IPS, the program can be installed in the storage unit ST via the input/output interface IF by using the removable recording medium RM inserted to the drive DU. Furthermore, the program can be received by the communication unit CU via a wired or wireless transmission medium and installed in the storage unit ST. In addition, the program can be preinstalled in the ROM M1 or the storage unit ST.),
determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other (Fig. 1, Paragraph [0048]- Kawashima discloses the feature matching unit 15 extracts a plurality of corresponding point pairs from the current image IMA and the keyframe image 41 based on image information regarding the extracted keyframe image 41 (information regarding the local feature of the feature point RFP) and the image information regarding the current image IMA (information regarding the local feature of the feature point FP).);
and determine a positioning result corresponding to the image acquisition device according to position information of the reference feature point and position information of the target feature point in each of the one or more target matching feature point pairs (Fig. 2, Paragraph [0049]- Kawashima discloses the position/orientation estimation unit 16 compares the current image IMA with the information regarding the keyframe image 41 based on the calculation result reflecting the weight w of each feature point FP in the current image IMA, and estimates the position and orientation of the device that has captured the current image IMA based on the comparison result.).
Kawashima fails to explicitly teach when executed by one or more processors, cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position; determine, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image.
However, Kwon explicitly teaches when executed by one or more processors, cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position (Fig. 1, Paragraph [0052]- Kwon discloses the camera unit 110 may generate image information including images for at least one or more facility objects at the positioning time of the GNSS receiver 120 while the vehicle is driving.);
when executed by one or more processors, cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position; determine, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image (Fig. 1, Paragraph [0093]- Kwon discloses thus, the precise positioning information may mean precise location information for the capturing position obtained by matching the object on the high-definition map with the facility object.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima of a non-transitory computer-readable storage medium storing one or more computer programs that, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Kwon when executed by one or more processors, cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position; determine, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image
Wherein having Kawashima’s system for position and orientation estimation wherein when executed by one or more processors, cause the one or more processors to: obtain a target image acquired by an image acquisition device at a reference position; determine, from one or more pre-stored high definition images corresponding to the reference position, a reference high definition image matching the target image
The motivation behind the modification would have been to allow for a more accuracy with a lower computational load, since both Kawashima and Kwon are systems that use images to determine a precise position. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Kwon’s system wherein further improved accuracy while reducing computational loads. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Kwon et al. (US 20210072405 A1) Paragraph [0122].
Kawashima in view of Kwon is silent to explicitly teach positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
However, Cai explicitly teaches positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image (Fig. 1, Paragraph [0031]- Cai discloses Since the candidate reference images are collected by the high-precision measurement device, the accuracy of the positioning information of each candidate reference image is higher than the accuracy of the positioning information of the to-be-corrected image.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon of a computer device comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Cai positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
Wherein having Kawashima’s system for position and orientation estimation wherein positioning precision of each of the one or more pre-stored high definition images being higher than positioning precision of the target image.
The motivation behind the modification would have been to allow for greater accuracy, since both Kawashima and Cai are systems that use images to determine a precise position. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Cai’s system wherein further improved accuracy. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Cai et al. (US 20230063099 A1) Paragraph [0033].
Claims 3 and 14 are rejected under 35 U.S.C 103 as being unpatentable over Kawashima et al. (US 20230222686 A1) hereafter referenced as Kawashima in view of Kwon et al. (US 20210072405 A1) hereafter referenced as Kwon, Cai et al. (US 20230063099 A1) hereafter referenced as Cai, and Jagadeesan et al. (US 20220051431 A1) hereafter referenced as Jagadeesan.
Regarding claim 3, Kawashima in view of Kwon and Cai teaches the method according to claim 2,
Kawashima further teaches wherein determining the one or more target matching feature point pairs according to the removal result of the target first-level outlier removal operation includes: determining one or more of the one or more candidate matching feature point pairs retained after the target first-level outlier removal operation as one or more reference matching feature point pairs (Fig. 1, Paragraph [0055]- Kawashima discloses the orientation calculation unit 162 calculates the position and orientation of the device using a regression analysis model in which a contribution degree of each of the inlier pairs has been corrected based on the weight w of each of the feature points FP of the current image IMA.);
performing a plurality of second-level outlier removal operations based on the one or more reference matching feature point pairs, each of the plurality of second-level outlier removal operations including: determining, according to an assumed rotation parameter, an assumed translation parameter, and three-dimensional position information of one or more reference feature points in the one or more reference matching feature point pairs, two-dimensional position information of the one or more reference feature points (Fig. 1, Paragraph [0057]- Kawashima discloses in Formula (9), ΔT represents the relative position and orientation. x.sub.j represents a three-dimensional coordinate with respect to the feature u.sub.j. proj represents a function of projecting a three-dimensional coordinate point onto two-dimensional coordinates of a camera screen);
Kawashima in view of Kwon and Cai fails to explicitly teach and determining a removal result and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs; determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect; and determining the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation.
However, Jagadeesan explicitly teaches and determining a removal result (Fig. 1, Paragraph [0111]- Jagadeesan discloses this algorithm takes a random 3D-2D correspondence as the control point. To address outliers, a soft weight assignment method is provided that uses reprojection errors to distinguish inliers and outliers, and integrate it into the algorithm.) and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs (Fig. 4A Paragraph [0158]- Jagadeesan discloses this termination condition is not based on the comparison of parameters of adjacent iterations because in R1PPnP, the dynamically updated weights w.sub.i may make the convergence process complex, especially when point o is an outlier. With an outlier as the control point, ∥R.sup.(k)−R.sup.(k−1)∥ may take many iterations to converge to zero, which is slow. With the change of detected number of inliers in a larger window size, the termination decision can be more robust and efficient.);
determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect (Fig. 4A Paragraph [0138]- Jagadeesan discloses FIG. 4A demonstrates a process that is approaching the correct global optimal results. Beginning with points p.sup.(k), the algorithm projects p.sup.(k) to their related lines of sight and obtains points q.sup.(k+1). Then, according to q.sup.(k+1), the algorithm updates the rotation R and scale factor μ to generate points p.sup.(k+1). In this process, the rotation and scale factor related to p.sup.(k+1) are closer to the truth compared to that related to p.sup.(k), and finally the algorithm will reach the correct solution.);
and determining the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation (Fig. 16B, Paragraph [0185]- Jagadeesan discloses we integrate a soft re-weighting method into an iterative PnP process to distinguish inliers and outliers, and employ the 1-point RANSAC scheme for selecting the control point.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon and Cai of a vision positioning method, performed by a computer device, determining one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Jagadeesan determining a removal result and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs; determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect; and determining the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation.
Wherein having Kawashima’s system for position and orientation estimation wherein determining a removal result and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs; determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect; and determining the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation.
The motivation behind the modification would have been to allow for a more robust system, since both Kawashima and Jagadeesan are systems that determine position and orientation using images. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Jagadeesan’s system wherein further improved robustness of the SLAM system. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Jagadeesan et al. (US 20220051431 A1) Paragraph [0264].
Regarding claim 14, Kawashima in view of Kwon and Cai teaches the computer device according to claim 13,
wherein the one or more computer programs further cause the one or more processors to: determine one or more of the one or more candidate matching feature point pairs retained after the target first-level outlier removal operation as one or more reference matching feature point pairs (Fig. 1, Paragraph [0055]- Kawashima discloses the orientation calculation unit 162 calculates the position and orientation of the device using a regression analysis model in which a contribution degree of each of the inlier pairs has been corrected based on the weight w of each of the feature points FP of the current image IMA.);
perform a plurality of second-level outlier removal operations based on the one or more reference matching feature point pairs, each of the plurality of second-level outlier removal operations including: determining, according to an assumed rotation parameter, an assumed translation parameter, and three-dimensional position information of one or more reference feature points in the one or more reference matching feature point pairs, two-dimensional position information of the one or more reference feature points (Fig. 1, Paragraph [0057]- Kawashima discloses in Formula (9), ΔT represents the relative position and orientation. x.sub.j represents a three-dimensional coordinate with respect to the feature u.sub.j. proj represents a function of projecting a three-dimensional coordinate point onto two-dimensional coordinates of a camera screen);
Kawashima in view of Kwon and Cai fails to explicitly teach and determining a removal result and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs; determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect; and determine the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation.
However, Jagadeesan explicitly teaches and determining a removal result (Fig. 1, Paragraph [0111]- Jagadeesan discloses this algorithm takes a random 3D-2D correspondence as the control point. To address outliers, a soft weight assignment method is provided that uses reprojection errors to distinguish inliers and outliers, and integrate it into the algorithm.) and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs (Fig. 4A Paragraph [0158]- Jagadeesan discloses this termination condition is not based on the comparison of parameters of adjacent iterations because in R1PPnP, the dynamically updated weights w.sub.i may make the convergence process complex, especially when point o is an outlier. With an outlier as the control point, ∥R.sup.(k)−R.sup.(k−1)∥ may take many iterations to converge to zero, which is slow. With the change of detected number of inliers in a larger window size, the termination decision can be more robust and efficient.);
determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect (Fig. 4A Paragraph [0138]- Jagadeesan discloses FIG. 4A demonstrates a process that is approaching the correct global optimal results. Beginning with points p.sup.(k), the algorithm projects p.sup.(k) to their related lines of sight and obtains points q.sup.(k+1). Then, according to q.sup.(k+1), the algorithm updates the rotation R and scale factor μ to generate points p.sup.(k+1). In this process, the rotation and scale factor related to p.sup.(k+1) are closer to the truth compared to that related to p.sup.(k), and finally the algorithm will reach the correct solution.);
and determine the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation (Fig. 16B, Paragraph [0185]- Jagadeesan discloses we integrate a soft re-weighting method into an iterative PnP process to distinguish inliers and outliers, and employ the 1-point RANSAC scheme for selecting the control point.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon and Cai of a computer device comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Jagadeesan determining a removal result and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs; determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect; and determining the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation.
Wherein having Kawashima’s system for position and orientation estimation wherein determining a removal result and a removal effect of the second-level outlier removal operation according to the two-dimensional position information of the one or more reference feature points and two-dimensional position information of one or more target feature points in the one or more reference matching feature point pairs; determining, from the plurality of second-level outlier removal operations, a target second-level outlier removal operation having an optimal removal effect; and determining the one or more target matching feature point pairs according to a removal result of the target second-level outlier removal operation.
The motivation behind the modification would have been to allow for a more robust system, since both Kawashima and Jagadeesan are systems that determine position and orientation using images. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Jagadeesan’s system wherein further improved robustness of the SLAM system. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Jagadeesan et al. (US 20220051431 A1) Paragraph [0264].
Claims 4 and 15 are rejected under 35 U.S.C 103 as being unpatentable over Kawashima et al. (US 20230222686 A1) hereafter referenced as Kawashima in view of Kwon et al. (US 20210072405 A1) hereafter referenced as Kwon, Cai et al. (US 20230063099 A1) hereafter referenced as Cai, and Bao et al. (US 20200043189 A1) hereafter referenced as Bao.
Regarding claim 4, Kawashima in view of Kwon and Cai teaches the method according to claim 1,
Kawashima in view of Kwon and Cai fails to explicitly teach wherein determining the positioning result includes: determining a projection error according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs, a camera intrinsic parameter of the image acquisition device, and an attitude parameter and a position parameter of the image acquisition device; optimizing the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter; and determining the positioning result according to the optimized attitude parameter and the optimized position parameter.
However, Bao explicitly teaches wherein determining the positioning result includes: determining a projection error (Fig. 1, Paragraph [0017]- Bao discloses after the relative pose T.sub.(i-1).fwdarw.i and a 3D-2D feature matching set χ={(X.sub.j, x.sub.j)} are determined, the camera pose C.sub.i is estimated by minimizing a relative error and a reprojection error:)
PNG
media_image1.png
100
456
media_image1.png
Greyscale
Bao Reprojection error formula
according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs (Fig. 1, Paragraph [0017]- Bao discloses after the relative pose T.sub.(i-1).fwdarw.i and a 3D-2D feature matching set χ={(X.sub.j, x.sub.j)} are determined, the camera pose C.sub.i is estimated by minimizing a relative error and a reprojection error:), a camera intrinsic parameter of the image acquisition device (Fig. 1, Paragraph [0018]- Bao discloses K is a known and constant camera intrinsics.), and an attitude parameter and a position parameter of the image acquisition device (Fig. 1, Paragraph [0024]- Bao discloses defining current camera motion parameter as C={C.sub.1, C.sub.2, . . . , C.sub.n.sub.c}, C.sub.i∈SE (3), where n.sub.c is the number of the cameras, defining current three-dimensional point position as χ={X.sub.1, X.sub.2, . . . X.sub.n.sub.p},X.sub.j∈R.sup.3, where n.sub.p is the number of three-dimensional points, defining the set of three-dimensional points visible to the camera i as V.sub.i.sup.c, defining the set of cameras visible to the three-dimensional point j as V.sub.i.sup.p, and iterating and optimizing with Gauss-Newton algorithm (wherein SE(3) includes rotation and translation which is seen as attitude and position respectively));
optimizing the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter (Fig. 1, Paragraph [0017]- Bao discloses after the relative pose T.sub.(i-1).fwdarw.i and a 3D-2D feature matching set χ={(X.sub.j, x.sub.j)} are determined, the camera pose C.sub.i is estimated by minimizing a relative error and a reprojection error:);
and determining the positioning result according to the optimized attitude parameter and the optimized position parameter (Fig. 1, Paragraph [0075]- Bao discloses the above Schur complement equation is solved by using a preconditioned conjugate gradient algorithm, so as to obtain the camera increment {tilde over (c)}, and for each camera i, if |{tilde over (c)}.sub.i|>ε.sub.c, then Ĉ.sub.i is updated with {tilde over (C)}.sub.i⊕{tilde over (c)}.sub.i.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon and Cai of a vision positioning method, performed by a computer device, determining one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Bao wherein determining the positioning result includes: determining a projection error according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs, a camera intrinsic parameter of the image acquisition device, and an attitude parameter and a position parameter of the image acquisition device; optimizing the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter; and determining the positioning result according to the optimized attitude parameter and the optimized position parameter.
Wherein having Kawashima’s system for position and orientation estimation wherein determining the positioning result includes: determining a projection error according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs, a camera intrinsic parameter of the image acquisition device, and an attitude parameter and a position parameter of the image acquisition device; optimizing the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter; and determining the positioning result according to the optimized attitude parameter and the optimized position parameter.
The motivation behind the modification would have been to allow for reducing errors, since both Kawashima and Bao are systems that determine position using images. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Bao’s system wherein reduced the cumulated errors. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Bao et al. (US 20200043189 A1) Paragraph [0003].
Regarding claim 15, Kawashima in view of Kwon and Cai teaches the computer device according to claim 12,
Kawashima in view of Kwon and Cai fails to explicitly teach wherein the one or more computer programs further cause the one or more processors to: determine a projection error according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs, a camera intrinsic parameter of the image acquisition device, and an attitude parameter and a position parameter of the image acquisition device; optimize the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter; and determine the positioning result according to the optimized attitude parameter and the optimized position parameter.
However, Bao explicitly teaches wherein the one or more computer programs further cause the one or more processors to: determine a projection error (Fig. 1, Paragraph [0017]- Bao discloses after the relative pose T.sub.(i-1).fwdarw.i and a 3D-2D feature matching set χ={(X.sub.j, x.sub.j)} are determined, the camera pose C.sub.i is estimated by minimizing a relative error and a reprojection error:)
PNG
media_image1.png
100
456
media_image1.png
Greyscale
Bao Reprojection error formula
according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs (Fig. 1, Paragraph [0017]- Bao discloses after the relative pose T.sub.(i-1).fwdarw.i and a 3D-2D feature matching set χ={(X.sub.j, x.sub.j)} are determined, the camera pose C.sub.i is estimated by minimizing a relative error and a reprojection error:), a camera intrinsic parameter of the image acquisition device (Fig. 1, Paragraph [0018]- Bao discloses K is a known and constant camera intrinsics.), and an attitude parameter and a position parameter of the image acquisition device (Fig. 1, Paragraph [0024]- Bao discloses defining current camera motion parameter as C={C.sub.1, C.sub.2, . . . , C.sub.n.sub.c}, C.sub.i∈SE (3), where n.sub.c is the number of the cameras, defining current three-dimensional point position as χ={X.sub.1, X.sub.2, . . . X.sub.n.sub.p},X.sub.j∈R.sup.3, where n.sub.p is the number of three-dimensional points, defining the set of three-dimensional points visible to the camera i as V.sub.i.sup.c, defining the set of cameras visible to the three-dimensional point j as V.sub.i.sup.p, and iterating and optimizing with Gauss-Newton algorithm (wherein SE(3) includes rotation and translation which is seen as attitude and position respectively));
optimize the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter (Fig. 1, Paragraph [0017]- Bao discloses after the relative pose T.sub.(i-1).fwdarw.i and a 3D-2D feature matching set χ={(X.sub.j, x.sub.j)} are determined, the camera pose C.sub.i is estimated by minimizing a relative error and a reprojection error:);
and determine the positioning result according to the optimized attitude parameter and the optimized position parameter (Fig. 1, Paragraph [0075]- Bao discloses the above Schur complement equation is solved by using a preconditioned conjugate gradient algorithm, so as to obtain the camera increment {tilde over (c)}, and for each camera i, if |{tilde over (c)}.sub.i|>ε.sub.c, then Ĉ.sub.i is updated with {tilde over (C)}.sub.i⊕{tilde over (c)}.sub.i.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon and Cai of a computer device comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Bao wherein the one or more computer programs further cause the one or more processors to: determine a projection error according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs, a camera intrinsic parameter of the image acquisition device, and an attitude parameter and a position parameter of the image acquisition device; optimize the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter; and determine the positioning result according to the optimized attitude parameter and the optimized position parameter.
Wherein having Kawashima’s system for position and orientation estimation wherein the one or more computer programs further cause the one or more processors to: determine a projection error according to three-dimensional position information of the reference feature point and two-dimensional position information of the target feature point in each of the one or more target matching feature point pairs, a camera intrinsic parameter of the image acquisition device, and an attitude parameter and a position parameter of the image acquisition device; optimize the attitude parameter and the position parameter of the image acquisition device by minimizing the projection error, to obtain an optimized attitude parameter and an optimized position parameter; and determine the positioning result according to the optimized attitude parameter and the optimized position parameter.
The motivation behind the modification would have been to allow for reducing errors, since both Kawashima and Bao are systems that determine position using images. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Bao’s system wherein reduced the cumulated errors. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Bao et al. (US 20200043189 A1) Paragraph [0003].
Claims 5 and 16 are rejected under 35 U.S.C 103 as being unpatentable over Kawashima et al. (US 20230222686 A1) hereafter referenced as Kawashima in view of Kwon et al. (US 20210072405 A1) hereafter referenced as Kwon, Cai et al. (US 20230063099 A1) hereafter referenced as Cai, Otsuka et al. (US 20220254035 A1) hereafter referenced as Otsuka, and Lasang et al. (US 20200278450 A1) hereafter referenced as Lasang.
Regarding claim 5, Kawashima in view of Kwon and Cai teaches the method according to claim 1,
Kawashima further teaches wherein the one or more high definition images are pre-stored in a visual fingerprint database that is constructed by: obtaining candidate high definition images acquired respectively by a plurality of cameras rigidly connected to a high definition device (Fig. 1 Paragraph [0039]- Kawashima discloses the environmental map MP is generated using a plurality of keyframe images 41 captured in the past. In the environmental map MP, image information (keyframe information) regarding the keyframe image 41 is registered in association with the position and orientation at imaging of the keyframe image 41.);
detecting feature points in the candidate high definition images (Fig. 1, Paragraph [0029]- Kawashima discloses the keyframe information includes, for example, information regarding positions of a plurality of feature points (registered feature points) RFP included in the keyframe image 41, local features (registered local features) in each of the feature points RFP, and image feature amounts (registered image feature amounts) calculated based on the local features in each of the feature points RFP.);
performing an outlier removal operation based on the matching feature point pairs to obtain inlier matching feature point pairs (Fig. 1, Paragraph [0119-120]- Kawashima discloses the information processing apparatus according to (2) or (3), wherein the position/orientation estimation unit includes an outlier removal unit that extracts a plurality of inlier pairs by robust estimation from a plurality of pairs of feature points prioritized according to the weight among a plurality of pairs of feature points corresponding to each other in the photographic image and the keyframe image.);
Kawashima in view of Kwon and Cai fails to explicitly teach performing intra-frame matching and inter-frame matching based on the feature points in the candidate high definition images to determine matching feature point pairs.
However, Otsuka explicitly teaches performing intra-frame matching (Fig. 1, Paragraph [0069]- Otsuka discloses using such pattern matching, the feature point comparison section 178 performs one of or both the association of feature points in a plurality of images captured at the same time by a plurality of cameras and the association of feature points in a plurality of frames of a moving image captured by the same camera.) and inter-frame matching (Fig. 9 Paragraph [0100]- Otsuka discloses the feature point comparison section 178 acquires pieces of correspondence information (correspondence information in the space direction) P1, P2, P3, . . . as to feature points in different frames captured at the same time by the plurality of cameras, and pieces of correspondence information (correspondence information in the time direction) M1, M2, . . . as to feature points in a plurality of frames captured at different times by the same camera.) based on the feature points in the candidate high definition images to determine matching feature point pairs (Fig. 1, Paragraph [0053]- Otsuka discloses a feature point comparison section 178 that acquires the correspondence relations between the feature points in the plurality of captured images, and a space information acquisition section 180 that acquires the real space information on the basis of the correspondence relations between the feature points.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon and Cai of a vision positioning method, performed by a computer device, determining one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Otsuka performing intra-frame matching and inter-frame matching based on the feature points in the candidate high definition images to determine matching feature point pairs.
Wherein having Kawashima’s system for position and orientation estimation wherein performing intra-frame matching and inter-frame matching based on the feature points in the candidate high definition images to determine matching feature point pairs.
The motivation behind the modification would have been to allow for high speed and high accuracy on obtaining information, since both Kawashima and Otsuka are systems that determine position and orientation using images. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Otsuka’s system wherein improved the system by allowing for high speed and high accuracy. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Otsuka et al. (US 20220254035 A1) Paragraph [0130]. Kawashima in view of Kwon, Cai, and Otsuka fails to explicitly teach performing triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong, to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs, the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image.
However, Lasang explicitly teaches and performing triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong (Fig. 1, Paragraph [0100]- Lasang discloses triangulation, using the position of each of the two feature points forming the pair in the two-dimensional image, and the position and orientation of camera 210 when each of the two two-dimensional images, in which the pair of feature points have been obtained, are imaged (S15).), to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs (Fig. 1, Paragraph [0100]- Lasang discloses each of the pairs of feature points matched in feature matching module 122, triangulation module 124 next calculates a three-dimensional position associated with the position of the matched feature points in the pair of two-dimensional images), the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image (Fig. 1, Paragraph [0100]- Lasang discloses the position and orientation of camera 210 used here is the position and orientation of camera 210 when each of the two-dimensional images, in which the pair of feature points has been obtained, is imaged.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon, Cai, and Otsuka of a vision positioning method, performed by a computer device, determining one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Lasang performing triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong, to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs, the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image.
Wherein having Kawashima’s system for position and orientation estimation wherein performing triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong, to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs, the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image.
The motivation behind the modification would have been to allow for ease of performing position estimation, since both Kawashima and Lasang are systems that determine position of objects. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Lasang’s system wherein improved the ease of performing position estimation. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Lasang et al. (US 20200278450 A1) Paragraph [0031].
Regarding claim 16, Kawashima in view of Kwon and Cai teaches the computer device according to claim 12,
Kawashima further teaches wherein the one or more computer programs further cause the one or more processors to: obtain candidate high definition images acquired respectively by a plurality of cameras rigidly connected to a high definition device (Fig. 1 Paragraph [0039]- Kawashima discloses the environmental map MP is generated using a plurality of keyframe images 41 captured in the past. In the environmental map MP, image information (keyframe information) regarding the keyframe image 41 is registered in association with the position and orientation at imaging of the keyframe image 41.);
detect feature points in the candidate high definition images (Fig. 1, Paragraph [0029]- Kawashima discloses the keyframe information includes, for example, information regarding positions of a plurality of feature points (registered feature points) RFP included in the keyframe image 41, local features (registered local features) in each of the feature points RFP, and image feature amounts (registered image feature amounts) calculated based on the local features in each of the feature points RFP.);
perform an outlier removal operation based on the matching feature point pairs to obtain inlier matching feature point pairs (Fig. 1, Paragraph [0119-120]- Kawashima discloses the information processing apparatus according to (2) or (3), wherein the position/orientation estimation unit includes an outlier removal unit that extracts a plurality of inlier pairs by robust estimation from a plurality of pairs of feature points prioritized according to the weight among a plurality of pairs of feature points corresponding to each other in the photographic image and the keyframe image.);
Kawashima in view of Kwon and Cai fails to explicitly teach perform intra-frame matching and inter-frame matching based on the feature points in the candidate high definition images to determine matching feature point pairs.
However, Otsuka explicitly teaches perform intra-frame (Fig. 1, Paragraph [0069]- Otsuka discloses using such pattern matching, the feature point comparison section 178 performs one of or both the association of feature points in a plurality of images captured at the same time by a plurality of cameras and the association of feature points in a plurality of frames of a moving image captured by the same camera.) and inter-frame matching (Fig. 9 Paragraph [0100]- the feature point comparison section 178 acquires pieces of correspondence information (correspondence information in the space direction) P1, P2, P3, . . . as to feature points in different frames captured at the same time by the plurality of cameras, and pieces of correspondence information (correspondence information in the time direction) M1, M2, . . . as to feature points in a plurality of frames captured at different times by the same camera.) based on the feature points in the candidate high definition images to determine matching feature point pairs (Fig. 1, Paragraph [0053]- Otsuka discloses a feature point comparison section 178 that acquires the correspondence relations between the feature points in the plurality of captured images, and a space information acquisition section 180 that acquires the real space information on the basis of the correspondence relations between the feature points.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon and Cai a computer device comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Otsuka perform intra-frame matching and inter-frame matching based on the feature points in the candidate high definition images to determine matching feature point pairs.
Wherein having Kawashima’s system for position and orientation estimation wherein perform intra-frame matching and inter-frame matching based on the feature points in the candidate high definition images to determine matching feature point pairs.
The motivation behind the modification would have been to allow for high speed and high accuracy on obtaining information, since both Kawashima and Otsuka are systems that determine position and orientation using images. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Otsuka’s system wherein improved the system by allowing for high speed and high accuracy. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Otsuka et al. (US 20220254035 A1) Paragraph [0130].
Kawashima in view of Kwon, Cai, and Otsuka fails to explicitly teach perform triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong, to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs, the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image.
However, Lasang explicitly teaches and perform triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong (Fig. 1, Paragraph [0100]- Lasang discloses triangulation, using the position of each of the two feature points forming the pair in the two-dimensional image, and the position and orientation of camera 210 when each of the two two-dimensional images, in which the pair of feature points have been obtained, are imaged (S15).), to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs (Fig. 1, Paragraph [0100]- Lasang discloses each of the pairs of feature points matched in feature matching module 122, triangulation module 124 next calculates a three-dimensional position associated with the position of the matched feature points in the pair of two-dimensional images), the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image (Fig. 1, Paragraph [0100]- Lasang discloses the position and orientation of camera 210 used here is the position and orientation of camera 210 when each of the two-dimensional images, in which the pair of feature points has been obtained, is imaged.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kawashima in view of Kwon, Cai, and Otsuka of a computer device comprising: one or more processors; and one or more memories storing one or more computer programs that, when executed by the one or more processors, determine one or more target matching feature point pairs, each including a target feature point in the target image and a reference feature point in the reference high definition image that match each other with the teachings of Lasang perform triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong, to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs, the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image.
Wherein having Kawashima’s system for position and orientation estimation wherein perform triangulation calculation according to the inlier matching feature point pairs and a pose corresponding to a candidate high definition image to which feature points in the inlier matching feature point pairs belong, to determine three-dimensional position information in a world coordinate system of the feature points in the inlier matching feature point pairs, the pose being a pose of one of the cameras acquiring the candidate high definition image during acquisition of the candidate high definition image.
The motivation behind the modification would have been to allow for ease of performing position estimation, since both Kawashima and Lasang are systems that determine position of objects. Wherein Kawashima’s system wherein improved accuracy of position and orientation estimation, while Lasang’s system wherein improved the ease of performing position estimation. Please see Kawashima et al. (US 20230222686 A1), Paragraph [0005] and Lasang et al. (US 20200278450 A1) Paragraph [0031].
Allowable Subject Matter
Claims 6, 8-10, 17, and 19 and dependent claims 7, 11, and 18 are therefrom objected to as being dependent upon rejected base claim, claim 1, respectively but would be allowable if rewritten in independent form including all of the limitations of the base claims and any intervening claims and to overcome any 112(a) and 112(b) rejections.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 6, the prior arts fail to explicitly teach, detecting, according to the generic camera model essential matrix and light representations corresponding to feature points in the matching feature point pair, whether the matching feature point pair is one of the inlier matching feature point pairs, as claimed in claim 6.
Regarding claim 8, the prior arts fail to explicitly teach before performing the intra-frame matching and the inter-frame matching based on the feature points in the candidate high definition images to determine the matching feature point pairs: for each candidate high definition image, determining a texture repetition element and a dynamic obstacle element in the candidate high definition image using a segmentation model, and masking the texture repetition element and the dynamic obstacle element in the candidate high definition image to obtain a masked candidate high definition image; wherein performing the intra-frame matching and the inter-frame matching based on the feature points in the candidate high definition images to determine the matching feature point pairs includes: performing the intra-frame matching and the inter-frame matching based on feature points in the masked candidate high definition images, to determine the matching feature point pairs, as claimed in claim 8.
Regarding claim 9, the prior arts fail to explicitly teach after every preset period of time, eliminating, based on a carrier-phase differential technology, a cumulative error of a pose of the high definition device determined using pre-integration, as claimed in claim 9.
Regarding claim 10, the prior arts fail to explicitly teach determining, by using an epipolar line search technology according to the standard definition image and the high definition image, associated elements existing in both the standard definition image and the target high definition image; and adjusting update time of three-dimensional position information of a feature point corresponding to the associated elements in the visual fingerprint database to acquisition time of the standard definition image, as claimed in claim 10.
Regarding claim 17, the prior arts fail to explicitly teach detect, according to the generic camera model essential matrix and light representations corresponding to feature points in the matching feature point pair, whether the matching feature point pair is one of the inlier matching feature point pairs, as claimed in claim 17.
Regarding claim 19, the prior arts fail to explicitly teach before performing the intra-frame matching and the inter-frame matching based on the feature points in the candidate high definition images to determine the matching feature point pairs: for each candidate high definition image, determine a texture repetition element and a dynamic obstacle element in the candidate high definition image using a segmentation model, and masking the texture repetition element and the dynamic obstacle element in the candidate high definition image to obtain a masked candidate high definition image; and perform the intra-frame matching and the inter-frame matching based on feature points in the masked candidate high definition images, to determine the matching feature point pairs, as claimed in claim 19.
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered
pertinent to applicant`s disclosure.
ZHANG et al. (US 20210264198 A1)- A positioning method and apparatus are provided. A specific embodiment of the method can include: acquiring description information of an object in an image to be positioned; searching in a database, based on the description information of the object in the image to be positioned, for preset images with description information matching the description information of the object in the image to be positioned, to obtain a set of preset images; matching the image to be positioned with the preset images in the set of preset images, to obtain an image matching the image to be positioned; and finally, determining a position of the image to be positioned based on preset position information corresponding to the image matching the image to be positioned......................Please see Fig. 1. Abstract.
LIN et al. (US 20200327695 A1)- This application discloses a repositioning method and apparatus in a camera pose tracking process, a device, and a storage medium, belonging to the field of augmented reality (AR). The method includes: obtaining a current image acquired after an i.sup.th anchor image in a plurality of anchor images; obtaining an initial feature point and an initial pose parameter in the first anchor image in the plurality of anchor images in a case that the current image satisfies a repositioning condition; performing feature point tracking on the current image relative to the first anchor image, to obtain a plurality of matching feature point pairs; filtering the plurality of matching feature point pairs according to a constraint condition, to obtain a filtered matching feature point pair; calculating a pose change amount of a camera from the initial pose parameter to a target pose parameter according to the filtered matching feature point pair; and performing repositioning according to the initial pose parameter and the pose change amount to obtain the target pose parameter of the camera.....................Please see Fig. 1. Abstract.
KOMURO et al. (US 20230039143 A1)- An own-position estimating device for estimating an own-position of a moving body by matching a feature extracted from an acquired image with a database in which position information and the feature are associated with each other in advance, includes an estimating unit estimating the own-position of the moving body by matching the feature extracted by the extracting unit with the database, and a determination threshold value adjusting unit adjusting a determination threshold value for extracting the feature, in which the determination threshold value adjusting unit acquires the database in a state in which the determination threshold value is adjusted, and adjusts the determination threshold value on the basis of the determination threshold value linked to each of the position information items in the database, and the extracting unit extracts the feature from the image by using the determination threshold value adjusted by the determination threshold value adjusting unit.......................Please see Fig. 1. Abstract.
Choi et al. (US 20210190512 A1)- According to an embodiment, a system includes at least one change detecting device that includes a map information storage unit receiving a high-definition map including a property of each road facility object and spatial coordinates of a feature point from a map updating server and storing the received high-definition map, an object coordinates obtaining unit recognizing at least one road facility object from the road image and obtaining a property of the recognized object and spatial coordinates of a feature point. and a changed object detecting unit comparing the property of the recognized object and the spatial coordinates of the feature point with the high-definition map and, if a change object is detected, transmitting object change information including a property and feature point spatial coordinates of the change object to the map updating server.........................Please see Fig. 1. Abstract.
Chui et al. (US 20210125402 A1)- Techniques for generating an image are disclosed. In some embodiments, a received input image is transformed to generate an output image using a machine learning based framework that is trained on a constrained set of images. The generated output image comprises an attribute learned by the machine learning based framework from the set of images..........................Please see Fig. 1. Abstract.
Cheng et al. (US 20230095500 A1)- Described is a method of calibrating a camera. The method comprises obtaining geographical coordinates of a selected physical point location within an image view of the camera and measuring an angle between an x-axis of a real-world coordinate system passing through said selected point location with respect to true north. The method includes using said obtained geographical coordinates, said measured angle, and projection data derived from characteristics of the camera to derive modified projection data for transforming a two-dimensional pixel coordinate system of the camera image view into a three-dimensional geographical coordinate system for point locations within the image view of the camera............................Please see Fig. 1. Abstract.
LAWLOR et al. (US 20220198700 A1)- A method, apparatus and computer program product are provided for establishing correspondences between images using through generation of a translation between the different perspectives. Methods may include: receiving first sensor data from a first image sensor, where the first sensor data includes a first image of an environment captured from a first perspective; receiving second sensor data from a second image sensor of a second image of the environment captured from a second perspective; identifying image correspondence points between the first sensor data and the second sensor data; computing pairwise vectors between corresponding pairs of first projected points of the first sensor data and second projected points of the second sensor data; clustering the pairwise vectors according to magnitude and orientation; and generating a translation vector from clusters of pairwise vectors, where the translation vector represents a shift of the second image sensor data to correspond to ground truth data............................Please see Fig. 1. Abstract.
Xu et al. (US 20190206116 A1)- A method for simultaneous localization and mapping. The method includes the step of detecting two-dimensional (2D) feature points from a current frame captured by a camera; matching the 2D feature points from the current frame directly to three-dimensional (3D) map points in a 3D map, so as to obtain correspondence between the 2D feature points and the 3D map points; and computing a current pose of the camera based on the obtained correspondence.............................Please see Fig. 1. Abstract.
ZIEGLER et al. (US 20140037189 A1)- A system, apparatus and method for determining a 3-D point cloud is presented. First a processor detects feature points in the first 2-D image and feature points in the second 2-D image and so on. This set of feature points is first matched across images using an efficient transitive matching scheme. These matches are pruned to remove outliers by a first pass of s using projection models, such as a planar homography model computed on a grid placed on the images, and a second pass using an epipolar line constraint to result in a set of matches across the images..............................Please see Fig. 1. Abstract.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LUCIUS C.G. ALLEN whose telephone number is (703)756-5987. The examiner can normally be reached Mon - Fri 8-5pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571)272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LUCIUS CAMERON GREEN ALLEN/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights