sNotice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the amendment filed on 1/20/2026. Claims 1, 9, 10, and 15 are amended. Claim 19 is newly added. Claims 1-19 are presently pending and are presented for examination.
Claim Objections
Claim 1 is objected to because of the following informalities. Appropriate correction is required.
Claim 1 recites standardized physical dimensions. The meaning of standardized is unclear. Examiner will interpret standardized physical dimensions as referring to any physical dimension that can be measured with a standard unit.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 6, 10, and 16 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. The limitations in claims 6, 10, and 16 are already included in independent claims they depend on. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-19 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Independent claim 1 is directed toward a machine, independent claim 9 is directed to a machine, and independent claim 15 is directed to a method. Therefore, each of the independent claim(s) 1, 9, and 15 along with the corresponding dependent claims 2-8, 10-14, and 16-18 are directed to a statutory category of invention under Step 1.
Under Step 2A, Prong 1, the claims are analyzed to determine whether one or more of the claims recites subject matter that falls within one of the following groups of abstract ideas: (1) mental processes, (2) certain methods of organizing human activity, and/or (3) mathematical concepts. In this case, the independent claim(s) 1, 9, and 15 is/are directed to an abstract idea without significantly more. Specifically, the claim(s), under its/their broadest reasonable interpretation(s) cover(s) certain mental processes and mathematical calculations. The language of independent claim 15 is used for illustration:
A vehicle localization method for a vehicle, the method comprising:
acquiring image data of a roadway scene from at least one camera mounted to the vehicle;
detecting a fixed roadside object including a shape and reading, from an optic label on the fixed roadside object, a standardized physical dimension of the shape (This is a mental process. A human could visually identify a fixed roadside object with a specific label in image data.);
computing perspective dimensions of the shape from the image data and determining (This is a mathematical calculation. Calculating the perspective dimensions, i.e. pixel dimensions, in image data is a specific mathematical calculation.), using the standardized physical dimension, a metric scaling factor (Using pixel dimensions and acquired shape dimensions to determine a pixel scaling factor is a specific mathematical calculation and therefore a mathematical process.);
generating, using the metric scaling factor (S), a sign anchored, metrically scaled local region-of-interest map larger than the fixed roadside object;
identifying in the image data, a representation of a second vehicle (This is a mental process. A human could visually identify a representation of a vehicle in image data.);
and receiving, from the second vehicle, second-vehicle location information; and
determining or updating location information of the vehicle within the region-of- interest map based on the representation of the second vehicle and the second-vehicle location information, and wirelessly sharing the location information cooperative localization by one or more neighboring vehicles.
As explained above, independent claim 15 recites at least one abstract idea. The other independent claim(s), claim(s) 1 and 9, which is/are similar in scope to claim 15 likewise recite(s) at least one abstract idea under Step 2A, Prong 1.
Under Step 2A, Prong 2, the claims are analyzed to determine whether the claim, as a whole, integrates the abstract idea into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements such as merely using a computer to implement an abstract idea, adding insignificant extra-solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application”; see at least MPEP 2106.04(d).
In this case, the mental processes are not integrated into a practical application. Independent claims 1, 9, and 18 recite the additional elements. These/this limitation(s) amount to implementing the abstract idea on a computer, add insignificant extra-solution activity, and/or generally link use of the judicial exception to a particular technological environment or field of use; see at least MPEP 2106.04(d). More specifically,
A camera … found in independent claim(s) 1, 9, and 15. This limitation amounts to generally linking the use of the abstract idea to a particular technological environment or field of use;
capturing [acquiring] an image… found in independent claim(s) 1, 9, and 15. This limitation amounts to generally linking the use of the abstract idea to a particular technological environment or field of use;
reading an optic label… found in independent claim(s) 1, 9, and 15. This limitation amounts to generally linking the use of the abstract idea to a particular technological environment or field of use.
transmit[ing]… the region-of-interest map… found in independent claim(s) 1 and 9. This limitation amounts to insignificant extra-solution activity.
Instantiate[ing]/generating… a metrically scaled local region-of-interest map… found in independent claims 1, 9, and 15. This limitation amounts to generally linking the use of the abstract idea to a particular technological environment or field of use.
non-transitory memory… found in independent claim(s) 9. This limitation amounts to implementing the abstract idea on a computer.
a controller… found in independent claim(s) 1 and 9. This limitation amounts to implementing the abstract idea on a computer.
Therefore, taken alone, the additional elements do not integrate the abstract idea into a practical application. Furthermore, looking at the additional limitation(s) as an ordered combination or as a whole, the limitations add nothing significant that is not already present when looking at the elements taken individually. Because the additional elements do not integrate the abstract idea into a practical application by imposing meaningful limits on practicing the abstract idea, independent claim(s) 1, 9, and 15 is/are directed to an abstract idea.
Under Step 2B, the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application in Step 2A, Prong Two, the additional element of limiting the use of the idea to one particular environment employs generic computer functions to execute an abstract idea and, therefore, does not add significantly more. Mere instruction to apply an exception using generic computer components and limiting the use of the abstract idea to a particular environment or field of use cannot provide an inventive concept. Additionally, as discussed above, the additional limitation(s) recited above is/are considered insignificant extra-solution activity.
A conclusion that an additional element is insignificant extra-solution activity in Step 2A must be re-evaluated in Step 2B to determine if the element is more than what is well-understood, routine, and conventional in the field. In this case, the additional limitations of a non-transitory memory and a controller … are well-understood, routine, and conventional activity, because the specification does not provide any indication that they is/are anything more than conventional computer(s) within a vehicle. Additionally, the remaining element(s) has/have been deemed insignificant extra-solution activity by one or more courts; see at least MPEP 2106.05(d) and MPEP 2106.05(g):
share the location information of the vehicle… is considered well-understood, routine, and conventional activity under buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network.).
Because the claims fail to recite anything sufficient to amount to significantly more than the judicial exception, independent claim(s) 1, 9, and 15 is/are patent ineligible under 35 U.S.C. 101.
Dependent claims 2-8, 10-14, and 16-18 have been given the full two-part analysis, including analyzing the additional limitations, both individually and in combination. Dependent claims 2-8, 10-14, and 16-18, when analyzed both individually and in combination, are also patent ineligible under 35 U.S.C. 101 based on the same analysis as above. The additional limitations recited in the dependent claims fail to establish that the dependent claims are not directed to an abstract idea. The additional limitations of the dependent claims, when considered individually and as an ordered combination, do not amount to significantly more than the abstract idea. Accordingly, claims 2-8, 10-14, and 16-18 are patent ineligible under 35 U.S.C. 101.
Claim 2 recites inclusion of more data in the optic label used for camera calibration and localization. This is generally linking to a field of endeavor.
Claim 3 recites receiving image data from another vehicle as opposed to using the vehicle’s own camera. This is insignificant extra-solution activity.
Claim 4 recites determining to update the location of the ego vehicle in response to location information received by a neighboring vehicle. This is a mental process (a human could see if their current vehicle estimate matches another estimate and determine whether to update their current estimate.).
Claim 5 recites a specific set of coordinates for the location information.
Claim 6 does not specify any further limitations to claim 1.
Claim 7 recites receiving location information relative to the fixed object, i.e. another coordinate system.
Claim 8 recites implementing the system on a computer. This is insignificant extra-solution activity.
Claim 10 adds no limitations to claim 9.
Claim 11 recites a limitation of size of the map created.
Claim 12 recites determining to update the location of the ego vehicle in response to location information received by a neighboring vehicle. This is a mental process (a human could see if their current vehicle estimate matches another estimate and determine whether to update their current estimate.).
Claim 13 recites including the location of a visible feature of a second vehicle in its location information. This is generally linking to a field of endeavor.
Claim 14 recites sharing location information with a neighboring vehicle. This is insignificant extra-solution activity.
Claim 16 recites no additional limitations to claim 15.
Claim 17 recites including the location of a visible feature of a second vehicle in its location information. This is generally linking to a field of endeavor.
Claim 18 recites sharing the location information with other vehicles. This is insignificant extra-solution activity.
Claim 19 recites operating in an environment without GNSS access. This is generally linking to a field of endeavor.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 8 are rejected under 35 U.S.C. 103 as being obvious over US 10642284, hereinafter “Barazovsky”, US 20130256411 A1, hereinafter “Schuler”, WO 2019237433 A1, hereinafter “Wang”, and US 20220333950 A1, hereinafter “Akbarzadeh”.
Regarding claim 1, Barazovsky discloses A vehicle localization system for a vehicle (See Abstract, the invention is used to determine the location of a UAV, which is a vehicle.), comprising:
a camera disposed on the vehicle and oriented to acquire image data (See Abstract, the UAV comprises a camera which is used to capture the image of a ground structure, i.e. a fixed object.);
a controller coupled to the camera and to a wireless transceiver (See column 6 paragraph 5, the invention comprises a locating component that processes imagery to determine a location of the vehicle. This is also a controller because it controls location estimation. See column 9 paragraph 3, the image of the ground structure, i.e. the fixed structure, is analyzed to determine viewed perspective and angular location or orientation attributes, i.e. perspective dimensions of the fixed object captured by the camera, in order to determine the offset location of the UAV to the fixed objection, i.e. update location information of the vehicle. See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring. See column 3 paragraph 3, the system communicates over a wireless network, i.e. the vehicle comprises a wireless transceiver.), the controller being configured to:
determine or update location information of the vehicle (See column 6 paragraph 5, the invention comprises a locating component that processes imagery to determine a location of the vehicle. This is also a controller because it controls location estimation. See column 9 paragraph 3, the image of the ground structure, i.e. the fixed structure, is analyzed to determine viewed perspective and angular location or orientation attributes, i.e. perspective dimensions of the fixed object captured by the camera, in order to determine the offset location of the UAV to the fixed objection, i.e. update location information of the vehicle.); and
transmit, via wireless transceiver, the location information to one or more neighboring vehicles for cooperative localization (See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring.).
Barazovsky does not explicitly disclose a roadway scene; the camera being further configured to read, from an optic label physically affixed to a fixed roadside object, a standardized physical dimension of a shape disposed on the fixed roadside object, and to capture the shape in the image data, compute, from the image data, perspective dimensions of the captured shape and, using the standardized physical dimension read from the optic label, determine a metric scaling factor that metrically scales the image data; instantiate, using the metric scaling factor, a sign-anchored, metrically scaled local region-of-interest map that is centered at the fixed roadside object and larger than the fixed roadside object; determine or update location information of the vehicle in the region-of-interest map; or transmit, via wireless transceiver, at least a portion of the region-of-interest map.
Schuler renders obvious the camera being further configured to read, from an optic label physically affixed to a fixed roadside object, a standardized physical dimension of a shape disposed on the fixed roadside object, and to capture the shape in the image data, compute, from the image data, perspective dimensions of the captured shape and, using the standardized physical dimension read from the optic label, determine a metric scaling factor that metrically scales the image data (See Abstract and [0012]-[0017], the system allows for calibration of a camera in a way that allows transformation of pixel data to absolute geometry by use of a scaling factor. The absolute side of the calibration code, i.e. optic label, is communicated and compared to the image data corresponding to the label to determine the transformation. It would be obvious to try to use this method in a roadway setting.);
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization disclosed by Barazovsky to include calibration of scaling factors for transformations to absolute geometry of Schuler. One of ordinary skill in the art would have been motivated to make this modification in order to improve calibration of the camera system and thereby size and location of objects seen in the camera, as suggested by Schuler at [0002]-[0010].
Schuler does not explicitly disclose a roadway scene; instantiate, using the metric scaling factor, a sign-anchored, metrically scaled local region-of-interest map that is centered at the fixed roadside object and larger than the fixed roadside object; determine or update location information of the vehicle in the region-of-interest map; transmit, via wireless transceiver, at least a portion of the region-of-interest map.
Wang renders obvious instantiate, using the metric scaling factor, a sign-anchored, local region-of-interest map that is centered at the fixed roadside object and larger than the fixed roadside object (See Abstract, the system establishes a coordinate center centered on a specific QR code, i.e. anchored by the affixed label. The system tracks the location of the QR code and the cleaning robot. This is a local region-of-interest map centered at the QR code. Tracking the location of the cleaning robot as it maneuvers indicates the region-of-interest is larger than the QR code, i.e. the roadside object. See page 2 paragraph 21-page 3 paragraph 7, the two-dimensional code identified by the camera is used to calibrate camera parameters and transform the coordinate system of the robot, i.e. camera calibration factors are used in determining the map. ) and
determine or update location information of the vehicle in the region-of-interest map (See Abstract, the system establishes a coordinate center centered on a specific QR code, i.e. anchored by the affixed label. The system tracks the location of the QR code and the cleaning robot. This is a local region-of-interest map centered at the QR code. Tracking the location of the cleaning robot as it maneuvers determines and updates the information of the vehicle in the map.).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization disclosed by Barazovsky and Schuler to include centering the map at a scanned QR code and using the QR code in determining camera calibration information of Wang. One of ordinary skill in the art would have been motivated to make this modification in order to improve localization using fused sensor data, as suggested by Wang at Abstract.
Barazovsky combined with Schuler and Wang does not explicitly disclose a roadway scene; instantiate a metrically scaled local region-of-interest map; or transmit, via wireless transceiver, at least a portion of the region-of-interest map.
Akbarzadeh renders obvious a roadway scene (See Abstract, the invention updates, i.e. creates, a map of the area surrounding a vehicle. See [0008] the surrounding area can comprise a road, i.e. the area is a roadway scene.);
instantiate a metrically scaled local region-of-interest map (See [0005]-[006], cameras can be used to determine the absolute location of objects and add the objects in their corresponding locations to the HD map update. It would be obvious to use the camera calibration of Schuler to determine the absolute location of the objects. This is metric scaling based on the calibration of the camera because it determines the absolute position of detected objects based on the calibration of the camera and therefore the scaling factor of Schuler. ; and
transmit, via wireless transceiver, at least a portion of the region-of-interest map (See Abstract, newly-determined locations of detected objects are transmitted to HD map services for updating. See [0155], transmission can take place by wireless transmission.).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization including calibration of a camera for determining object location by a scaling factor disclosed by Barazovsky, Schuler, and Wang to include determining the absolute coordinates of detected objects and transmitting the updated maps wirelessly of Akbarzadeh, specifically including use of the camera calibration parameters including scaling factor. One of ordinary skill in the art would have been motivated to make this modification in order to improve updating of maps used in navigation, as suggested by Akbarzadeh at [0002]-[0005].
Regarding claim 8, Barazovsky combined with Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 1. Barazovsky discloses wherein the controller includes or is associated with non-transitory memory having program code instructions which, when executed by the controller, causes the controller to update the location information and to share the updated location information with the neighboring vehicles (See column 6 paragraph 5, the invention comprises a locating component that processes imagery to determine a location of the vehicle. This is also a controller because it controls location estimation. See column 5 paragraph 3, the UAV architecture is used to implement the methods described. The UAV architecture includes non-transitory computer readable media coupled to a processor. See column 5 paragraph 5, the non-transitory computer-readable media stores executable instructions accessible to the processor. This indicates that the processor and thereby controller execute the instructions.).
Claims 2 and 6 are rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Schuler, Wang, and Akbarzadeh, in further view of US 20190219404, hereinafter “Ahn”.
Regarding claim 2, Barazovsky combined with Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 1.
Tatarnikov discloses wherein the optic label includes coordinates of the fixed object and the controller is operable to determine a position of the vehicle based on a determined distance and orientation of the vehicle relative to the fixed object and the coordinates of the fixed object (See [0031], the positions of each camera and its angle to the marker, and thereby the fixed object it is fixed upon, are calculated and used to determine a precise position of the vehicle. Locations of parts of the marker, and thereby the location of the marker itself, are used in the positioning calculation. See [0029], the markers detection module uses encoded information from the image input in order to locate a position marker. See [0010], the position can be coordinates.).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky combined with Schuler, Wang, and Akbarzadeh to include reading the locations of optic labels from the optic labels themselves of Tatarnikov. One of ordinary skill in the art would have been motivated to make this modification because use of the exact location, combined with the angles to each camera to the label will allow for precise estimation of the vehicle’s location, as suggested by Tatarnikov at [0031].
Barazovsky combined with Schuler, Wang, Akbarzadeh, and Tatarnikov does not explicitly disclose the location information comprising the position of at least one visible feature of the vehicle. Disclosure from Ahn renders obvious the location information comprising the position of at least one visible feature of the vehicle (See [0022], the location and pose estimation system is used to operate an autonomous vehicle. To safely operate an autonomous vehicle using location estimation, it is necessary to estimate the location of the boundaries of the vehicle, all parts of which are visible features of the vehicle.)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky combined with Schuler, Wang, Akbarzadeh, and Tatarnikov to include the estimating the location of the boundaries of the vehicle of Ahn. One of ordinary skill in the art would have been motivated to make this modification because this would allow avoidance of collisions when operating autonomously, as suggested by Ahn at [0022].
Regarding claim 6, Barazovsky combined with Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 1. Barazovsky discloses a fixed object (See Fig. 6 and column 1 paragraph 9, the location of the vehicle calculated is an offset relative to a specific ground structure). Barazovsky combined with Schuler, Wang, and Akbarzadeh does not explicitly disclose the location information comprises a local map of a region of interest. Ahn, in the same field of endeavor and solving a related problem, renders obvious the location information comprises a local map of a region of interest surrounding the fixed object (See [0018], a pose system comprising multiple localizers attempts to localize the vehicle within one sub-map. If the localization is determined not to be accurate, the localizer attempts to localize the vehicle within a different submap. The first localizer sends a map change message, which corresponds to a local map of the region of interest, to a second localizer.). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle localization using fixed objects disclosed by Barazovsky combined with Schuler, Wang, and Akbarzadeh to include the multiple maps, each surrounding a fixed object, and communicated between vehicles, as suggested by Ahn. One of ordinary skill in the art would have been motivated to make this modification to narrow the area the localizer must compare its data to, thereby improving speed of computation and accuracy, as suggested by Ahn at [0017]. Claims 3-4 are rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Schuler, Wang, and Akbarzadeh in further view of NPL document “Multiple Vehicle Cooperative Localization Under Random Finite Set Framework”, hereinafter “Zhang”.
Regarding claim 3, Barazovsky combined with Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 1. Barazovsky discloses receive location information from one or more of the neighboring vehicles (See column 3 paragraph 1, UAVs communicate their temporal location, i.e. location information, to other UAVs. Other UAVs “in the area” receive this information, i.e. are neighboring vehicles.), and
selectively further update the location information of the vehicle based upon the location information received from the one or more of the neighboring vehicles (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation. This is selectively updating the location based upon the location information received.). Barazovsky does not explicitly disclose identify in image data from the camera a representation of the one or more neighboring vehicles or update the location information of the vehicle based upon the representation of the one or more neighboring vehicles. Zhang renders obvious identify in image data from the camera a representation of the one or more neighboring vehicles (See page 1405 column 1 paragraph 4, the sensors used by the vehicles for localization include a camera, i.e. use image data. See page 1406 column 1 paragraph 10, the vehicles measure the relative position of other vehicles. Using a camera as a sensor for this purpose inherently comprises identifying the neighboring vehicle in image data.); and update the location information of the vehicle based upon the representation of the one or more neighboring vehicles (See page 1406 column 1 paragraph 9-page 1406 column 2 paragraph 1, the information collected is used to update the location information of all vehicles. See page 1410 column 2 paragraph 9 for a second reference that all vehicles are localized, i.e. have their location information updated, simultaneously.). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation, which includes selectively updating the location of the vehicle based on whether the location data received from other vehicles was up to date, disclosed by Barazovsky combined with Schuler, Wang, and Akbarzadeh to include using up-to-date image data from the camera to update the location of the vehicle based on the representation of other vehicles of Zhang. One of ordinary skill in the art would have been motivated to make this modification because cooperative localization can provide a more precise estimate than individual localization, as suggested by Zhang at page 1405 column 1 paragraph 4.
Regarding claim 4, Barazovsky combined with Schuler, Wang, Akbarzadeh, and Zhang renders obvious the limitations of claim 3. Barazovsky further discloses determine whether or not to update the location information of the vehicle based upon the location information received from the one or more of the neighboring vehicles (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation. This is selectively updating the location based upon the location information received.), and the further update of the location information of the vehicle is in response to the determination of whether or not to update the location information of the vehicle (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation. This is selectively updating the location based upon the location information received.).
Claims 5 is rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Schuler, Wang, Akbarzadeh, and Tatarnikov in view US 20180322653, hereinafter “Tatarnikov”.
Regarding claim 5, Barazovsky combined with Schuler, Wang, Akbarzadeh, and Zhang renders obvious the limitations of claim 3. Barazovsky renders obvious wherein the location information from the one or more of the neighboring vehicles is relative to a second fixed object (See column 4 paragraph 4-column 5 paragraph 1, the other UAVs can calculate their position relative to other known ground structures than the first UAV, i.e. a second fixed object. See further Fig. 6 and column 1 paragraph 9, the location calculated is an offset relative to a specific ground structure.).
Tatarnikov discloses having an optic label disposed thereon (See [0029], the marker detection module extracts the encoded information from the marker in the image input, i.e. reads the data from the optic label. The image of the marker is itself an image of a shape also on the same object. See [0026], the visual marker is installed on a fixed object, in this case a specific indoor column. See [0027], the geometric features of the marker including size, which is read from the marker, are used to determine the position of the vehicle.). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle localization using fixed objects disclosed by Barazovsky to include reading optical labels that assist with localization of Tatarnikov. One of ordinary skill in the art would have been motivated to make this modification because in order to improve vehicle localization where GPS data is not available, as suggested by Tatarnikov at [0002]-[0006].
Claim 7 is rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Schuler, Wang, and Akbarzadeh, in further view of US 20210005085, hereinafter “Cheng”. Regarding claim 7, Barazovsky combined with Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 1. Barazovsky discloses receive location information (See column 3 paragraph 1, UAVs communicate their temporal location, i.e. location information, to other UAVs. Other UAVs “in the area” receive this information, i.e. are neighboring vehicles.), and selectively update the location information based upon the second location information (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation. This is selectively updating the location based upon the location information received). Barazovsky combined with Schuler, Wang, and Akbarzadeh does not explicitly disclose second location information of the vehicle relative to the fixed object, or the second location information comprising distance information between the fixed object and the vehicle as measured by the fixed object. Cheng renders obvious second location information of the vehicle relative to the fixed object (See [0059], the road-side unit (RSU), which is a fixed object, determines the location of the vehicle, and sends the location, i.e. second location information, to the vehicle. See [0007], the RSU transmits its location to the vehicle. The relative position of the vehicle to the RSU is uniquely determined by the global position of the vehicle and the RSU’s location. Examiner therefore asserts that location information relative to the RSU is therefore sent to the vehicle.), the second location information comprising distance information between the fixed object and the vehicle as measured by the fixed object (See [0059], the RSU, i.e. fixed object, determines that a vehicle is in range using its own sensors, which is measuring distance to the RSU. The RSU determines the vehicle’s location using the measured data and sends the location data to the vehicle. See [0007], the RSU transmits its location to the vehicle. The distance between the vehicle and the RSU is uniquely determined by the vehicle’s location and the RSU’s location. Examiner asserts that sending the vehicle location information is therefore second location information comprising distance information between the fixed object and the vehicle.). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle localization using fixed objects, including receiving localization information from other vehicles, disclosed by Barazovsky combined with Schuler, Wang, and Akbarzadeh, to include active localization by fixed units of Cheng. One of ordinary skill in the art would have been motivated to make this modification because including active localization information from fixed units allows the vehicles to not solely rely on their own sensors for control, providing redundancy and therefore safety, as suggested by Cheng at [0003]. Claims 9, 12, 14-15, and 18 are rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Schuler, Wang, Akbarzadeh, and Zhang.
Regarding claim 9, Barazovsky, in the same field of endeavor and solving a related problem, discloses A vehicle localization (See Abstract, the invention is used to determine the location of a UAV, which is a vehicle.) program code product stored in non-transitory memory having instructions which, when executed by a cause the controller to perform operations (See column 6 paragraph 5, the invention comprises a locating component that processes imagery to determine a location of the vehicle. This is also a controller because it controls location estimation. See column 5 paragraph 3, the UAV architecture is used to implement the methods described. The UAV architecture includes non-transitory computer readable media coupled to a processor. See column 5 paragraph 5, the non-transitory computer-readable media stores executable instructions accessible to the processor. This indicates that the processor and thereby controller execute the instructions.) comprising: acquiring image data from at least one front-facing camera on a vehicle (See Abstract, the UAV comprises a camera which is used to capture the image of a ground structure. The direction of the camera is an obvious variant.); receiving, from the second vehicle, second-vehicle location information (See column 3 paragraph 1, UAVs communicate their temporal location, i.e. location information, to other UAVs.); determining or updating vehicle location information based on the second-vehicle location information, and transmitting the vehicle location information for consumption by one or more neighboring vehicles (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation. See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring). Barazovsky does not explicitly disclose, identifying a representation of a second vehicle in the image data; based on the representation of the second vehicle in the image data and the location information for the second vehicle, within the metrically scaled local region-of-interest map, detecting in the image data a fixed roadside object having a shape of known standardized physical dimension and reading, from an optic label on the fixed roadside object, the standardized physical dimension; computing perspective dimensions of the shape from the image data and determining, using the standardized physical dimension, a metric scaling factor; or instantiating, using the metric scaling factor, a metrically scaled local region-of- interest map anchored at the fixed roadside object. Zhang renders obvious identifying in the image data a representation of a second vehicle (See page 1405 column 1 paragraph 4, the sensors used by the vehicles for localization include a camera, i.e. use image data. See page 1406 column 1 paragraph 10, the vehicles measure the relative position of other vehicles. Using a camera as a sensor for this purpose inherently comprises identifying a representation of the second vehicle in image data.); and updating vehicle location information of the vehicle based on the representation of the second vehicle in the image data (See page 1406 column 1 paragraph 9-page 1406 column 2 paragraph 1, the information collected is used to update the location information of all vehicles. See page 1410 column 2 paragraph 9 for a second reference that all vehicles are localized, i.e. have their location information updated, simultaneously.). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky to include using image data from the camera to update the location of the vehicle based on the representation of other vehicles of Zhang. One of ordinary skill in the art would have been motivated to make this modification because cooperative localization can provide a more precise estimate than individual localization, as suggested by Zhang at page 1405 column 1 paragraph 4. detecting in the image data a fixed roadside object having a shape of known standardized physical dimension and reading, from an optic label on the fixed roadside object, the standardized physical dimension (See Abstract and [0012]-[0017], the system allows for calibration of a camera in a way that allows transformation of pixel data to absolute geometry by use of a scaling factor. The absolute side of the calibration code, i.e. optic label, is communicated and compared to the image data corresponding to the label to determine the transformation. It would be obvious to try to use this method in a roadway setting);
computing perspective dimensions of the shape from the image data and determining, using the standardized physical dimension, a metric scaling factor (See Abstract and [0012]-[0017], the system allows for calibration of a camera in a way that allows transformation of pixel data to absolute geometry by use of a scaling factor. The absolute side of the calibration code, i.e. optic label, is communicated and compared to the image data corresponding to the label to determine the transformation. It would be obvious to try to use this method in a roadway setting.);
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization disclosed by Barazovsky and Zhang to include calibration of scaling factors for transformations to absolute geometry of Schuler. One of ordinary skill in the art would have been motivated to make this modification in order to improve calibration of the camera system and thereby size and location of objects seen in the camera, as suggested by Schuler at [0002]-[0010].
Barazovsky combined with Zhang and Schuler does not explicitly disclose instantiating, using the metric scaling factor, a metrically scaled local region-of- interest map anchored at the fixed roadside object.
Wang renders obvious instantiating, using the metric scaling factor, a local region-of- interest map anchored at the fixed roadside object (See Abstract, the system establishes a coordinate center centered on a specific QR code, i.e. anchored by the affixed label. The system tracks the location of the QR code and the cleaning robot. This is a local region-of-interest map centered at the QR code. Tracking the location of the cleaning robot as it maneuvers indicates the region-of-interest is larger than the QR code, i.e. the roadside object. See page 2 paragraph 21-page 3 paragraph 7, the two-dimensional code identified by the camera is used to calibrate camera parameters and transform the coordinate system of the robot, i.e. camera calibration factors are used in determining the map);
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization disclosed by Barazovsky, Zhang, and Schuler to include centering the map at a scanned QR code and using the QR code in determining camera calibration information of Wang. One of ordinary skill in the art would have been motivated to make this modification in order to improve localization using fused sensor data, as suggested by Wang at Abstract.
Barazovsky combined with Zhang, Schuler, and Wang does not explicitly disclose metrically scaled local region-of-interest map.
Akbarzadeh renders obvious a metrically scaled local region-of-interest map (See [0005]-[006], cameras can be used to determine the absolute location of objects and add the objects in their corresponding locations to the HD map update. It would be obvious to use the camera calibration of Schuler to determine the absolute location of the objects. This is metric scaling based on the calibration of the camera because it determines the absolute position of detected objects based on the calibration of the camera and therefore the scaling factor of Schuler.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization including calibration of a camera for determining object location by a scaling factor disclosed by Barazovsky, Schuler, Zhang, and Wang to include determining the absolute coordinates of detected objects and transmitting the updated maps wirelessly of Akbarzadeh, specifically including use of the camera calibration parameters including scaling factor. One of ordinary skill in the art would have been motivated to make this modification in order to improve updating of maps used in navigation, as suggested by Akbarzadeh at [0002]-[0005].
Regarding claim 12, Barazovsky combined with Schuler, Zhang, Wang, and Akbarzadeh renders obvious the limitations of claim 9. Barazovsky further discloses determining whether or not to update the location information of the vehicle based upon the location information received from the second vehicle (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation. This is selectively updating the location based upon the location information received.), and
updating the location information of the vehicle in response to the determination of whether or not to update the location information of the vehicle (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation. This is selectively updating the location based upon the location information received.).
Regarding claim 14, Barazovsky combined with Schuler, Zhang, Wang, and Akbarzadeh renders obvious the limitations of claim 9. Barazovsky further discloses sharing the location information of the vehicle with one or more neighboring vehicles (See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring.).
Regarding claim 15, Barazovsky, in the same field of endeavor and solving a related problem, discloses A vehicle localization method for a vehicle (See Abstract, the invention is used to determine the location of a UAV, which is a vehicle.) the method comprising: acquiring image data from at least one camera mounted to the vehicle (See Abstract, the UAV comprises a camera which is used to capture the image of a ground structure.); receiving, from the second vehicle, second-vehicle location information (See column 3 paragraph 1, UAVs communicate their temporal location, i.e. location information, to other UAVs.); determining or updating location information of the vehicle within the region-of-interest map based on the second-vehicle location information (See column 11 paragraph 4, a UAV in the area receives the known locations at given times of other UAVs, i.e. location information. If a sufficient number of other UAV locations is received in a short enough time frame, the known locations and times are used to update the first UAV’s position, which is location information, using triangulation.), and
wirelessly sharing the location information for cooperative localization by one or more neighboring vehicles (See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring. See column 3 paragraph 3, the system communicates over a wireless network, i.e. the information is shared wirelessly.). Barazovsky does not explicitly disclose image data of a roadway scene; identifying, in the image data, a representation of a second vehicle; based on the representation of the second vehicle in the image data and the location information for the second vehicle; or new stuff. Zhang renders obvious identifying, in the image data, a representation of a second vehicle (See page 1405 column 1 paragraph 4, the sensors used by the vehicles for localization include a camera, i.e. use image data. See page 1406 column 1 paragraph 10, the vehicles measure the relative position of other vehicles. Using a camera as a sensor for this purpose inherently comprises identifying a representation of the second vehicle in image data.); and based on the representation of the second vehicle in the image data (See page 1406 column 1 paragraph 9-page 1406 column 2 paragraph 1, the information collected is used to update the location information of all vehicles. See page 1410 column 2 paragraph 9 for a second reference that all vehicles are localized, i.e. have their location information updated, simultaneously.). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky to include using image data from the camera to update the location of the vehicle based on the representation of other vehicles of Zhang. One of ordinary skill in the art would have been motivated to make this modification because cooperative localization can provide a more precise estimate than individual localization, as suggested by Zhang at page 1405 column 1 paragraph 4.
Barazovsky combined with Zhang does not explicitly disclose image data of a roadway scene; detecting a fixed roadside object including a shape and reading, from an optic label on the fixed roadside object, a standardized physical dimension of the shape; computing perspective dimensions of the shape from the image data and determining, using the standardized physical dimension, a metric scaling factor; or generating, using the metric scaling factor (S), a sign anchored, metrically scaled local region-of-interest map larger than the fixed roadside object. Schuler renders obvious the detecting a fixed roadside object including a shape and reading, from an optic label on the fixed roadside object, a standardized physical dimension of the shape (See Abstract and [0012]-[0017], the system allows for calibration of a camera in a way that allows transformation of pixel data to absolute geometry by use of a scaling factor. The absolute side of the calibration code, i.e. optic label, is communicated and compared to the image data corresponding to the label to determine the transformation. It would be obvious to try to use this method in a roadway setting.); and
computing perspective dimensions of the shape from the image data and determining, using the standardized physical dimension, a metric scaling factor (See Abstract and [0012]-[0017], the system allows for calibration of a camera in a way that allows transformation of pixel data to absolute geometry by use of a scaling factor. The absolute side of the calibration code, i.e. optic label, is communicated and compared to the image data corresponding to the label to determine the transformation. It would be obvious to try to use this method in a roadway setting.).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization disclosed by Barazovsky to include calibration of scaling factors for transformations to absolute geometry of Schuler. One of ordinary skill in the art would have been motivated to make this modification in order to improve calibration of the camera system and thereby size and location of objects seen in the camera, as suggested by Schuler at [0002]-[0010].
Barazovsky combined with Zhang and Schuler does not explicitly disclose image data of a roadway scene, or generating, using the metric scaling factor (S), a sign anchored, metrically scaled local region-of-interest map larger than the fixed roadside object
Wang renders obvious generating, using the metric scaling factor (S), a sign anchored, local region-of-interest map larger than the fixed roadside object (See Abstract, the system establishes a coordinate center centered on a specific QR code, i.e. anchored by the affixed label. The system tracks the location of the QR code and the cleaning robot. This is a local region-of-interest map centered at the QR code. Tracking the location of the cleaning robot as it maneuvers indicates the region-of-interest is larger than the QR code, i.e. the roadside object. See page 2 paragraph 21-page 3 paragraph 7, the two-dimensional code identified by the camera is used to calibrate camera parameters and transform the coordinate system of the robot, i.e. camera calibration factors are used in determining the map. )
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization disclosed by Barazovsky and Schuler to include centering the map at a scanned QR code and using the QR code in determining camera calibration information of Wang. One of ordinary skill in the art would have been motivated to make this modification in order to improve localization using fused sensor data, as suggested by Wang at Abstract.
Barazovsky combined with Zhang, Schuler, and Wang does not explicitly disclose image data of a roadway scene or a metrically scaled local region-of-interest map.
Akbarzadeh renders obvious image data of a roadway scene (See Abstract, the invention updates, i.e. creates, a map of the area surrounding a vehicle. See [0008] the surrounding area can comprise a road, i.e. the area is a roadway scene.);
a metrically scaled local region-of-interest map (See [0005]-[006], cameras can be used to determine the absolute location of objects and add the objects in their corresponding locations to the HD map update. It would be obvious to use the camera calibration of Schuler to determine the absolute location of the objects. This is metric scaling based on the calibration of the camera because it determines the absolute position of detected objects based on the calibration of the camera and therefore the scaling factor of Schuler.)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for cooperative localization including calibration of a camera for determining object location by a scaling factor disclosed by Barazovsky, Zhang, Schuler, and Wang to include determining the absolute coordinates of detected objects and transmitting the updated maps wirelessly of Akbarzadeh, specifically including use of the camera calibration parameters including scaling factor. One of ordinary skill in the art would have been motivated to make this modification in order to improve updating of maps used in navigation, as suggested by Akbarzadeh at [0002]-[0005].
Regarding claim 18, Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 15. Barazovsky further discloses sharing the location information of the vehicle with one or more neighboring vehicles (See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring.).
Claims 10-11 are rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Zhang, Schuler, Wang, Akbarzadeh, and Tatarnikov in view of Ahn.
Regarding claim 10, Barazovsky combined with Schuler, Zhang, Wang, and Akbarzadeh renders obvious the limitations of claim 9. Barazovsky further discloses sharing the location information of the vehicle with one or more neighboring vehicles (See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring.). Barazovsky combined with Schuler, Zhang, Wang, and Akbarzadeh does not explicitly disclose the image data includes a shape of a representation of a fixed object, wherein updating location information of the vehicle is based on perspective dimensions of the shape of the fixed object representation captured by the camera and actual dimensions of the shape read from an optic label disposed on the fixed object or the location information comprising location information of at least one visible feature of the vehicle.
Tatarnikov, in the same field of endeavor and solving a related problem, discloses the image data includes a shape of a representation of a fixed object, wherein updating location information of the vehicle is based on perspective dimensions of the shape of the fixed object representation captured by the camera and actual dimensions of the shape read from an optic label disposed on the fixed object (See [0029], the marker detection module extracts the encoded information from the marker in the image input, i.e. reads the data from the optic label. The image of the marker is itself an image of a shape also on the same object. See [0026], the visual marker is installed on a fixed object, in this case a specific indoor column. See [0027], the geometric features of the marker including size, which is read from the marker, are used to determine the position of the vehicle. See [0031], the positions of each camera and its angle to the marker, and thereby the fixed object it is fixed upon, are calculated and used to determine a precise position of the vehicle. Locations of parts of the marker, and thereby the location of the marker itself, are used in the positioning calculation.).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle localization using fixed objects disclosed by Barazovsky and Zhang to include reading optical labels that assist with localization of Tatarnikov. One of ordinary skill in the art would have been motivated to make this modification because in order to improve vehicle localization where GPS data is not available, as suggested by Tatarnikov at [0002]-[0006].
Barazovsky combined with Schuler, Zhang, Wang, Akbarzadeh, and Tatarnikov does not explicitly disclose the location information comprising the position of at least one visible feature of the vehicle. Disclosure from Ahn renders obvious the location information comprising the position of at least one visible feature of the vehicle (See [0022], the location and pose estimation system is used to operate an autonomous vehicle. To safely operate an autonomous vehicle using location estimation, it is necessary to estimate the location of the boundaries of the vehicle, all parts of which are visible features of the vehicle.)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky, Schuler, Zhang, Wang, Akbarzadeh, and Tatarnikov to include the estimating the location of the boundaries of the vehicle of Ahn. One of ordinary skill in the art would have been motivated to make this modification because this would allow avoidance of collisions when operating autonomously, as suggested by Ahn at [0022]. Regarding claim 11, Barazovsky combined with Barazovsky, Schuler, Zhang, Wang, Akbarzadeh, Tatarnikov, and Ahn renders obvious the limitations of claim 10. Ahn further renders obvious wherein the location information comprises a local map of a region of interest surrounding the fixed object, the region of interest being larger than the fixed object (See [0018], a pose system comprising multiple localizers attempts to localize the vehicle within one sub-map. If the localization is determined not to be accurate, the localizer attempts to localize the vehicle within a different submap. The first localizer sends a map change message, which corresponds to a local map of the region of interest, to a second localizer.). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle localization using fixed objects disclosed by Barazovsky, Tatarnikov, and Zhang to include the multiple maps, each surrounding a fixed object, and communicated between vehicles, as suggested by Ahn. One of ordinary skill in the art would have been motivated to make this modification to narrow the area the localizer must compare its data to, thereby improving speed of computation and accuracy, as suggested by Ahn at [0017]. Claims 16 and 19 are rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Zhang, Schuler, Wang, and Akbarzadeh in view of Tatarnikov.
Regarding claim 16, Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 15. Barazovsky further discloses sharing the location information of the vehicle with one or more neighboring vehicles (See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring.). Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh does not explicitly disclose the image data includes a shape of a representation of a fixed object, wherein updating location information of the vehicle is based on perspective dimensions of the shape of the fixed object representation captured by the camera and actual dimensions of the shape read from an optic label disposed on the fixed object.
Tatarnikov, in the same field of endeavor and solving a related problem, discloses the image data includes a shape of a representation of a fixed object, wherein updating location information of the vehicle is based on perspective dimensions of the shape of the fixed object representation captured by the camera and actual dimensions of the shape read from an optic label disposed on the fixed object (See [0029], the marker detection module extracts the encoded information from the marker in the image input, i.e. reads the data from the optic label. The image of the marker is itself an image of a shape also on the same object. See [0026], the visual marker is installed on a fixed object, in this case a specific indoor column. See [0027], the geometric features of the marker including size, which is read from the marker, are used to determine the position of the vehicle. See [0031], the positions of each camera and its angle to the marker, and thereby the fixed object it is fixed upon, are calculated and used to determine a precise position of the vehicle. Locations of parts of the marker, and thereby the location of the marker itself, are used in the positioning calculation.).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle localization using fixed objects disclosed by Barazovsky and Zhang to include reading optical labels that assist with localization of Tatarnikov. One of ordinary skill in the art would have been motivated to make this modification because in order to improve vehicle localization where GPS data is not available, as suggested by Tatarnikov at [0002]-[0006].
Regarding claim 19, Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 15. Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh does not explicitly disclose operating in an environment without Global Navigation Satellite System access and generating the region-of-interest map and the vehicle location information without Global Navigation Satellite System position fixes.
Tatarnikov renders obvious operating in an environment without Global Navigation Satellite System access and generating the region-of-interest map and the vehicle location information without Global Navigation Satellite System position fixes (See Abstract and [0002]-[0006], the invention uses markers for vehicle location in order to improve vehicle positioning when GPS, i.e. GNSS access, is not available.).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh to include operating in regions without GPS of Tatarnikov. One of ordinary skill in the art would have been motivated to make this modification because this would allow better vehicle positioning when GPS is not available, as suggested by Tatarnikov at [0002].
Claims 13 and 17 are rejected under 35 U.S.C. 103 as being obvious over Barazovsky, Zhang, Schuler, Wang, and Akbarzadeh in view of Ahn.
Regarding claim 13, Barazovsky combined with Schuler, Zhang, Wang, and Akbarzadeh renders obvious the limitations of claim 9. Barazovsky combined with Schuler, Zhang, Wang, and Akbarzadeh does not explicitly disclose the location information of the second vehicle comprises localization information for a visible feature of the second vehicle. Disclosure from Ahn, in the same field of endeavor and solving a related problem, renders obvious the location information of the second vehicle comprises localization information for a visible feature of the second vehicle (See [0022], the location and pose estimation system is used to operate an autonomous vehicle. To safely operate an autonomous vehicle using location estimation, it is necessary to estimate the location of the boundaries of the vehicle, all parts of which are visible features of the vehicle. This is localization information.)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky and Zhang to include the estimating the location of the boundaries of the vehicle of Ahn. One of ordinary skill in the art would have been motivated to make this modification because this would allow avoidance of collisions when operating autonomously, as suggested by Ahn at [0022].
Regarding claim 17, Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh renders obvious the limitations of claim 15. Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh does not explicitly disclose the location information of the second vehicle comprises localization information for a visible feature of the second vehicle. Disclosure from Ahn, in the same field of endeavor and solving a related problem, renders obvious the location information of the second vehicle comprises localization information for a visible feature of the second vehicle (See [0022], the location and pose estimation system is used to operate an autonomous vehicle. To safely operate an autonomous vehicle using location estimation, it is necessary to estimate the location of the boundaries of the vehicle, all parts of which are visible features of the vehicle. This is localization information.)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system for vehicle location estimation disclosed by Barazovsky combined with Zhang, Schuler, Wang, and Akbarzadeh to include the estimating the location of the boundaries of the vehicle of Ahn. One of ordinary skill in the art would have been motivated to make this modification because this would allow avoidance of collisions when operating autonomously, as suggested by Ahn at [0022].
Response to Arguments
(A) Applicant argues “Claim 10 was objected to because the preamble recited "The program code product of claim 9." The preamble has been amended to recite "The vehicle localization program code product," as required by the Office Action.”
As to (A), Examiner agrees that the objection has been overcome.
(B) Applicant argues “The amended claims are not directed to an abstract idea. As amended, the independent claims are directed to specific computer-vision and V2X operations that improve the functioning of a vision-based localization system and distributed cooperative localization. The amended claims recite reading a standardized physical dimension of a shape from an optic label on a fixed roadside object and computing a metric scaling factor by comparing perspective dimensions in the image with the actual standardized dimensions. This is more than "data analysis"-it is a specific geometric computation that eliminates a foundational numerical indeterminacy (scale) in monocular vision. See ||||[0029]-[0034] (label 16 on sign 14; camera 12 captures image 22 of shape/box 34); |||||0034]-[0035] (controller 25 uses actual dimensions 18/20 VS image dimensions 24/26/28/30 and projective geometry; focal point 32); Fig. 1 (14, 16, 34, 18/20, 24/26/28/30, 32); Figs. 3-5 (known points and corresponding image points).
The currently amended claims explicitly generate a scaled local region-of-interest (ROI) map centered on the sign (and larger than the sign) and then compute/update vehicle pose within that specific ROI. This is a novel approach and processing context that changes how sensor data is structured and used. See |[0062] ("reconstructs a map region of interest around the road sign 14 fixed size gives scale may extend region of interest"); |[0064] (flow 606-ROI creation at scale from images containing the sign). Fig. 6 (block 606). The currently amended claims recite transmitting at least portions of the ROI and/or location information to neighboring vehicles, enabling immediate alignment and cooperative localization rather than each vehicle recomputing from scratch. See ||[0063] (master vehicle shares updated localization/map with slaves), T[0065] (share at 612 via V2V/V2X; neighbors update at 614), T[0069] (V2X-enabled collaboration for improved localization when others have not yet seen the sign). Figs. 6-7 (blocks 612-614; master/slave). Dependent claims are directed to, and the specification describes, feature-level landmarks (e.g., mid-point of the rear plate) as secondary landmarks (T[0065]; Fig. 6, 612), selective acceptance/rejection [T[0066] permitting rejection of untrusted smart-sign data; ||[0067] cross-validation against hacked/damaged signs), GNSS-denied operation (T[0064], tunnel), and ROI larger than sign reconstructed at scale (T[0062]). Result under Prong One: The claims are not directed to a judicial exception. They are directed to a specific improvement in computer-implemented localization and V2X coordination, reciting defined inputs, computations, data structures, and transmissions that alter how the system processes and shares sensor data. See 11[0029]-[0037], [0062]- [0069]; Figs. 1, 3-7. Even assuming arguendo an abstract idea were implicated (it is not), the currently amended claims are integrated into a practical application in multiple, independent ways. The currently amended claims are tied to a vehicle-mounted camera (12) and a controller (25) operating in roadway scenes; the steps include reading an optic label (16) on a sign (14), computing projective geometry to derive scale, instantiating a scaled ROI, and wirelessly transmitting to peers (T10029]-[0037], [0062]-[0065]; Figs. 1, 6-7). Raw images and label data are transformed into a metrically-scaled ROI (block 606) that the system uses to compute pose and control that is shared (T[0062]-[0065]; Fig. 6). The metrics (scale, ROI, pose) directly affect navigation/localization and peer alignment, not merely display (10064]-[0065]; Figs. 6-7). And the pipeline expressly addresses tunnels/urban canyons (T[0064])- a technology-specific problem-with a technology-specific solution (label-anchored metric scale ROI cooperative sharing). Moreover, taken as an ordered combination, the recited operations go beyond well-understood, routine, and conventional activities. Reading a standardized physical dimension from an optic label (16) on an ordinary roadside sign (14) and computing metric scale from perspective vs. actual image geometry (dimensions 18/20 vs. 24/26/28/30, focal point 32) to resolve monocular scale is not a conventional step in generic VIO/SLAM. This patent application teaches projective transformation-based scale/pose from the captured shape (34) (T100331-[0035], [0037]; Fig. 1). The sign-anchored, metrically-scaled ROI (block 606) is not a mere "displayed result"; it is a compact, scale-resolved coordinate frame used to compute pose, compare features (block 608), update localization (block 610), and control what is transmitted to peers (blocks 612-614) (T[0062]-[0065]; Fig. 6). This application sets forth that the currently amended claims are not routine boilerplate steps (e.g., scale from label-anchored shape geometry; ROI at scale; security cross-validation) thereby providing a sufficient factual basis that the recited combination is significantly more than any alleged abstract idea (see TT[0033]- [0037], [0062]-[0069]). The currently amended claims are directed to operations (camera imaging, pixel-wise geometry, ROI reconstruction, V2X message exchange) that cannot be performed mentally, particularly at real-time vehicular latencies (T10062]-[0065]; Fig. 6). The currently amended claims, supported by the written description and the drawings, are directed to a particular improvement in computer-implemented localization and distributed V2X operation-optic-label/shape-based metric scaling, sign-anchored ROI instantiation, and protocolized cooperative sharing (e.g., 11[0029]-[0037], [0062]-[0069]; Figs. 1, 3-7). Under Step 2A (both prongs) and Step 2B, the claims recite technological, implementation-specific steps and data structures that constitute significantly more than any alleged abstract idea. Applicant, therefore, respectfully requests withdrawal of the § 101 rejection.”
As to (B), Examiner does not find the argument persuasive.
Step 2A Prong One: All independent claims recite at least one judicial exception, as described in the 101 section.
Step 2A Prong Two: The additional limitations are not sufficient to show integration into a practical application. More specifically, the limitations specifically mentioned by Applicant are obvious from the prior art made of record in the current office action.
Step 2B: Examiner does not find the argument persuasive. The combination of conventional technologies in obvious ways is not sufficient to show a technical solution to a technical problem.
(C) Applicant argues “The amended independent claims are directed to reading a standardized physical dimension (e.g., of the visible symbol/box 34) from an optic label 16 on a fixed sign 14, and comparing with perspective image dimensions 24/26/28/30 to compute metric scale (distance/orientation)-Fig. 1; 11[0029]-[0035]; instantiating a local region-of-interest around the sign that is reconstructed at scale and larger than the sign-Fig. 6 (block 606); ||[0062] and T[0064]; and transmitting (V2V/V2X) the ROI/pose so neighboring vehicles can update/localize without recomputation-Fig. 6 (blocks 612-614), Fig. 7 (master/slave); TT[0063], [0065], [0069]. Dependent claims further narrow to feature-level landmarks (e.g., mid-point of the rear plate) (T[0065]; Fig. 6, 612), selective acceptance/rejection ([0066]) and cross-validation (T[0067]) for smart-sign data, operation in GPS-denied settings (T[0064]), and ROI larger than sign with scaled reconstruction (T[0062]). The specification teaches reading actual dimensions of the sign's visible symbol/box 34 from the optic label 16 on the sign 14 (TT[0029]-[0033]; Fig. 1) and using projective geometric transformations to compare the image dimensions 24/26/28/30 with actual dimensions 18/20 referenced to the camera focal point 32, thereby deriving distance/orientation (i.e., metric scale) (T10034]-[0035]; Fig. 1). The disclosure explicitly reconstructs a map in a region of interest around the sign and states that the fixed size of the sign provides information to reconstruct the map at scale; the ROI may be extended beyond the sign (T[0062]; Fig. 6, block 606; also |[0064] describing 606). After localization/correction, a master vehicle shares updated data (localization/map) with slave vehicles (T[0063]; Fig. 7). The flow shows share at 612 and neighbor update at 614 (T[0065]; Fig. 6, 612-614); even vehicles that have not seen the sign improve localization (T[0069]). The controller may localize visual features (e.g., mid-point of the rear plate) (T[0065]; Fig. 6, 612); smart sign 14 can provide distances, which vehicles may accept or reject (T[006]); vehicles may cross-validate to detect damaged/hacked signs (T[0067]) or cross-validate peer results (T[0068]). The currently amended claims are directed to reading actual dimensions from optic label 16 and comparing to perspective image dimensions 24/26/28/30 of the shape 34 to compute metric scale (T10029]-[0035]; Fig. 1). The cited art of record does not disclose a vision-only, object-dimension anchoring method that derives metric scale from a known-size shape read from the sign's label, and that then uses projective geometry to compute distance/orientation. The specification shows such a geometric pipeline with numerals and math context (T1[0034]-[0035]; Fig. 1, Figs. 3-5). The currently amended claims are directed to creating a scaled local ROI around the sign and extending it-Fig. 6, 606; ||[0062] ("reconstructed at scale"; "extend the region of interest"); |[0064] (implementation of 606). The applied references lean on prebuilt maps, infrastructure-maintained coverage maps, or estimator-centric frameworks-not on-the-fly, vehicle-generated, sign-anchored, scaled ROI maps derived from label-anchored geometry (T[0062], 606). After ROI/pose are established, share at 612, neighbors update at 614; master/slave role behavior-Fig. 6, 612-614; Fig. 7; TT[0063], [0065], [0069]. The applied art does not disclose publishing a sign-anchored, scaled ROI with defined message-flow enabling peers to "snap-in" without recomputing from scratch. The spec squarely teaches this cooperative pipeline, with structure/flow numerals (TT[0063], [0065], [0069]; Figs. 6-7). The currently amended claims are also directed to feature-level reference (e.g., mid-point of the rear plate) (T[0065]; Fig. 6, 612) and smart-sign information that is accepted or rejected and cross-validated (T1066]-[0068]). The applied prior art references do not teach elevating vehicle features to landmarks coherently tied to a sign-anchored ROI, nor a fleet-based accept/reject and cross-validation layer keyed to that ROI. This patent application discloses and claims both. (TM[0065]-[0068]). The claims, as currently amended, replace reliance on prebuilt sub-maps or infrastructure coverage by a vision-only, label-anchored metric bootstrap that yields a sign-anchored, scaled ROI (110034]-[0035], [0062]; Fig. 6, 606). This shift conflicts with the map/infrastructure premises attributed to the references applied against the claims in the Office Action. Combining the prior art references in the manner suggested in the Office Action would not be a "straightforward combination." The prior art of record lacks a teaching or motivation to obtain metric scale from printed/encoded dimensions of a sign's symbol (read from label 16) and then to reconstruct an ROI at scale (T10033]-[0035], [0062]). The why and how of doing so come from this patent application (e.g., projective transform and explicit pipeline), not from the prior art of record. disclose publishing a sign-anchored, scaled ROI with defined message-flow enabling peers to "snap-in" without recomputing from scratch. The spec squarely teaches this cooperative pipeline, with structure/flow numerals (TT[0063], [0065], [0069]; Figs. 6-7). The currently amended claims are also directed to feature-level reference (e.g., mid-point of the rear plate) (T[0065]; Fig. 6, 612) and smart-sign information that is accepted or rejected and cross-validated (T1066]-[0068]). The applied prior art references do not teach elevating vehicle features to landmarks coherently tied to a sign-anchored ROI, nor a fleet-based accept/reject and cross-validation layer keyed to that ROI. This patent application discloses and claims both. (TM[0065]-[0068]). The claims, as currently amended, replace reliance on prebuilt sub-maps or infrastructure coverage by a vision-only, label-anchored metric bootstrap that yields a sign-anchored, scaled ROI (110034]-[0035], [0062]; Fig. 6, 606). This shift conflicts with the map/infrastructure premises attributed to the references applied against the claims in the Office Action. Combining the prior art references in the manner suggested in the Office Action would not be a "straightforward combination." The prior art of record lacks a teaching or motivation to obtain metric scale from printed/encoded dimensions of a sign's symbol (read from label 16) and then to reconstruct an ROI at scale (T10033]-[0035], [0062]). The why and how of doing so come from this patent application (e.g., projective transform and explicit pipeline), not from the prior art of record.”
As to (C), Examiner does not find the argument persuasive. Barazovsky recites transmitting vehicle pose information to neighboring vehicles for cooperative localization (See column 7 paragraph 6-column 8 paragraph 1, the locating component receives the temporal location, i.e. location information, from other UAVs in order to perform triangulation. The UAVs perform triangulation in reference to known ground structures, indicating at least one ground structure must be in the common vicinity of at least two UAVs. This indicates that at least two UAVs are neighboring.).
Applicant’s remaining arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for the remaining teachings or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20140343842 A1 which relates to localization using road markings.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AUSTIN ROBERT CHENNAULT whose telephone number is (571)272-4606. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hitesh Patel can be reached at (571) 270-5442. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AUSTIN ROBERT CHENNAULT/Examiner, Art Unit 3667
/Hitesh Patel/Supervisory Patent Examiner, Art Unit 3667 6/25/26
Prosecution Insights