DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined
under the first inventor to file provisions of the AIA .
Under 35 USC § 101
Claims 1-12 are directed to patent-eligible subject matter under 35 U.S.C. § 101. Although the claims may recite mathematical concepts in connection with
determining and applying a transformation between coordinate systems, the claims as a whole integrate any such mathematical concept into a practical application. In particular, the claims apply the coordinate transformation to mapping data obtained by environmental scanning by a UAV associated with a mining vehicle to transform the mapping data from another mapping source. Thus, the claimed mathematical operations are applied as part of a technological process for spatially registering and integrating mapping data obtained from physical mapping sources into a common mining-worksite model, rather than merely performing the mathematical operations in the abstract. Accordingly, the claims are not directed to a judicial exception under Step 2A, Prong Two, and are eligible under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 13 is rejected under 35 USC § 101 because they are directed to non-statutory subject matter.
The descriptions or expressions of the programs are not physical “things.” They are neither computer components nor statutory processes, as they are not “acts” being performed. Such claimed computer programs do not define any structural and functional interrelationships between the computer program and other claimed elements of a computer, which permit the computer program’s functionality to be realized. In contrast, a claimed a non-transitory computer-readable medium encoded with a computer program is a computer element which defines structural and functional interrelationships between the computer program and the rest of the computer which permit the computer program’s functionality to be realized and is thus statutory. Accordingly, it is important to distinguish claims that define descriptive material per se from claims that define statutory inventions.
In order to overcome this rejection, the following language is suggested:
“13. (Currently amended) A non-transitory computer readable medium encoded with a computer program code for …”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all
obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 and 5-13 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Pub. No. US 2021/0131821) (hereinafter Wang ’821 in view of Wang et al. (Pub. No. US 2017/0109577) (hereinafter Wang ‘577).
As per claims 1, 3, 9, 10 and 13, Wang ’821 teaches an apparatus comprising at least one processor configured to perform collaborative mapping between an unmanned aerial vehicle (UAV) and a ground vehicle. Wang ‘821 discloses a ground vehicle including a computing device and scanning sensor and an aerial vehicle including another computing device and scanning sensor, wherein the UAV and ground vehicle cooperate to obtain and construct maps of their environment (see ¶¶ [0026],[0058]-[0064], Fig. 1 and Figs. 6-9).
Wang ’821 teaches detect first mapping data, the first mapping data being based on environment scanning by an unmanned aerial vehicle, UAV, associated with a vehicle (see Abstract and ¶¶ [0058] and [0073], i.e., aerial vehicle having one or more scanning sensors with which the aerial vehicle scans its environment and ground vehicle having its own scanning sensors; ¶¶ [0052]-[0053], [0060], [0073], [0082], [0091] and Fig. 5, i.e., receiving first scanning data from computing device coupled to the aerial vehicle, wherein the first scanning data is obtained using a first scanning sensor coupled to the aerial vehicle and includes first mapping data generated based on point-cloud data collected by the first scanning sensor; ¶ [0053], i.e., the first scanning sensor may be a LiDAR sensor and that second mapping data may be generated based on point-cloud data collected by a second scanning sensor of the ground vehicle).
Wang ’821 further teaches a close physical and operational association between the UAV and ground vehicle. For example, the ground vehicle may include a platform from which the aerial vehicle launches and lands, and the ground vehicle may determine when additional scanning data are required and launch the aerial vehicle collect such data (see ¶ [0059]). Thus, Wang ’821 teaches a UAV associated with a ground vehicle for collaboratively scanning and mapping the vehicle environment.
Wang ’821 further teaches that the first mapping data is associated with a local coordinate system of the UAV and/or the vehicle (see ¶¶ [0048] and [0064], i.e., the scanning data may comprise point-cloud data representing the three-dimensional environment and that each point in the point-cloud is associated with a position in a scanner reference frame determined relative to the scanning sensor). Thus, the point-cloud mapping data are represented in a local/scanner coordinate system associated with the UAV scanning sensor.
Wang ’821 further teaches that positioning data of the movable object are used to convert the position in the scanner reference frame to an output reference frame in a world coordinate system (see ¶ [0048]). Wang ’821 additionally teaches coordinate transformations between image coordinates and world coordinates and teaches using projection/perspective transformations to map positions between coordinate systems (see ¶ [0048]). Accordingly, Wang ’821 teaches transforming mapping data from a local/scanner reference coordinate system into a common/world reference coordinate system based on a transformation defining the relationship between the coordinate systems.
Wang ’821 further teaches that the UAV and ground vehicle can each perform SLAM to generate respective maps of the environment (see ¶¶ [0059]-[0060]). The UAV can transmit its map to the ground vehicle, which combines the UAV map generated by the ground vehicle to produce a higher-precision combined map (see ¶¶ [0065] and [0067]).
Regarding “provide the second mapping data for a worksite model comprising having third mapping data from another mapping source” Wang ’821 teaches that the SLAM generated by the UAV is transmitted to the ground vehicle and combined with the SLAM generated by the ground vehicle to extend the local map generated by the ground vehicle (see ¶ [0065]). Wang ’821 further teaches that first mapping data generated from point-cloud data collected by the aerial-vehicle scanning sensor are combined with second mapping data generated from data collected by the ground-vehicle scanning sensor to increase the coverage area of the local map maintained by the ground vehicle (see ¶ [0067]). The mapping data generated by the ground-vehicle sensor, therefore, constitute mapping data from another mapping source.
Wang ’821 further teaches, with respect to the combination of the maps, that the ground vehicle identifies an overlapping portion of the UAV and ground-vehicle real-time maps and combines them into a combined real-time map. The maps may be merged using scan-matching techniques or feature matching, and locations in each map are mapped to the other based on the overlapping portion (¶ [0082]). Wang ’821 further teaches converting the coordinate system of an image-based UAV map to match the coordinate system of point-cloud data collected by the ground vehicle (see ¶ [0082]). Thus, Wang ’821 teaches providing UAV mapping information transformed into common coordinate system to a model/map already comprising independently obtained ground-vehicle mapping information.
Wang ’821, however, fails to explicitly teach that the ground vehicle is specifically a mining vehicle, that the mapped environment is specifically a mining worksite, or that the common/world reference system and resulting model are specifically a worksite reference system and worksite model of a mining worksite.
Wang ‘577, however, teaches that the aerial system may be an unmanned aerial system (UAs) equipped with a camera and GPS receiver and configured to fly over the worksite while capturing aerial images for mapping the worksite (see ¶ [0018]). Wang ‘577 further teaches that the machines operating at the worksite may include excavators, loaders, dozers, motor graders, haul trucks, and/or other equipment that move around the area and perform tasks therein (see ¶ [0021]). Thus, Wang ‘577 explicitly teaches mine-site embodiment, Wang ‘577 teaches a UAV/UAS operating in conjunction with mobile mine-site machines to obtain mapping information concerning the mine worksite.
Wang ‘577 additionally teaches that the mapping unit receives the aerial images from the UAs and constructs a preliminary 3D terrain map and calibrates the preliminary 3D terrain map based on location information associated with the machines (see ¶ [0025]). The machines include RTK GPS receivers and determine global locations of ground-control points associated with the machines (¶ [0023]). Accordingly, Wang ‘577 explicitly demonstrates that UAV-derived mapping data and machine-derived global-location information were conventionally integrated into common 3D worksite model in a mine-site environment.
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to apply the collaborative UAV/ground-vehicle mapping architecture taught by Wang ’821 to the mine-site environment and mine-site machines taught by Wang ‘577, such that Wang ’821’s ground vehicle is a mining vehicle and its collaboratively generated common map constitutes a mining-worksite model expressed in a worksite reference coordinate system, because Wang ’577 explicitly teaches that mine sites and similar worksites commonly undergo geographic alteration by machines and workers and that it is therefore useful to generate a three-dimensional terrain map of such worksites using an unmanned aerial system together with machines operating at the worksite (see ¶¶ [0002] and [0067]), thereby enabling Wang ’821 collaborative UAV/vehicle scanning, coordinate transformation, and multi-source map construction techniques to provide a more comprehensive and accurately referenced map of the changing mining worksite.
As per claims 2 and 11, the combination of Wang ’821 and Wang ‘577 teaches the system as stated above.
Wang ’821 further teaches “updating a real-time map of the environment in which a movable object is operating and simultaneously maintaining a position of the moveable object within the real-time map. deals with a computational problem of constructing or updating a map of an unfamiliar environment while simultaneously keeping track of an agent's local with it” using SLAM (see ¶ [0074]) and applying UAV-derived mapping information to a map also containing mapping information obtained from the ground vehicle (see ¶¶ [0073]-[0078]).
Wang ’577 teaches employing UAV mapping in a mining-worksite environment and maintaining a three-dimensional terrain model of the mine site using mapping information associated with the UAV and machines operating at the worksite (see ¶ [0067]).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to use the transformed/aligned UAV mapping of Wang ‘821 to update the mine-worksite model of the combination of Wang ’821 and Wang ’577 because Wang ’821 explicitly teaches that SLAM is used to construct and update a real-time environment map and further teaches combining UAV-generated mapping information with mapping information obtained from a ground vehicle, thereby maintaining an updated and more comprehensive representation of the environment as additional environmental scanning data are acquired.
As per claim 5, the combination of Wang ’821 and Wang ‘577 teaches the system as stated above. Wang ‘821 further teaches that the UAV’s position may be defined relative to the ground vehicle or as an absolute position in the world coordinate system (see ¶ [0078]). More particularly, Wang ‘821 teaches that UAV may provide its positions as GNSS coordinates or as real0time relative position obtained through combining the initial position and the real-time movement state of the aerial vehicle (see ¶ [0080]).
Thus, Wang ‘821 teaches defining UAV location/mapping information based upon an initial/source position together with subsequent UAV movement. In combination with Wang ‘821’s teaching of converting mapping information into the common/world coordinate system, it would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to define the initial/source UAV position in the common worksite coordinate system and determine subsequent coordinates based upon movement relative to that initial position because Wang ‘821 explicitly teaches deriving UAV position from its initial position and real-time movement, thereby allowing UAV mapping information acquired during movement to be referenced to the common worksite coordinate system.
As per claim 6, the combination of Wang ’821 and Wang ‘577 teaches the system as stated above.
Wang ’821 further teaches that UAVs carry computing devices and sensors enabling them to implement Simultaneous Localization and Mapping (SLAM) and that UAV scans its environment using cameras, LiDAR, or other scanning sensors while flying and constructs a localization map (see ¶ [0073]). Wang ’821 explains that SLAM simultaneously constructs/updates the map and maintains the UAV’s location within the map (see ¶ [0074]). Wang ’821 further explicitly states that the aerial vehicle may map its environment using SLAM techniques (see ¶ [0078]).
Accordingly, Wang ’821 explicitly teaches that the first UAV mapping data is defined by a SLAM unit/function of the UAV, as required by claim 6.
As per claim 7, the combination of Wang ’821 and Wang ‘577 teaches the system as stated above. Wang ’821 further teaches multiple possible locations for performing the mapping processing. Th UAV contains computing devices and performs mapping/SLAM onboard (see ¶¶ [00073]-[0078]), while UAV scanning data and maps may alternatively be transmitted to the ground vehicle for processing and combination with the ground-vehicle map (see ¶¶ [0076] and [0078]). Wang ’821 therefore, teaches both the claimed UAV implementation and the claimed vehicle implementation.
Wang ’821 further teaches transmission of the UAV map/mapping data to the ground vehicle and transmission of the combined map back to UAV (see ¶¶ [0076] and [0078]).
As per claims 8 and 12, the combination of Wang ’821 and Wang ‘577 teaches the system as stated above. Wang ’821 further teaches that UAV transmits its map to the ground vehicle, which combines the UAV map with the map generated by the ground vehicle (see ¶ [0078]). Thus, Wang ’821 teaches generating fourth mapping data based on environment scanning by the vehicle and adding both the UAV-derived mapping data and vehicle-derived data into a common model. Wang ’821 further teaches that the system uses a server (see ¶ [0040]).
Allowable Subject Matter
Claim 4 is objected to as being dependent upon a rejected base claim, but would
be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 4 distinguish over the prior art of record because none of the prior art of record teaches or fairly suggest an apparatus for mining worksite mapping, comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: perform, on the basis of the first point cloud data portion and worksite model point cloud data, point cloud matching operation to detect a second point cloud data portion of the worksite model point cloud data best fitting with the first point cloud data portion; and define the transformation function by processing differences of coordinate values of the first point cloud data portion to associated coordinate values of the second point cloud data portion, in combination with the rest of the claim limitation as claimed and defined by the applicant.
Prior art
The prior art made record and not relied upon is considered pertinent to applicant’s
disclosure:
Sagalovich [‘632] discloses an autonomous guidance system can include an autonomous vehicle that utilizes both GNSS RTK data and SLAM data to navigate an outdoor environment. In some embodiments, the autonomous vehicle includes a GNSS antenna and at least one SLAM component. In some embodiments, the GNSS antenna is used in connection with a base station and at least one satellite. In some embodiments, the autonomous guidance system interacts with a mobile device associated with an operator.
Staab et al. [‘722] discloses generating a 3D point cloud and registering the 3D point cloud to the surface of the Earth (sometimes called “geo-locating”). A method can include capturing, by unmanned vehicles (UVs), image data representative of respective overlapping subsections of the object, registering the overlapping subsections to each other, and geo-locating the registered overlapping subsections.
Appelman et al. [‘752] discloses a system and method of navigating a vehicle, the vehicle comprising a scanning device and a self-contained navigation system (SCNS) operatively connected to a computer, the method comprising: operating the scanning device for repeatedly executing a scanning operation, each operation includes scanning an area surrounding the vehicle, thereby generating respective scanning output data; operating the computer for generating, based on the scanning output data, a relative map representing at least a part of the area, the map having known dimensions and being relative to a position of the vehicle, wherein the map comprises cells, each cell classified to a class from at least two classes, comprising traversable and non-traversable, and characterized by dimensions equal or larger than an accumulated drift value of the SCNS; wherein non-traversable cells correspond to identified obstacles; receiving SCNS data and updating a position of the vehicle relative to the cells based on the SCNS data.
Contact information
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to MOHAMED CHARIOUI whose telephone number is (571)272-2213. The examiner can normally be reached Monday through Friday, from 9 am to 6 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Schechter can be reached on (571) 272-2302. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Mohamed Charioui
/MOHAMED CHARIOUI/Primary Examiner, Art Unit 2857
Prosecution Insights