DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments, see the section titled “A. Claim Rejections under 35 U.S.C. 103” starting on page 10 of the reply filed 17/773,018 have been fully considered but they are not persuasive. Applicant argues that the prior art or the cited references discloses the content of amended claim 1 in its entirety; however, the Examiner disagrees.
In the first paragraph beginning on page 13 of the reply filed 07/15/2025, Applicant argues that “Since the solution of Claim 1 now utilizes only point locations of one to two reference points pre-determined in the worksite coordinate system, it must also simultaneously utilize sufficient amounts of sensor data to be able to determine three-dimensional location and orientation of the machine and, thereafter, the working tool of the machine”; however, the Examiner disagrees. The Examiner understands that the claim language does not limit the utilization of only point locations of one to two reference points. For example, the claim states “when the single tracking apparatus is set external from the machine, the single tracking apparatus acquires data by tracking both the one to two reference point locations and the locations of the at least three marker points” where the Examiner opines that the limitation requires at most that a set of one to two reference point locations are tracked.
Furthermore, it is the Examiner’s opinion that Friend discloses when the single tracking apparatus is set external from the machine, the single tracking apparatus acquires data by tracking both the one to two reference point locations and the locations of the at least three marker points (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); see also paragraphs [0039-0044] where Friend discloses examples of loading machines, such as loading machines 180 and 181 (machine), including one or more fiducials mounted on excavator 400, such as fiducials 430a, 430b, 430c, 430d, 430e, 430f, and 430g, or one or more fiducials mounted on wheel loader 550, such as fiducials 580a, 580b, 580c, and 580d). The Examiner further notes that in paragraph [0046] and fig. 6, Friend depicts two fiducials 631a and 631b of reference location 630 in ana example.
Similar reasoning is applied to Applicant’s arguments in the third paragraph beginning on page 16 of the reply filed 07/15/2025
In the second paragraph beginning on page 17 of the reply filed 07/25/2025, Applicant further argues that “Friend does not disclose or reasonably suggest utilizing a single tracking apparatus to track one to two reference points and utilize sufficient amount of sensor data regarding inclination and heading of one of the tracking apparatus or the machine, and definitely does not disclose or reasonably suggest utilizing environment modelling apparatus for acquiring data to determine location and orientation of a machine and its working tool in the worksite coordinate system”; however, the Examiner disagrees.
It is the Examiner’s opinion that Friend discloses at least when the single tracking apparatus is set external from the machine, the single tracking apparatus acquires data by tracking both the one to two reference point locations and the locations of the at least three marker points (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); see also paragraphs [0039-0044] where Friend discloses examples of loading machines, such as loading machines 180 and 181 (machine), including one or more fiducials mounted on excavator 400, such as fiducials 430a, 430b, 430c, 430d, 430e, 430f, and 430g, or one or more fiducials mounted on wheel loader 550, such as fiducials 580a, 580b, 580c, and 580d); wherein
when the single tracking apparatus tracks one reference point location with respect to the single tracking apparatus, the at least one sensor acquires inclination and heading data of one of: the tracking apparatus or the machine (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); in paragraph [0033], Friend discloses that processor 130 may be configured to use information obtained from IMU 170 for determining changes in position and orientation of haul machine 110 between determinations of position and orientation using information obtained from perception device 120, which may be able at a much lower rate than the IMU information; see also paragraph [0033] where Friend discloses that the inertial measurement unit (IMU) 170 typically comprises three orthogonally oriented accelerometers for detecting changes in position, three orthogonally oriented gyroscopes for detecting changes in orientation, and, in some cases, three orthogonally oriented magnetometers serving as a three-dimensional electronic compass),
when the single tracking apparatus tracks two reference point locations with respect to the single tracking apparatus, the at least one sensor acquires at least one of: inclination or heading of one of: the tracking apparatus or the machine (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); in paragraph [0033], Friend discloses that processor 130 may be configured to use information obtained from IMU 170 for determining changes in position and orientation of haul machine 110 between determinations of position and orientation using information obtained from perception device 120, which may be able at a much lower rate than the IMU information; see also paragraph [0033] where Friend discloses that the inertial measurement unit (IMU) 170 typically comprises three orthogonally oriented accelerometers for detecting changes in position, three orthogonally oriented gyroscopes for detecting changes in orientation, and, in some cases, three orthogonally oriented magnetometers serving as a three-dimensional electronic compass), and
executing designed operations by the working tool of the machine in the worksite based on the determined location and orientation of both the machine and the working tool of the machine in the worksite coordinate system (In paragraph [0040], Friend discloses that a common use pattern for excavator 400 is to place excavator 400 at a fixed position for a period of time while performing multiple sequences of digging with and filling bucket 420 and depositing the contents of the bucket 420 into a haul machine at a destination location for receiving materials from excavator 400; in paragraph [0056], Friend discloses, for example, that processor 130 assists with or automatically guides haul machine 110 toward the desired destination location or position; see also paragraphs [0059] and [0061], where Friend discloses that the desired destination position or location may be automatically identified or where processor 130 may be configured to automatically stop haul machine 110 if, via the information obtained from nearby fiducials or other sources, processor 130 anticipates haul machine 110 may collide with something, such as a loading machine associated with the desired destination location).
In the final paragraph beginning on page 17 of the reply filed 07/15/2025, Applicant further argues that “The skilled person would not turn to Yao or Clark since Friend already provides a solution to determine location and orientation of a machine in a worksite by tracking the three-dimensional reference fiducial. The solution of Friend is such that it needs the three-dimensional fiducial and, thus, a smaller amount of reference data than the three-dimensional reference fiducial in Friend cannot be fixed with additional sensor data regarding inclination and/or heading of the machine or the perception device.” However, as expressed above, it is the Examiner’s opinion that the claim language may be broader than intended by Applicant, and that the language of amended claim 1 does not require “a smaller amount of reference data” than is disclosed by Friend.
Furthermore, the Examiner understands the teachings of Yao and Clark to be motivated at least as follows:
Yao is considered to be analogous to the claimed invention in that they both pertain to obtaining sensor data at a same time instant to predict the state of an apparatus and its environment at the same time instant. It would be obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Yao with the method as disclosed by Friend, wherein obtaining the sensor data at the same time instant as taught by Yao is advantageous in that it ensures the sensor data being used to determine the position and orientation are obtained simultaneously, maintaining the relative accuracy of the measurements to each other, for example, thereby increasing the accuracy of the determinations which utilize the measurements.
Clark is considered to be analogous to the claimed invention in that they both pertain to environment modelling including obtaining spatial data that is a surface profile of the ground. It would be obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Clark with the method as disclosed by the combination of Friend and Yao, where the ability to provide real-time data of the machine gives a redundant check and validation of three dimensional machine control system performance, and where using this data in real time gives the machine control system the ability to fine tune the automatic controls to achieve required tolerances in the generated surface, as suggested by Clark in paragraph [0018], which may advantageously increase the contextual accuracy of operation of the machine, for example.
Therefore, the grounds of rejection are maintained. See the rejections below.
Claim Objections
Claim 1 is objected to because of the following minor informality: “the at least three marker points are attached to the machine and has locations determined in the machine coordinate system” appears as if it should instead read “the at least three marker points are attached to the machine and have locations determined in the machine coordinate system” or equivalent.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-6, 8-9, 11-14, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Friend (US 2014/0146167 A1), in view of Yao (US 2019/0102668 A1) and Clark (US 2010/0046800 A1).
Regarding claim 1, Friend discloses a method for determining situational awareness in a worksite, the method comprising:
setting a single tracking apparatus at least one of: external from the machine, when the single tracking apparatus has a location in the worksite coordinate system, for at least tracking both at least one reference point and at least three marker points, wherein the at least one reference point is in the worksite and has a location determined in the worksite coordinate system and the at least three marker points are attached to the machine and have locations determined in the machine coordinate system (In paragraph [0021], Friend discloses that a location may be a coordinate such as distances from a reference point that specifies where a machine is or may be situated at a mining site; the examiner understands the machine coordinate system to be at least a coordinate system for providing the relative locations of the haul machine 110 and loading machine 180; in paragraph [0022], Friend discloses fiducials 190a and 190b (marker points) on loading machine 180 (machine); in paragraph [0023], Friend discloses a haul machine 110 (tracking apparatus) including a perception device 120 and processor 130; in paragraph [0046], Friend discloses fiducials 631a and 631b (reference points) indicating a reference location 630 in a mining site 600 (worksite); see also paragraphs [0039-0044] where Friend discloses examples of loading machines, such as loading machines 180 and 181 (machine), including one or more fiducials mounted on excavator 400, such as fiducials 430a, 430b, 430c, 430d, 430e, 430f, and 430g, or one or more fiducials mounted on wheel loader 550, such as fiducials 580a, 580b, 580c, and 580d);
acquiring data by the single tracking apparatus by tracking reference point and marker point locations with respect to the single tracking apparatus (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus)), wherein
when the single tracking apparatus is set external from the machine, the single tracking apparatus acquires data by tracking both the one to two reference point locations and the locations of the at least three marker points (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); see also paragraphs [0039-0044] where Friend discloses examples of loading machines, such as loading machines 180 and 181 (machine), including one or more fiducials mounted on excavator 400, such as fiducials 430a, 430b, 430c, 430d, 430e, 430f, and 430g, or one or more fiducials mounted on wheel loader 550, such as fiducials 580a, 580b, 580c, and 580d);
acquiring data by at least one sensor (In paragraph [0033], Friend discloses that haul machine 110 may include an inertial measurement unit (IMU) 170, which typically comprises three orthogonally oriented accelerometers for detecting changes in position, three orthogonally oriented gyroscopes for detecting changes in orientation, and, in some cases, three orthogonally oriented magnetometers serving as a three-dimensional electronic compass), wherein
when the single tracking apparatus tracks one reference point location with respect to the single tracking apparatus, the at least one sensor acquires inclination and heading data of one of: the tracking apparatus or the machine (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); in paragraph [0033], Friend discloses that processor 130 may be configured to use information obtained from IMU 170 for determining changes in position and orientation of haul machine 110 between determinations of position and orientation using information obtained from perception device 120, which may be able at a much lower rate than the IMU information; see also paragraph [0033] where Friend discloses that the inertial measurement unit (IMU) 170 typically comprises three orthogonally oriented accelerometers for detecting changes in position, three orthogonally oriented gyroscopes for detecting changes in orientation, and, in some cases, three orthogonally oriented magnetometers serving as a three-dimensional electronic compass),
when the single tracking apparatus tracks two reference point locations with respect to the single tracking apparatus, the at least one sensor acquires at least one of: inclination or heading of one of: the tracking apparatus or the machine (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); in paragraph [0033], Friend discloses that processor 130 may be configured to use information obtained from IMU 170 for determining changes in position and orientation of haul machine 110 between determinations of position and orientation using information obtained from perception device 120, which may be able at a much lower rate than the IMU information; see also paragraph [0033] where Friend discloses that the inertial measurement unit (IMU) 170 typically comprises three orthogonally oriented accelerometers for detecting changes in position, three orthogonally oriented gyroscopes for detecting changes in orientation, and, in some cases, three orthogonally oriented magnetometers serving as a three-dimensional electronic compass),
receiving by at least one position determination unit
data acquired by the at least one sensor (In paragraph [0033], Friend discloses that processor 130 may be configured to use information obtained from IMU 170 for determining changes in position and orientation of haul machine 110 between determinations of position and orientation using information obtained from perception device 120, which may be able at a much lower rate than the IMU information),
data related to the single tracking apparatus comprising at least the data acquired by the single tracking apparatus (In paragraph [0025], Friend discloses that processor 130 (position determination unit) determines the location or position of a fiducial based on data from perception device 120; the examiner understands that the acquired data from the perception device 120 (of tracking apparatus) must at least be received by the processor 130 (position determination unit) to be used in the determination made by the processor 130), and
determining by the at least one position determination unit, based at least in part on the received data, the location and orientation of both the machine and a working tool of the machine in the worksite coordinate system (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses that processor 130 may be configured to obtain information from fiducial 631a (reference point) via perception device 120 for determining a position of mining truck 300 (tracking apparatus); see also paragraph [0021], where Friend defines “position” to be a location also having a respective orientation for a machine); and
executing designed operations by the working tool of the machine in the worksite based on the determined location and orientation of both the machine and the working tool of the machine in the worksite coordinate system (In paragraph [0040], Friend discloses that a common use pattern for excavator 400 is to place excavator 400 at a fixed position for a period of time while performing multiple sequences of digging with and filling bucket 420 and depositing the contents of the bucket 420 into a haul machine at a destination location for receiving materials from excavator 400; in paragraph [0056], Friend discloses, for example, that processor 130 assists with or automatically guides haul machine 110 toward the desired destination location or position; see also paragraphs [0059] and [0061], where Friend discloses that the desired destination position or location may be automatically identified or where processor 130 may be configured to automatically stop haul machine 110 if, via the information obtained from nearby fiducials or other sources, processor 130 anticipates haul machine 110 may collide with something, such as a loading machine associated with the desired destination location).
Friend does not explicitly disclose setting an environment modelling apparatus at least one of: on a machine, when the environment modelling apparatus has a location in a machine coordinate system, or external from the machine, when the environment modelling apparatus has a location in a worksite coordinate system, for at least detecting surrounding areas as spatial data of or relating to the worksite, wherein the spatial data is at least surface profile of a ground;
tracking at same time instants both at least one reference point and at least three marker points;
wherein the acquiring data by the single tracking apparatus of instants reference point and marker point locations with respect to the single tracking apparatus is at same time instants;
acquiring data by the environment modelling apparatus by tracking with respect to the environment modelling apparatus spatial data of or relating to the worksite, wherein the spatial data comprises a surface profile of a ground;
wherein the acquiring by the at least one sensor of inclination and heading data of one of: the tracking apparatus or the machine is at same time instant;
receiving by the at least one position determination unit data related to the environment modelling apparatus comprising at least the data acquired by the environment modelling apparatus; and
providing the tracked surface profile of the ground as as-built data as determined situational awareness in the worksite.
However, Yao teaches tracking at same time instants both at least one reference point and at least three marker points (In paragraph [0047], Yao discloses that a current state, s, includes not only a vehicle's [tracking apparatus] state but also includes the environment's state (e.g., measurement of the vehicle 105 with respective to the environment) at the same time, t, and for example the state, s, at time, t, includes: sensor data including current camera views, including: current images of all the cameras 112 installed on the vehicle 105 and other sensory measurements such as current GNSS data from the satellite receiver 132, current compass reading, current IMU reading, etc.; data derived from current and/or past sensor data including: current distance to other environmental references, etc.);
wherein the acquiring data by the single tracking apparatus of instants reference point and marker point locations with respect to the single tracking apparatus is at same time instants (In paragraph [0047], Yao discloses that a current state, s, includes not only a vehicle's [tracking apparatus] state but also includes the environment's state (e.g., measurement of the vehicle 105 with respective to the environment) at the same time, t, and for example the state, s, at time, t, includes: sensor data including current camera views, including: current images of all the cameras 112 installed on the vehicle 105 and other sensory measurements such as current GNSS data from the satellite receiver 132, current compass reading, current IMU reading, etc.; data derived from current and/or past sensor data including: current distance to other environmental references, etc.); and
wherein the acquiring by the at least one sensor of inclination and heading data of one of: the tracking apparatus or the machine is at same time instant (In paragraph [0047], Yao discloses that a current state, s, includes not only a vehicle's [tracking apparatus] state but also includes the environment's state (e.g., measurement of the vehicle 105 with respective to the environment) at the same time, t, and for example the state, s, at time, t, includes: sensor data including current camera views, including: current images of all the cameras 112 installed on the vehicle 105 and other sensory measurements such as current GNSS data from the satellite receiver 132, current compass reading, current IMU reading, etc.; data derived from current and/or past sensor data including: current distance to other environmental references, etc.).
Yao is considered to be analogous to the claimed invention in that they both pertain to obtaining sensor data at a same time instant to predict the state of an apparatus and its environment at the same time instant. It would be obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Yao with the method as disclosed by Friend, wherein obtaining the sensor data at the same time instant as taught by Yao is advantageous in that it ensures the sensor data being used to determine the position and orientation are obtained simultaneously, maintaining the relative accuracy of the measurements to each other, for example, thereby increasing the accuracy of the determinations which utilize the measurements.
The combination of Friend and Yao does not explicitly disclose setting an environment modelling apparatus at least one of: on a machine, when the environment modelling apparatus has a location in a machine coordinate system, or external from the machine, when the environment modelling apparatus has a location in a worksite coordinate system, for at least detecting surrounding areas as spatial data of or relating to the worksite, wherein the spatial data is at least surface profile of a ground;
acquiring data by the environment modelling apparatus by tracking with respect to the environment modelling apparatus spatial data of or relating to the worksite, wherein the spatial data comprises a surface profile of a ground;
receiving by the at least one position determination unit data related to the environment modelling apparatus comprising at least the data acquired by the environment modelling apparatus; and
providing the tracked surface profile of the ground as as-built data as determined situational awareness in the worksite.
However, Clark teaches setting an environment modelling apparatus at least one of: on a machine, when the environment modelling apparatus has a location in a machine coordinate system, or external from the machine, when the environment modelling apparatus has a location in a worksite coordinate system, for at least detecting surrounding areas as spatial data of or relating to the worksite, wherein the spatial data is at least surface profile of a ground (In paragraphs [0018-0019], Clark teaches update and mapping a three dimensional worksite surface dynamically from the earthmoving machine while it is being operated over a worksite, wherein to derive a dynamic, three-dimensional terrain model, the system includes a pair of scanners 12 and 14 that are mounted on the machine 16);
acquiring data by the environment modelling apparatus by tracking with respect to the environment modelling apparatus spatial data of or relating to the worksite, wherein the spatial data comprises a surface profile of a ground (In paragraphs [0018-0019], Clark teaches update and mapping a three dimensional worksite surface dynamically from the earthmoving machine while it is being operated over a worksite, wherein to derive a dynamic, three-dimensional terrain model, the system includes a pair of scanners 12 and 14 that are mounted on the machine 16);
receiving by the at least one position determination unit data related to the environment modelling apparatus comprising at least the data acquired by the environment modelling apparatus (In paragraphs [0018-0019], Clark teaches update and mapping a three dimensional worksite surface dynamically from the earthmoving machine while it is being operated over a worksite, wherein to derive a dynamic, three-dimensional terrain model, the system includes a pair of scanners 12 and 14 that are mounted on the machine 16); and
providing the tracked surface profile of the ground as as-built data as determined situational awareness in the worksite (In paragraphs [0024-0025], Clark teaches that a worksite surface behind the machine is scanned at 52 and the three-dimensional data is stored at 56; see also paragraphs [0021-0022] where Clark teaches that the terrain model may also be displayed to the operator of the machine 16 on display 30).
Clark is considered to be analogous to the claimed invention in that they both pertain to environment modelling including obtaining spatial data that is a surface profile of the ground. It would be obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Clark with the method as disclosed by the combination of Friend and Yao, where the ability to provide real-time data of the machine gives a redundant check and validation of three dimensional machine control system performance, and where using this data in real time gives the machine control system the ability to fine tune the automatic controls to achieve required tolerances in the generated surface, as suggested by Clark in paragraph [0018], which may advantageously increase the contextual accuracy of operation of the machine, for example.
Regarding claim 3, Friend further discloses determining at least one of: accuracy level or validity of the determined location and orientation of the machine in the worksite (In paragraph [0032], Friend discloses that processor 130 may be configured to base its location determination on location information received from geolocation unit 160, although processor 130 may be configured to not use location information received from geolocation unit 160 if its precision or accuracy fail to meet predetermined requirements).
Regarding claim 4, Friend further discloses wherein the data related to the single tracking apparatus further comprises at least one of: inclination angle of the tracking apparatus, heading of the tracking apparatus, us, location and orientation of the tracking apparatus in at least one of: a machine coordinate system, the worksite coordinate system or a world coordinate system or at least one of: accuracy level or validity of at least one of the previous (In paragraph [0021], Friend discloses that a "position," in this disclosure, is a location also having a respective orientation for a machine; in paragraph [0032], Friend discloses that processor 130 may be configured to base its location determination on location information received from geolocation unit 160, although processor 130 may be configured to not use location information received from geolocation unit 160 if its precision or accuracy fail to meet predetermined requirements).
Regarding claim 5, Clark further teaches wherein the data related to the at least one environment modelling apparatus is further at least one of: inclination angle of the environment modelling apparatus, heading of the environment modelling apparatus, location and orientation of the environment modelling apparatus in at least one of: a machine coordinate system, the worksite coordinate system or a world coordinate system (In paragraph [0009], Clark teaches that The scanning arrangement may include one or more GNSS receivers carried on machine for determining the position and orientation of the machine, a laser transmitter which projects a reference beam of laser light, and one or more laser light detectors on the machine to detect the reference beam of light wherein the position of the one or more laser light detectors may be determined with respect to the laser transmitter and the position and orientation of the pair of scanners may be determined, an inertial guidance system for determining the position and orientation of the machine including the position and orientation of the pair of scanners, or one or more inclinometers for determining the orientation of the machine).
Regarding claim 6, Friend further discloses wherein the spatial data comprises at least one of: pictorial data, point cloud data or data with implicit or explicit reference to a location relative to at least one of: the worksite or the Earth (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 (of haul machine 110) identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0046], Friend discloses fiducials 631a and 631b (reference points) indicating a reference location 630 in a mining site 600 (worksite); see also paragraph [0032] where Friend discloses a geolocation unit 160, from which processor 130 may obtain an absolute position, typically including latitude, longitude, and altitude, of haul machine 110 on the surface of the Earth).
Regarding claim 8, Clark further teaches receiving by the at least one environment modelling unit
one or more indications relating to at least one of: work phase or work stage of respective areas (In paragraphs [0024-0025], Clark teaches that a worksite surface behind the machine is scanned at 52 and the three-dimensional data is stored at 56, wherein a desired surface contour is compared at 62 with the actual worksite surface contour behind the machine, and those areas of the worksite surface that require further work are noted and stored at 64), and
data related to the at least one environment modelling apparatus (In paragraphs [0024-0025], Clark teaches that a worksite surface behind the machine is scanned at 52 and the three-dimensional data is stored at 56), and wherein
by taking into account the one or more indications
it is derived from the data related to the at least one environment modelling apparatus a georeferenced spatial data of respective areas (In paragraphs [0024-0025], Clark teaches that a worksite surface behind the machine is scanned at 52 and the three-dimensional data is stored at 56, wherein a desired surface contour is compared at 62 with the actual worksite surface contour behind the machine, and those areas of the worksite surface that require further work are noted and stored at 64; see also paragraph [0026] where Clark teaches that GPS/GNSS receivers may be used on the machine 16 to determine machine position and orientation); and
saved at least in part the georeferenced spatial data (In paragraphs [0024-0025], Clark teaches that a worksite surface behind the machine is scanned at 52 and the three-dimensional data is stored at 56, wherein a desired surface contour is compared at 62 with the actual worksite surface contour behind the machine, and those areas of the worksite surface that require further work are noted and stored at 64).
Regarding claim 9, Clark further teaches wherein the step of saving at least in part the georeferenced spatial data further comprises:
determining, based at least in part on the data received from the at least one environment modelling apparatus, the areas the georeferenced spatial data of which is to be saved (In paragraphs [0024-0025], Clark teaches that a worksite surface behind the machine is scanned at 52 and the three-dimensional data is stored at 56, wherein a desired surface contour is compared at 62 with the actual worksite surface contour behind the machine, and those areas of the worksite surface that require further work are noted and stored at 64); and
saving the georeferenced spatial data of the areas determined to be saved (In paragraphs [0024-0025], Clark teaches that a worksite surface behind the machine is scanned at 52 and the three-dimensional data is stored at 56, wherein a desired surface contour is compared at 62 with the actual worksite surface contour behind the machine, and those areas of the worksite surface that require further work are noted and stored at 64).
Regarding claim 11, Friend further discloses resolving, by the at least one position determination unit which determined the location and orientation of the machine and the working tool of the machine in the worksite coordinate system, data regarding at least one of: a tracking apparatus, an environment modelling apparatus, an object or another machine (In paragraph [0025], Friend discloses that processor 130 (position determination unit) determines the location or position of a fiducial based on data from perception device 120; the examiner understands that the acquired data from the perception device 120 (of tracking apparatus) must at least be received by the processor 130 (position determination unit) to be used in the determination made by the processor 130); and
transmitting the data resolved at least one of:
as part of data related to the respective tracking apparatus, environment modelling apparatus, object or another machine (In paragraph [0022], Friend discloses fiducials 190a and 190b (marker points) on loading machine 180 (machine); in paragraph [0023], Friend discloses a haul machine 110 (tracking apparatus) including a perception device 120 and processor 130; in paragraph [0046], Friend discloses fiducials 631a and 631b (reference points) indicating a reference location 630 in a mining site 600 (worksite)); or
as data receivable by at least one position determination unit (In paragraph [0025], Friend discloses that processor 130 (position determination unit) determines the location or position of a fiducial based on data from perception device 120; the examiner understands that the acquired data from the perception device 120 (of tracking apparatus) must at least be received by the processor 130 (position determination unit) to be used in the determination made by the processor 130).
Regarding claim 12, Friend further discloses initializing the single tracking apparatus by
determining location and orientation of the tracking apparatus in at least one of: a machine coordinate system or the worksite coordinate system if the tracking apparatus is set on at least one of: another machine (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 (of haul machine 110) identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine)); and
determining location and orientation of the tracking apparatus in the worksite coordinate system if the tracking apparatus is set external from the machine (In paragraphs [0023-0025], Friend discloses that perception device 120 and processor 130 (of haul machine 110) identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine)).
Regarding claim 13, Clark further teaches initializing the environment modelling apparatus by
determining location and orientation of the environment modelling apparatus in at least one of: a machine coordinate system or the worksite coordinate system if the environment modelling apparatus is set on at least one of: the machine or another machine (In paragraph [0009], Clark teaches that The scanning arrangement may include one or more GNSS receivers carried on machine for determining the position and orientation of the machine, a laser transmitter which projects a reference beam of laser light, and one or more laser light detectors on the machine to detect the reference beam of light wherein the position of the one or more laser light detectors may be determined with respect to the laser transmitter and the position and orientation of the pair of scanners may be determined, an inertial guidance system for determining the position and orientation of the machine including the position and orientation of the pair of scanners, or one or more inclinometers for determining the orientation of the machine), and
determining the location and orientation of the environment modelling apparatus in the worksite coordinate system if the environment modelling apparatus is set external from the machine or another machine (In paragraph [0009], Clark teaches that The scanning arrangement may include one or more GNSS receivers carried on machine for determining the position and orientation of the machine, a laser transmitter which projects a reference beam of laser light, and one or more laser light detectors on the machine to detect the reference beam of light wherein the position of the one or more laser light detectors may be determined with respect to the laser transmitter and the position and orientation of the pair of scanners may be determined, an inertial guidance system for determining the position and orientation of the machine including the position and orientation of the pair of scanners, or one or more inclinometers for determining the orientation of the machine).
Regarding claim 14, Clark further teaches wherein the determined situational awareness is further at least one of spatial data (In paragraphs [0018-0019], Clark teaches update and mapping a three dimensional worksite surface dynamically from the earthmoving machine while it is being operated over a worksite, wherein to derive a dynamic, three-dimensional terrain model, the system includes a pair of scanners 12 and 14 that are mounted on the machine 16).
Regarding claim 17, Friend further discloses wherein the data related to at least one of: the single tracking apparatus further comprises at least one of: data from sensors installed on the respective apparatus, data from sensors installed on an attachment point of the apparatus, data resolved by at least one of: any position determination unit or any apparatus at least one of: by tracking the respective apparatus or as a result of any calculations relating to the respective apparatus or at least one of: accuracy level or validity of at least one of the previous (In paragraphs [0023-0025], Friend discloses that perception device 120 such as a camera, LIDAR, or hybrid system, and processor 130 (of haul machine 110) identifies the presence of at least one fiducial 190a in the field of view and determines a relative distance and orientation to the fiducial 190a (marker point) and/or loading machine 180 (machine); in paragraph [0032], Friend discloses that processor 130 may be configured to base its location determination on location information received from geolocation unit 160, although processor 130 may be configured to not use location information received from geolocation unit 160 if its precision or accuracy fail to meet predetermined requirements).
Regarding claim 18, Friend further discloses wherein
if the single tracking apparatus is set on the worksite and if it comprises a tracking device for tracking the location of the tracking apparatus with one or more GNSS antennas (In paragraph [0032], Friend discloses a geolocation unit 160, from which processor 130 may obtain an absolute position, typically including latitude, longitude, and altitude, of haul machine 110 on the surface of the Earth such as GPS),
the single tracking apparatus further comprises at least one of: a camera, a stereocamera, or a lidar as a tracking device (In paragraphs [0023-0025], Friend discloses that perception device 120 such as a camera, LIDAR, or hybrid system).
Regarding claim 19, Friend further discloses wherein determination, by the at least one position determination unit, of the location and orientation of the machine in the worksite is additionally based at least in part on data received from one or more sensors installed on at least one of: another machine; wherein the sensors comprise at least one of: position, orientation, inclination, heading or distance travelled of at least one of: the machine or another machine (In paragraph [0032], Friend discloses a geolocation unit 160, from which processor 130 may obtain an absolute position, typically including latitude, longitude, and altitude, of haul machine 110 on the surface of the Earth such as GPS; in paragraphs [0023-0025], Friend discloses that perception device 120 such as a camera, LIDAR, or hybrid system).
Claims 2 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Friend (US 2014/0146167 A1), Yao (US 2019/0102668 A1), and Clark (US 2010/0046800 A1), in view of Kamat (US 2015/0168136 A1).
Regarding claim 2, although in paragraph [0021], Friend discloses that a "position," in this disclosure, is a location also having a respective orientation for a machine, the combination of Friend, Yao, and Clark does not explicitly disclose determining by the at least one position determination unit, based at least in part on the received data, at least one of: direction of travel or alternative direction of travel of the machine in the worksite.
However, Kamat teaches determining by the at least one position determination unit, based at least in part on the received data, at least one of: direction of travel or alternative direction of travel of the machine in the worksite (In paragraph [0039], Kamat teaches mathematical representations of 3D poses of the excavator's 10 different articulated components including matrices for position, yaw, pitch, and roll for the crawlers 19 of the excavator 10; in paragraph [0025], Kamat teaches that the base 12 of the excavator 10 moves forward and backward via its crawlers 19).
Kamat is considered to be analogous to the claimed invention in that they both pertain to determining a direction of travel for the machine. It would be obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Kamat with the method as disclosed by the combination of Friend, Yao, and Clark, where doing so may be advantageous in that the 3D pose of the particular articulated components of the machine can be understood relative to each other as well as to the coordinate system, improving the contextual accuracy of the pose determination for example.
Regarding claim 20, Friend further discloses wherein the machine is an excavator (In paragraph [0022], Friend discloses that examples of loading machine 180 include, but are not limited to, excavators).
The combination of Friend, Yao, and Clark does not explicitly disclose wherein determination, by the at least one posit