DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
2. The Amendment filled 08/05/2026 in response to Non-Final Office Action mailed 05/05/2026 has been entered.
3. Claims 1-20 are currently pending.
Response to Arguments
4. Applicant’s arguments, see pg. 6-8 have been fully considered and are persuasive. Specifically, the examiner notes that arguments presented with regard to the amended claim language are convincing, specifically regarding Lautala failing to specifically disclose wherein decimation is performed extending outwardly from the utility object and wherein the decimation comprises a limitation of a number of points per grid square within the predetermined area around the utility object. Therefore, the 103 rejections of claims 1, 4-7, 10-11, 16, and 19 have been withdrawn. However, upon further consideration, a new grounds of rejection is made in view of U.S. Publication No. 2022/0414362 to Lautala et al. (hereinafter Lautala), in view of “View-Dependent Dynamic Point Cloud Compression” to Zhu et al. (hereinafter Zhu), and further in view of U.S. Publication No. 2023/0419659 to Uljanovs et al. (hereinafter Uljanovs).
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
6. Claims 1, 4-7, 10-11, 16, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2022/0414362 to Lautala et al. (hereinafter Lautala) and further in view of “View-Dependent Dynamic Point Cloud Compression” to Zhu et al. (hereinafter Zhu), and further in view of U.S. Publication No. 2023/0419659 to Uljanovs et al. (hereinafter Uljanovs).
7. Regarding Claim 1, Lautala discloses a method for dynamically capturing and processing utility infrastructure LIDAR data for improved assessment accuracy and processing efficiency, the method comprising ([par. 0049, ln. 1-18] “The present disclosure provides a method for optimizing image data for generating orthorectified image(s) related to an area of interest in an environment… generally, existing raw aerial imagery captured by aerial vehicles (such as satellites, aeroplanes, helicopters, drones, etc.) contains distortions (or image distortions) induced due to several reasons including, but not limited to, sensor orientation, topographical variation, curvature of the earth and so forth… Generally, the orthorectified image is a correctly identified and/or located image, wherein the pixels of the image to be orthorectified using the method are rearranged (or relocated) to accurately place the displaced pixels to their correct position.”):
determining an orientation of a mobile LIDAR system ([par. 0051, ln. 1-8] “…the image data comprises at least one of the list of timestamps, locations, and orientations for each image taken by the aerial vehicle during a given time range for capturing the image data.”);
dynamically capturing, with the mobile LIDAR system, LIDAR data corresponding to a region of interest by collecting LIDAR samples based on the orientation and the region of interest ([par. 0053, ln. 1-33] “…the aerial vehicle is configured to perform three-dimensional movements for capturing the image data from different angles and orientations with respect to the area of interest to enable the system to track or monitor the environment... Typically, the control signal comprises data related to the trajectory or path to be followed by the aerial vehicle to capture the first image dataset related to the area of interest in the environment based on the method.”, [par. 0054, ln. 1-9] “…the image data comprises at least one of selected from: Light Detection and Ranging (LiDAR) data, hyperspectral imaging (HSI) data, Global Navigation System Satellite (GNSS) data, Inertial Measurement Unit (IMU) data… Optionally, the LiDAR unit… is mounted on an aerial vehicle that is employed for capturing a given LiDAR dataset of the environment.”);
identifying a utility object based at least in part on the LIDAR samples ([Fig. 4 and 5], [par 0067, ln. 12-25] “…Herein, the method comprises pre-identifying the non-interesting areas i.e., the locations and/or objects that do not match the need and/or out of the scope of implementation… in a powerline maintenance setup, only the area (for example, including a distance of up to 100 m from the powerlines) surrounding the powerlines comprise the area of interest and all the other objects and areas in the environment are considered to be non-interesting areas. This is done so as to segregate the area of interest from the non-interesting areas in the environment, and selectively process data pertaining to only the area of interest at a high resolution. This reduces the data storage space required, data storage costs incurred, data transfer time and processing time.”, [par. 0069, ln. 1-12] “The method comprises receiving attribute information related to each of the multiple identified objects. Upon identifying each of the multiple objects in the area of interest, the method comprises receiving attribute information related to each of the multiple identified objects. The term “attribute information” refers to the qualitative and/or quantitative characteristic information that may be recorded and/or analysed associated with each of the multiple identified objects. Examples of attribute information include, but is not limited to, location of object, dimensions of object, colour of object, texture of object, type of object, growth rate of object, and so forth.”);
decimating the LIDAR samples within a predetermined area {extending outwardly from the utility object, the decimated LIDAR samples comprising LIDAR samples indicative of the utility object, wherein the decimating comprises limiting a number of points per grid square within the predetermined area}, ([par 0067, ln. 12-25], [par. 0068, ln. 1-20] “Further… upon pre-identifying the one or more non-interesting areas, the aerial vehicle is configured to capture a relatively lower resolution image for the non-interesting areas while capturing the plurality of images of the area of interest, to reduce the size of the first image dataset... the capturing of low-resolution images for non-interesting areas is applied to a digital elevation model (DEM) of the environment to reduce the DEM complexity in the non-interesting areas for the given capture and thus to reduce the overall size of the first image dataset. The reduction in overall size of the first image data results in reduced memory storage, memory storage costs, associated computation time and power consumption.”, [par. 0073, ln. 1-34] “The method comprises processing the first image dataset to either discard or down-sample areas other than the maximum relevant second area captured therein, to obtain a second image dataset. In an example, the method employs the GNSS data, IMU data, and LiDAR data to determine areas required to be down-sampled or discarded such as areas wherein the image data is captured while the aerial vehicle flies along a tight curve, having high speeds or lying outside the area of interest. Typically, the image data captured under extreme conditions such as tight curves, high speeds, high vibrations (vibrations typically seems to happen when the aerial vehicle takes a hard curve and/or accelerates), high roll/pitch angle (camera pointing to a different location) are considered to be non-interesting or having a bad quality, and thus may be discarded or down-sampled to reduce the size of the image data. Typically, in an implementation scenario, the non-interesting areas i.e., the areas in the environment other than the area encompassed by the maximum relevant second area are either discarded from the first dataset or down-sampled to obtain the second image dataset. For example, the areas in the environment apart from the maximum relevant second area are captured at a lower resolution relative to the captured images in the second image dataset. Beneficially, by processing the first image dataset to either discard or down-sample the non-interesting areas other than the maximum relevant second area captured therein, the overall size of the image dataset to be processed is reduced drastically. Herein, the reduction in the overall size of the dataset results in reduced memory storage costs, associated computation time and power consumption. As a result of the reduced computation time, the reduced memory and power consumption, the present method provides faster and efficient generation of orthorectified image(s).”);
generating depth-encoded {multi-perspective} images of the utility object using the decimated LIDAR samples ([par. 0054, ln. 14-21] “The plurality of data points represent objects (such as, buildings, vegetation, and the like) on and above a ground surface in a three-dimensional space of the environment. Optionally, location of a given data point is expressed as (x, y, z) coordinates along X, Y, and Z axes, respectively of a given coordinate system employed for the environment. Optionally, the plurality of data points is collectively referred to as point clouds.”, [par. 0053, ln. 1-33] see “…capturing the image data from different angles and orientations with respect to the area of interest …”); and
determining at least one condition of the utility object ([par. 0071, ln. 4-21] “In an exemplary scenario, an object such as a tree may be likely to fall onto one or more components of a power distribution infrastructure (such as, the poles and/or the hanging powerlines in the power distribution infrastructure) in proximity of said tree. If any tree falls onto the power distribution infrastructure, it could lead to disruption in delivering electric power and/or could cause fire due to a circuit break. In this regard, it is of critical importance that such risky trees (or objects) are identified and timely removed or trimmed in order to prevent damage and failure of the power distribution infrastructure, for operation of the power distribution infrastructure to be maintained reliably… digitally identifying the risky objects enables efficient management whilst ensuring that operation of the power distribution infrastructure is maintained. This facilitates reduction in cost of vegetation management, better vegetation management planning, and the like.”).
However, Zhu teaches wherein dynamically captured point cloud can encoded by generating depth-encoded multi-perspective images of an object ([pg. 766, Fig. 1], [pg. 766, col. 1, B. Our Approach, par. 1, ln. 1 to col. 2, par. 1, ln. 19] “Therefore, in this paper, we propose the view-dependent DPC compression for networked applications, as sketched in Fig. 1, a.k.a, View-PCC. In general, our work belongs to the category that applies the 3D-to-2D dimensional reduction and HEVC-based 2D video coding. Our focus is to devise high efficiency 3D-to-2D dimensional reduction via hybrid global and local projections to support view-dependent extraction (via partial stream decoding) and adaptation-based streaming. More specifically, we first perform the global orthographic projection to map the original 3D object onto four faces (e.g., front, back, left, right) of a cube, which is referred to as the multi-view projection for mimicking natural observation viewpoints from four different perpendicular orientations (see Fig. 2a). Different from the normal clustering based patch projection used in MPEG video-based point cloud compression (V-PCC) [9] where we could not directly derive individual view by decoding partial substreams because points or voxels belong to the same view may be dispersed among different projection planes due to their diverse normal attributes, our global projection method enables network friendly view-dependent streaming, by directly applying the perspective projection of 3D voxels to a specific image plane. To further minimize the projection loss, i.e., enforcing points captured by the image planes as many as possible, we have applied the multi-layer projection, where an extra image layer (e.g., geometry, and texture plane) of each view is generated by replacing the corresponding pixels at existing layer with their nearest and available neighbors along with the projection direction. Similar nearest neighbor replacement (NNR) strategy is also utilized across views. These would generally guarantee the completeness of each individual view when partially decoding associated streams.”, [pg. 769, Fig. 2, see depth encoded images in (b), mutli-view (a)], [pg. 769, col. 1, III. Multi-View Mutli-Layer Global Projection, par. 1, ln. 1 to col. 2, par. 2, ln. 11] “We have applied the parallel orthographic projection, where the projection lines are perpendicular to the image planes of a cube [45]. Here, we only use the front, back, right and left faces of a cube, to mimic the natural viewing behavior in reality, as shown in Fig. 2a. These cube faces correspond to multiple views or multi-view in practice that can be encoded independently and in parallel. Each view or viewpoint of a 3D point cloud can be well reconstructed by decoding associated partial streams of corresponding projected image plane. Dynamic navigation in 3D space, such as view switch, can be also easily supported by adapting the streams of corresponding image planes. Such adaptation can be implemented using chunk or segment-based transport protocols, such as the Dynamic Adaptive Streaming over HTTP (DASH) [46], MPEG Media Transport (MMT) [47], etc, where image plane sequences are compressed and encapsulated into time-aligned chunks or segments at a variety of quality (or bitrate) scales [48], [49] for efficient network streaming. Each view-dependent image plane captures the corresponding geometry and texture information projected from a 3D point cloud in Fig. 2a. Note that geometry and texture images are compressed separately. Plane oriented projection is fulfilled in a consecutive order of front, back, left and right. For a specific projection orientation, points in 3D point cloud that are the closest to plane, are captured; In practice, points may be already projected to previous planes. In order to maintain the complete and natural surface appearance, nearest neighbors will be chosen as the replacement, if they are available; otherwise, same points are retained for projection…”, [pg. 770, col. 2, A. Patch Re-Processing, par. 2, ln. 1 to par. 5, ln. 5] “2) Projection Plane Prioritization: In addition to the same four planes used for global projection, we introduce the bottom plane as the fifth one to host the local patches only.2 Total five projection planes Pk are applied with k from 0 to 4. Given the fact that total number of local points is far less than those global-projected ones, we have enforced the local projection to a single plane out of those five options. Since the symmetric characteristics of paired left-right and front back planes where left (or front) plane would have the same number of projected points as the right (or back) one, five planes are categorized into three classes, i.e., bottom, left-right and front-back. For any
V
j
, it leads to the prioritized plane list
P
→
j
, of which
P
→
j
(
0
)
is the optimal plane having the least loss, i.e., with the most projected points, or the least averaged Euclidean distance when first two planes having the same and the largest number of projected points. In practice, the optimal plane
P
→
j
(
0
)
may not be the one used for the final compression when taking the projection occlusion, temporal coherency into consideration. Thus,
P
→
j
will be used when patch re-arrangement is required. Note that the same procedure is iteratively processed for all ordered patches. After being projected to a certain plane
P
→
j
(
k
)
, a 3D patch
V
j
with voxel locations (
x
→
j
,
y
→
j
,
z
→
j
) having positive attributes, can be represented by
C
j
,
P
→
j
(
k
)
having 2D pixels at effective locations (
u
→
j
,
v
→
j
,
d
→
j
(
u
→
j
,
v
→
j
)
) with valid geometry and texture components.”, [pg. 774, Fig. 6, see depth-encoded images top]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize Lautala and Zhu as within the same field of image processing for dynamically captured 3D point clouds, and as analogous to the claimed invention. The motivation to combine is disclosed in Zhu, wherein it allows for efficient compression and streaming without loss of quality ([pg. 766, col. 1, par. 2, ln. 3-22] “we attack the DPC compression from an alternative angle. First, we assume the networked point cloud application perspective, where not only the signal redundancy in point cloud data, but also the network transmission priority, can be used for optimizing the rate distortion performance [22]. Second, such network transmission priority is motivated by the biological characteristics of human visual system (HVS) that our eyes can only perceive the media content within current viewport (or field of view- FoV) in the front, which is similar to view-dependent saliency especially for the visible part of the object [23]. Thus, in principle, we only need to stream the desirable view of the DPC per user’s request, by which the point cloud content consumption can be fulfilled by streaming view-dependent content appropriately. Thus, UnEqual Quality (UEQ)-based compression can be performed across views. For example, we could put high quality (e.g., high-bitrate) view within current viewport or FoV, but reduced or low-quality (e.g., low-bitrate) views elsewhere, for bandwidth efficient network streaming without quality of experience (QoE) loss…”, [pg. 767, col. 1, C. Contributions, par. 1, ln. 1-23] “In a short summary, the novelty parts of this work can be highlighted as follows. • View-dependent streaming methodology well fits the viewing characteristics of the HVS, and can be leveraged for significant network resource reduction but without QoE loss (e.g., applying UEQ scales across views). While this is often overlooked in typical DPC compression methods, such as the V-PCC [9]. • Our hybrid global and local projection mechanism applied in this work not only offers the high-efficiency DPC compression, but also enables view-dependent streaming and adaptation over the bandwidth-constrained network. • We offer the NNR based multi-view, multi-layer strategy for global projection, and patch-based local projection (e.g., intra displaced arrangement/inter temporal alignment), to maximize the number of points captured by the image planes for total reconstruction loss reduction, and improve the spatial and temporal coherency for better compression efficiency. • Though the HEVC is used to compress projected images of our View-PCC and V-PCC, our projection approach is fundamentally distant from the V-PCC, leading to very different processing schemes as detailed in Table I.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would find this particularly relevant to Lautala, wherein the dataset is transmitted in flight ([par. 0058, ln. 9-19] “Typically, the first image dataset is obtained by imaging via at least one imaging device of the aerial vehicle (such as, airborne or satellite sensors and/or drones comprising the at least one imaging device) on a target area i.e., the area of interest. Additionally, the first image dataset may be received directly from a local or remote system and/or database comprising the first image dataset or the image data. In an example, the data source is a drone or unmanned aerial vehicle (UAV) configured with a LiDAR and/or HSI device to capture and transmit the first image dataset.”), and thus would benefit from efficient compression, reduced loss of quality, and reduced network usage of Zhu by transmitting the LIDAR point cloud in an analogous fashion. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu through known means, with no change to their respective function, and the combination would have yielded nothing more than predictable results. Specifically, one of ordinary skill in the art would have combined Lautala with Zhu such that Lautala generated depth-encoded multi-perspective images as taught in Zhu for more efficient storage and transmission.
A combination of Lautala and Zhu does not specifically disclose decimating extending outwardly from the utility object, the decimated LIDAR samples comprising LIDAR samples indicative of the utility object, wherein the decimating comprises limiting a number of points per grid square within the predetermined area.
However, Uljanovs specifically teaches decimating LIDAR extending outwardly from the utility object ([Fig. 3C and 3D], [par. 0065, ln. 6-30] “… That is, in order to efficiently capture the geographical region… define each of the point-cloud data tiles to be of the predetermined size having the length and width… a buffer of up to 50% in either direction for at least one of the length and/or the width of the predetermined size by utilizing the sliding window technique to effectively process the entire point-cloud dataset by efficiently identifying or classifying the object(s) and capturing the spatial relationships therebetween. Herein, in an exemplary scenario of an electrical corridor, in order to capture the powerlines and towers while maintaining a small size (approximately, 10 megabytes (MB) at full density per tile), the predetermined size of 10 m×10 m was selected (since, training tests with 5 m and 20 m sizes resulted in deteriorated results), with a 5 m buffer (i.e., 50% of the width or length) in each direction such that no geographical relations are lost when training on the point-cloud-data. Further, the sliding window approach is adopted for capturing all spatial relationships within the point-cloud-data which fall within the 10 m range in the dataset. Notably, the 10 m×10 m tile configuration is more accurate than corresponding to 5×5 m and 20×20 m tiles, in terms of final accuracy and thus improves the efficiency of the method.”, [par. 0068, ln. 1-26] “…dividing each of the selected point-cloud data tiles into a plurality of voxels of a predetermined volume. That is, upon sampling the plurality of point-cloud data tiles based on the one or more pre-defined classes, the method includes dividing each of the selected point-cloud data tiles into a plurality of voxels having a predetermined volume. The “voxel” refers to a unit of graphical information defining a point in a three-dimensional space of the point-cloud data and represents a value (such as, location) in the three-dimensional space. Typically, each of the plurality of point-cloud data tiles are divided, based on the one or more pre-defined classes, into the plurality of voxels having a predetermined volume that may be varied based on the implementation. In an example, each voxel may have a volume of
125
c
m
3
(cubic-centimetres). In another example, each voxel may have a volume of
1000
c
m
3
. It will be appreciated that each of the one or more predefined classes may have a corresponding predetermined value defining the plurality of voxels. For example, the method includes selecting a pre-determined volume of
8000
c
m
3
for one of the one or more pre-defined classes indicative of an electric tower. Beneficially, dividing each of the selected point-cloud data tiles into a plurality of voxels of a predetermined volume is used to reduce memory consumption and to improve system performance.”, [par. 0072, ln. 1-32] “…the method further comprises normalizing the point-cloud data tiles using a Farthest Point Sampling (FPS) filter… to beneficially reduce the resolution of points across layers. As discussed, the point-cloud data tiles of a predetermined size (such as, 10 m×20 m and a buffer of 50% i.e., 5 m) having the predefined point-cloud density (such as,
0.008
m
3
per voxel) is utilized, and herein normalized using the FPS filter; wherein if M points are sampled, then each point is selected based on the most distance from the rest of the M−1 points in the point-cloud data tiles. Herein, the filtering may be done on-line or “on-the-fly” while the network is training. Since, FPS filtering is randomly initiated, every new epoch (period) would be slightly different, as a different set of 4096 points is sampled from the original tile. Beneficially, the 20 m×20 m ((5+10+5)×(5+10+5)) point-cloud data tiles with a density of one point per
0.008
m
3
voxel are filtered using the FPS algorithm, to normalise and reduce the number of points in each tile to 4096 points or fewer (in case, the original tile already contains fewer points). Notably, upon further analysis and experimentation of the FPS parameters, no significant effect in increasing the number of points (i.e., from 4096 to 8192) may be noticed, while the results may deteriorate by reducing the number of points (i.e., from 4096 to 2048), and as a result, 4096 is selected as the optimal number. Beneficially, such an implementation of the FPS filter via the method results in a more efficient method since the point-cloud data tiles are quickly sampled via increased GPU acceleration, and thus, novel set of points are obtained for each training epoch, while managing the size constraint.”), the decimated LIDAR samples comprising LIDAR samples indicative of the utility object, wherein the decimating comprises limiting a number of points per grid square within the predetermined area ([par. 0070, ln. 1-23] “…the method further comprises defining the predetermined volume for each of the plurality of voxels to be 15-25 cm×15-cm×15-25 cm, to have a data point density per voxel from
0.001
to
0.016
m
3
(cubic-meter). Typically, before training via the method or processing the point-cloud, the density of the points within the point-cloud dataset (or point-cloud density) and the volume of each of the plurality of voxels may be varied based on the implementation. As used herein, the “point-cloud density” refers to an indicator of the resolution of the point-cloud data, wherein a higher point density means more information (i.e., high resolution) while lower density means less information (i.e., low resolution). Thus, upon experimentation and analysis, the volume of each of the voxels is predetermined to be defined in a range between 15 cm to 25 cm (length) by 15 cm to 25 cm (width) by 15 cm to 25 cm (height) i.e., in a range between
3375
c
m
3
and
15625
c
m
3
. In an example, the predetermined volume of each of the plurality of voxels is 20 cm×20 cm×20 cm i.e.,
8000
c
m
3
and the data point density per voxel varies from
0.001
to
0.016
m
3
. Beneficially, defining the predetermined volume for each of the plurality of voxels is used to reduce memory consumption and to improve system performance.”, [par. 0072, ln. 1-32]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize Lautala, Zhu, and Uljanovs as within the same field of image processing for dynamically captured 3D point clouds, and Lautala and Uljanovs and further within the same field of LiDAR for electrical utility inspection, and as analogous to the claimed invention. The motivation to combine is disclosed in Uljanovs, wherein decimating improves performance by reducing memory consumption and increasing accuracy ([par. 0065, ln. 6-30], [par. 0068, ln. 1-26], [par. 0070, ln. 1-23]), and additionally, by allowing for the data to be used for machine learning training ([par. 0072, ln. 1-32]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, and further combined the method of the combination of Lautala and Zhu with the decimating and point per-grid square limitation of Uljanovs, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the method of the combination of Lautala and Zhu such that the method further decimated the LIDAR extending around the utility object with a point per-grid square limitation analogous to Uljanovs.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 1.
8. Regarding Claim 4, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala further discloses determining an initial position of the utility object based at least in part on one or more of the LIDAR and GIS data ([par. 0051, ln. 1-8] “…the image data comprises at least one of the list of timestamps, locations, and orientations for each image taken by the aerial vehicle during a given time range for capturing the image data. Notably, the image data is orthorectified using the method and mapped to actual mapped locations in the form of a data representation, such as raster file that comprises of each hyperspectral band orthorectified to specified coordinate system.”, [par. 0054, ln. 14-21], [par. 0081, ln. 1-7] “…the aerial vehicle transmits the image data to the data processing arrangement, wherein the image data comprises at least of: the timestamps with location and/or position for each image, sensor arrangement (or model) data, locations of the nearby multiple objects from a digital twin modeling, Digital Elevation Model (DEM) data.”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 4.
9. Regarding Claim 5, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala further discloses compensating the initial position based on one or more of a speed and an orientation of the LIDAR system to correct position errors of the utility object ([par. 0089, ln. 1-36] “… data processing arrangement is further configured to generate the first image dataset… inedited at least two subsets… of the image dataset having overlapped area of the area of interest captured in each of the at least two subsets… is further configured for identifying the at least two subsets of the image dataset having the overlapped area of interest and selecting one of the at least two subsets for capturing the said overlapped area of the area of interest based on a selection criteria, wherein the selection criteria comprises at least one of selected from: speeds of the aerial vehicle while capturing the at least two subsets, orientations of the aerial vehicle while capturing the at least two subsets, lighting conditions while capturing the at least two subsets. In some cases, such as while capturing image data along the area of interest with different orientations and angles causing overlapping of the captured images in the image dataset, the aerial vehicle tends to capture at least two image datasets of the same object, location and/or overlapped area. And, to eliminate the similar images data from the image dataset, the data processing arrangement is further configured for selecting one of the at least two subsets of the capture image dataset based on the selection criteria. Typically, the at least two subsets are compared to select one of the at least two subsets having better attributes and/or conditions while capturing the image dataset, wherein the comparison is based on at least one of the speed of the aerial vehicle, orientations of the aerial vehicle, lighting conditions, weather conditions and so forth. Thus, beneficially, the data processing arrangement processes only one of the at least two subsets i.e., only the areas that have the best capture conditions, to reduce the associated data storage costs and computation time and make the system faster and efficient”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 5.
10. Regarding Claim 6, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala discloses the method further comprises capturing, with a camera ([par. 0055, ln. 1-26] “Herein, the term “hyperspectral data” refers to a spatially sampled dataset comprising a plurality of pixels related to hyperspectral images captured or collected by hyperspectral imaging devices across the electromagnetic spectrum… The term “hyperspectral imaging or HSI” refers to a type of spectral imaging for inferring spectral characteristics of an image, wherein the spectra is divided to N different wavelengths. Notably, the sampling of the hyperspectral data may or may not be spatially regular based on the implementation. However, the irregular spatial sampling of the HSI data used herein may be orthorectified, i.e., normalized in a given plane, such as the x-y plane.”, [par. 0076, ln. 25-30] “…each slice of the second image data comprises at least the HSI camera model parameters, HSI camera calibration parameters, HSI capture timing data, HSI pixels, GPS location and alignment data, and the DEM that covers the area of interest (or the maximum relevant second area).”), one or more images of the region of interest ([par. 0054, ln. 1-9] see “…image data comprises… hyperspectral imaging (HSI) data”); determining one or more positions and directions of the camera corresponding to the capturing of the one or more images ([par. 0076, ln. 25-30], [par. 0079, ln. 1-9] “For example, in cases of diagonal flights, the diagonal flight covers the maximum area and hence it makes the DEM covering it to grow rapidly. Notably, the two or more second image dataset slices are stored as plans in memory. For example, each dataset slice may comprise X lines of Camera trajectory, (x0, y0, x1, y1) bounding box of the DEM, and camera model as simple text data.”, [par. 0081, ln. 1-7]); and determining, based on the one or more positions and directions of the camera and the one or more captured images, a location of the utility object ([par 0067, ln. 12-25], [par. 0069, ln. 1-12], [par. 0076, 25-30], [par. 0077, ln. 1-8] “Notably, the exact data format depends on the orthorectification algorithm to be applied, wherein the data-splitting or division algorithm takes into account at least the trajectory overlapping and object location(s). Typically, the overlapping data allows the method to select the highest quality of available pixels, and the asset locations allow the method to discard the uninteresting parts during capturing of the image data.”, [par. 0081, ln. 1-7]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 6.
11. Regarding Claim 7, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 6. Lautalla further discloses wherein the one or more positions and directions of the camera is determined by using a second GPS receiver and a second accelerometer ([par. 0057, ln. 1-16] “…“GNSS data” refers to Global Navigation Satellite System (GNSS) i.e., a constellation of satellites providing positioning and timing data to GNSS receivers. The term “IMU data” refers to Inertial Measurement Unit (IMU) data i.e., a collection of measurement tools including a plurality of parameters for tracking or locating an object (stationary or moving) in the environment. The IMU unit comprises a plurality of sensors for tracking objects, including accelerometers, gyroscopes, magnetometers and the like to provide the IMU data. For example, the parameters include, but is not limited to, location, height, roll, pitch, yaw, and timing. Optionally, other type of sensors, for measuring/monitoring other parameters may be used for e.g., sensors to monitor exposure levels, sun brightness, weather data, humidity, wind speed, etc.”, [par. 0073, ln. 1-34] see “…the method employs the GNSS data, IMU data, and LiDAR data to determine areas required to be down-sampled…”, [par. 0076, 25-30]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 7.
12. Regarding Claim 10, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala further discloses wherein the mobile LIDAR system comprises one or more of a backpack, a drone, an aircraft, and a terrestrial based LIDAR-mounted vehicle ([par. 0052, ln. 2-11] “… an unmanned or manned aerial vehicle operatively coupled to the data processing arrangement for capturing image data for generating orthorectified image(s). The aerial vehicle may be a drone (such as small drone, rotor drone, fixed wing drone, a quadcopter, a reconnaissance drone and the like). Optionally, the aerial vehicle may be an unmanned aerial vehicle (UAV) such as a drone fixed wing aircraft, a rotary wing aircraft and the like or a manned aerial vehicle implemented as a helicopter, a quatrocopter, an octocopter, and the like.”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 10.
13. Regarding Claim 11, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala further discloses wherein the region of interest is a selectable or predetermined region ([par. 0062, ln. 13-18] “…while using hyperspectral image (HSI) data, the spectrum for each pixel in the hyperspectral image data is identified and compared with existing data or with the help of a human expert to accurately identify find, distinguish, and identify objects, materials and/or detecting processes in the area of interest.”, [par 0067, ln. 12-25]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 11.
14. Regarding Claim 16, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala further discloses applying pre-decimation to areas surrounding the region of interest to facilitate equipment identification ([par. 0073, ln. 1-34]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 16.
15. Regarding Claim 19, the claim language is analogous to claim 1 with the exception of “A non-transitory computer-readable storage medium storing instructions that are configured to cause one or more processors to perform a method…”, Lautala discloses a non-transitory computer-readable storage medium storing instructions configured to cause one or more processors to perform the method ([par. 0053, ln. 21-23] “The controller includes a combination of hardware and software components... the controller may include a processor, memory and input/output peripherals and software to be executable by the processor and stored in the memory.”, [par. 0079, ln. 7-16] “…the dataset slice may be stored or written as file data on the memory (or hard drive)… The method requires only pointers of data in the memory, whereas any external party may require the data to be stored in the memory (or RAM)...”, [par. 0068, ln. 1-20] “…in some cases, the ground may be considered as a non-interesting area, or may be objects shorter than a predefined height (for example, 20 m) are considered non-interesting, or may be only a 100 m buffer around the objects is considered interesting, etc. depending on the implementation.”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the non-transitory computer-readable storage medium of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu and the decimating and point per-grid square limitation of Uljanovs to obtain the invention as specified in claim 19.
16. Claims 2-3, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2022/0414362 to Lautala in view of “View-Dependent Dynamic Point Cloud Compression” to Zhu, in view of U.S. Publication No. 2023/0419659 to Uljanovs, and further in view of “Accuracy–Power Controllable LiDAR Sensor System with 3D Object Recognition for Autonomous Vehicle” to Lee et al. (hereinafter Lee).
17. Regarding Claim 2, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala discloses further determine a speed of the mobile LIDAR system and collecting the LIDAR samples at a {rate} based on the speed ([par. 0065, ln. 9-17] “…the generation of the first image dataset comprises selecting one of the at least two subsets for capturing the said overlapped area of the area of interest based on a selection criteria, wherein the selection criteria comprises at least one of selected from: speeds of the aerial vehicle while capturing the at least two subsets…”, [par. 0073, ln. 1-34] see “…image data captured under extreme conditions such as… high speeds… may be discarded or down-samples…”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Lautala discloses selecting and/or collecting LIDAR samples based on the speed of the vehicle, since “collecting” given its broadest reasonable interpretation would encompass the collection of only the LIDAR samples for which speed is not high as disclosed in Lautala. However, Lautala does not specifically disclose wherein the “rate” of the LIDAR is based on the speed, specifically, the collecting of Lautala fails to disclose changing LIDAR acquisition parameters based on the speed, and therefore a “rate” of the LIDAR is not disclosed in Lutalta. Likewise, Zhu and Uljanovs do not specifically disclose determining a speed of the mobile LIDAR system and collecting the LIDAR samples at a rate based on the speed.
However, Lee teaches collecting the LIDAR samples at a rate based on the speed of the mobile LIDAR system ([pg. 1, Abstract, par. 1, ln. 7-10] “we propose algorithms to improve the inefficient power consumption of conventional LiDAR sensors, and efficiently reduce power consumption in two ways: (a) controlling the HAR to vary the laser transmission period (TP) of a laser diode (LD) depending on the vehicle’s speed and (b) reducing the static power consumption using a sleep mode, depending on the surrounding environment.”, [pg. 9, Table 3], [pg. 9, 3.2.1 Speed Detection-Based LiDAR Sensor Control, par. 1, ln. 1-16] “LiDAR sensor with the same HAR,
T
p
, detects the object more accurately at a short distance than at a long distance because of the angle at which the laser is transmitted. Therefore, when considering the distance to the object, if the laser’s HAR is low, the LiDAR sensor can be used for short-range object detection, and the higher the laser’s HAR, the more suitable it will be for detecting long-range objects. We designed the LiDAR sensor so that the LD’s
T
p
depends on the vehicle’s speed, as depicted in Algorithm 1. When the vehicle’s speed is faster than 100 km/h, the LiDAR sensor uses the maximum accuracy to detect distant objects at high speed. However, when the vehicle’s speed is slower than 100 km/h,
N
x
is reduced for low-power operation. As shown in Table 3, when the vehicle’s speed is faster than 100 km/h, the LiDAR sensor transmits the laser 580 times to the maximum number of laser transmissions (
N
x
.
m
a
x
) with 0.25° HAR. When the vehicle is driving at medium-high speed (80–100 km/h), the LiDAR sensor transmits the laser 483 times at 0.3° HAR, so our method reduces
N
x
by 16.73%.
N
x
is decreased by 28.62% with 414 laser transmissions and 0.35° HAR, when the vehicle’s speed is medium (60–80 km/h). For medium-low speeds (40–60 km/h), the LD transmits the laser 362 times, which reduce
N
x
by 37.56%, and the HAR is 0.4°. If the vehicle’s speed is slower than 40 km/h, then the minimum number of laser transmissions (
N
x
.
m
i
n
), 322, can decrease
N
x
by 44.48% with a 0.45° HAR.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Lautala, Zhu, Uljanovs, and Lee as within the same field of image processing for dynamically captured 3D point clouds, and Lautala, Uljanovs, and Lee as further within the same field of mobile LiDAR imaging methods, and as analogous to the claimed invention. The motivation to combine is disclosed in Lee, wherein it reduces power consumption efficiency ([pg. 1, Abstract, par. 1, ln. 7-10]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of the combination of Lautala, Zhu, and Uljanovs with the collecting of LiDAR samples at a rate based on speed of Lee through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combine the method of the combination of Lautala, Zhu, and Uljanovs such that the LiDAR was acquired at a rate based on speed of the vehicle as taught in Lee.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs, and the collecting of LiDAR samples at a rate based on speed of Lee to obtain the invention as specified in claim 2.
18. Regarding Claim 3, a combination of Lautala, Zhu, Uljanovs, and Lee teaches the method of claim 2. Lautala teaches wherein the speed of the mobile LIDAR is determined by using a first GPS receiver, and wherein the orientation of the mobile LIDAR system is determined by using a first accelerometer ([par. 0057, ln. 1-16], [par. 0073, ln. 1-34]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that the GNSS and/or IMU system is analogous to a first GPS received that can determine a speed of the vehicle and an accelerometer for determining the orientation of the vehicle, and that said data is specifically used for at least the acquisition of the LiDAR data as disclosed in Lautala. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs, and the collecting of LiDAR samples at a rate based on speed of Lee to obtain the invention as specified in claim 2.
19. Regarding Claim 12, a combination of Lautala, Zhu, Uljanovs teaches the method of claim 1. Lautala further discloses wherein dynamically capturing the LIDAR data comprises {adjusting a data collection rate to control a point cloud density} and dynamically decimating the LIDAR data based on the region of interest ([par 0067, ln. 12-25], [par. 0068, ln. 1-20], [par. 0073, ln. 1-34]). Analogous to claim 2, Lautala does not specifically disclose adjusting a data collection rate to control a point cloud density. Likewise, Zhu does not specifically teach adjusting a data collection rate to control a point cloud density. Uljanovs teaches adjusting to control a point cloud density ([par. 0065, ln. 6-30], [par. 0068, ln. 1-26], [par. 0070, ln. 1-23], [par. 0072, ln. 1-32]), but the examiner notes that this is performed after the data has been collected as a filtering process, and Uljanovs does not specifically teach wherein the data collection rate is modified to perform the adjustment to control for density.
However, Lee teaches adjusting a data collection rate to control a point cloud density ([pg. 1, Abstract, par. 1, ln. 7-10], [pg. 9, Table 3], [pg. 9, 3.2.1 Speed Detection-Based LiDAR Sensor Control, par. 1, ln. 1-16]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that controlling the transmission period of the LiDAR system is directly analogous to controlling the horizontal angular resolution ([pg. 3, 2. Related Works, par. 4, ln. 1-2] “The LiDAR sensor controls and changes the laser transmission period (𝑇𝑃), which is the HAR, according to the vehicle’s speed and surrounding environment.”), and because the resolution of a LiDAR image is equivalent to the number of points acquired by the LiDAR system, you are effectively controlling the horizonal point cloud density ([pg. 13, Fig. 15], see (a) 580 point-(f) 290 point (sleep) which equates the number of points directly to the number of laser transmissions). The motivation to combine would have been obvious to one of ordinary skill in the art, and is discloses in Lee, wherein the accuracy (i.e., density of the point cloud) is directly proportional to the collection rate and power consumption rate ([pg. 7, Fig. 7], [pg. 6, par. 7, ln. 1 to pg. 7, par. 1, ln. 4] “
T
p
refers to the time interval for transmitting the laser at the HAR intervals. As shown in Figure 7, if the HAR is increased by narrowing
T
p
, the LiDAR sensor’s power consumption increases as the accuracy of the object detection increases. In addition, if the HAR is decreased by widening
T
p
, the power consumption decreases as the accuracy decreases. Therefore, because
T
p
of the LD, which determines the HAR, and the accuracy of the object detection are proportional, power consumption increases as
T
p
narrows for the same period of time.”), and therefore, arguments analogous to claim 2 are further applicable to claim 12. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs and the collecting of LiDAR samples at a rate based on speed of Lee to obtain the invention as specified in claim 12.
20. Regarding Claim 13, a combination of Lautala, Zhu, Uljanovs, and Lee teaches the method of claim 12. Rejections analogous to claim 1 are further applicable to claim 13. Specifically, Lautala, Zhu, and Lee do not specifically disclose wherein dynamically decimating the LIDAR data further utilizes voxel decimation to limit a number of points per grid square. However, Uljanovs specifically teaches wherein dynamically decimating the LIDAR data further utilizes voxel decimation to limit a number of points per grid square ([par. 0065, ln. 6-30], [par. 0068, ln. 1-26], [par. 0070, ln. 1-23]). The motivation to combine remains analogous to claim 1 ([par. 0065, ln. 6-30], [par. 0068, ln. 1-26], [par. 0070, ln. 1-23], [par. 0072, ln. 1-32]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of the combination of Lautala, Zhu, and Uljanovs with the collecting of LiDAR samples at a rate based on speed of Lee through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the voxel decimating and point per-grid square limitation of Uljanovs and the collecting of LiDAR samples at a rate based on speed of Lee to obtain the invention as specified in claim 13.
21. Regarding Claim 20, a combination of Lautala, Zhu, and Uljanovs teaches the non-transitory computer readable storage medium of claim 19. Lautala further teaches determine a speed of the mobile LIDAR system ([par. 0065, ln. 9-17] [par. 0073, ln. 1-34]); collect the LIDAR samples at a {rate} based on the speed ([par. 0065, ln. 9-17] [par. 0073, ln. 1-34]); {adjust a data collection rate to control a point cloud density}; and dynamically decimate the LIDAR data based on the region of interest ([par 0067, ln. 12-25], [par. 0068, ln. 1-20], [par. 0073, ln. 1-34]). Analogous to claims 2 and 12, Lautala, Zhu, and Uljanovs do not specifically disclose adjusting a LiDAR collection rate or point cloud density.
However, Lee teaches adjusting a data collection rate to control a point cloud density, and wherein the rate is adjusted based on the speed ([pg. 1, Abstract, par. 1, ln. 7-10], [pg. 9, Table 3], [pg. 9, 3.2.1 Speed Detection-Based LiDAR Sensor Control, par. 1, ln. 1-16]). Rejections analogous to a combination of 2 and 12 are further applicable to claim 20 in view of the non-transitory computer readable storage medium of the combination of Lautala and Zhu. The motivation to combine remains analogous to claims 2 and 12. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the non-transitory computer readable storage medium of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs, and the collecting of LiDAR samples at a rate based on speed of Lee to obtain the invention as specified in claim 20.
22. Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2022/0414362 to Lautala in view of “View-Dependent Dynamic Point Cloud Compression” to Zhu, in view of U.S. Publication No. 2023/0419659 to Uljanovs, and further in view of U.S. Publication No. 2023/0127756 to Desai et al. (hereinafter Desai).
23. Regarding Claim 8, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 6. Lautala does not specifically disclose determining the location of the utility object is further based on triangulation using a determined position of the camera. Likewise, Zhu and Uljanovs do not specifically disclose determining the location of the utility object is further based on triangulation using a determined position of the camera
However, Desai teaches determining the location of an object is further based on triangulation using a determined position of the camera ([Fig. 9A-C] see electric utility pole, [par. 0044, ln. 1-17] “FIG. 4A illustrates an example SfM map with a correctly mapped key point, according to some embodiments. In trigonometry and geometry, triangulation is the process of determining the location of a point by forming triangles to the point from known points. For purposes of identifying various objects of interest in a visualization mapping (SfM or lidar), many points located within the map may correspond to key points. Example key points of an object located in a viewing space (e.g., of a vehicle) may include, but not be limited to, object corners, edges, surfaces, etc. For illustration purposes only, FIGS. 4A and 4B will be reduced to triangulating a location of a single point. However, the process may be repeated for any number of points in the SfM map. Corresponding depth mapping of these points may assist… an autonomous driving system 100 to understand the environment (e.g., proximate objects) in its immediate path and therefore avoid collisions.”, [par. 0046, ln. 1-13] “A spatial triangle 405 may be formed by two cameras 406 and 408 and a point located within their field of view. Within this spatial triangle 405, the distance between the cameras is the base b1 of the triangle and may be known… cameras 406 and 408 may each retain a geospatial location (e.g., using global positioning satellite (GPS)). By determining a distance between center points on their respective image planes (image plane 1 and image plane 2), the base (b1) of the triangle may be determined. The image plane in a camera is a surface the light is focused onto after passing through a photographic lens… in digital cameras the image plane is the surface of the digital image sensor.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Lautala, Zhu, Uljanovs, and Desai as within the same field of image processing for dynamically captured 3D point clouds, and Lautala and Desai as further within the same field of joint LiDAR-camera systems for mobile imaging systems, and as analogous to the claimed invention. Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize triangulation as a standard and widely used technique for determining positions of objects and/or cameras using images. The motivation to combine the method of the combination of Lautala, Zhu, and Uljanovs with Desai would have been obvious to one of ordinary skill in the art, and is disclosed in Desai, in that the triangulated positions could be used to further improve accuracy by comparison and/or combination with depth/LiDAR data, and wherein the triangulated positions offer an alternative to LiDAR when LiDAR may not be applicable and/or accurate ([Fig. 6] see 608-612, [par. 0069, ln. 1-15] “At step 612, the image processing system compares a first depth of a key point (triangulated point) within the SfM depth map with a common key point in the splatted lidar map to validate these key points. A depth comparison of a depth of a key point in the SfM map and the same key point within the localization prior determines differences in depths between the two modalities for creating a point could map. Incorrectly triangulated points (not the same depth) get marked as outliers if their depths do not lie within a set threshold (e.g., X cm) of the depth obtained from the depth map image (e.g., some minor errors may be allowed for points on a common surface). In a non-limiting example, X may be 5 (cm)…”, [par. 0074-0077, ln. 1-15] “The technology described herein has many benefits… provides a computer solution to a problem (inaccurate depth calculations)… Another benefit is removal of outlier key points within an SfM map and establishing a degree of confidence for points not recognized as outliers... Another benefit is elimination of false detections based on incorrect triangulations or improperly calculated depth maps... Another benefit is reduction of collisions of autonomous vehicles with objects based on a more accurate representation of the object's key points and associated depths.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of the combination of Lautala, Zhu, and Uljanovs with the triangulation of Desai through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs, and the triangulation of Desai to obtain the invention as specified in claim 8.
24. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2022/0414362 to Lautala in view of “View-Dependent Dynamic Point Cloud Compression” to Zhu, in view of U.S. Publication No. 2023/0419659 to Uljanovs, and further in view of U.S. Publication No. 2021/0073692 to Saha et al. (hereinafter Saha).
25. Regarding Claim 9, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala does not specifically disclose outputting a work order based on the determining of the at least one condition of the utility object. Likewise, Zhu and Uljanovs do not specifically disclose outputting a work order based on the determining of the at least one condition of the utility object.
However, Saha teaches outputting a work order based on the determining of the at least one condition of the utility object ([par. 0187, ln. 1-18] “The example alert module 1504 is configured to receive an indication of the detected condition from each of the detection modules described in reference to the detection module 1502 and, based on the detected condition, transmit an alert for the detected condition to one or more personnel associated with the utility infrastructure, emergency responders, or other individuals and/or computing devices associated with the same… the transmitted alert includes an indication that the detected condition is (or poses) a hazard (or potential hazard) to the utility infrastructure. The alert module 1504 may also be configured to store the detected conditions in a database, from which the condition information may be accessed for a variety of uses including… preemptive de-energization of power line in extreme weather… to avoid ignitions from downed power line or structure or equipment”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Lautala, Zhu, Uljanovs, and Saha as within the same field of image processing for dynamically captured 3D point clouds, and Lautala, Uljanovs, and Saha as further within the same field of LiDAR imaging for utility infrastructure inspection, and as analogous to the claimed invention. Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that the notification as disclosed in Saha to perform an action (e.g., de-energization of power line) constitutes a “work order” given the broadest reasonable interpretation of “work order”. Furthermore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to include a “work order” based on the condition of the utility object, since Lautala and Saha are specifically described in the context of regular maintenance of utility infrastructure, and thus in the case where said infrastructure is in need of maintenance (e.g., trees growing too close to power lines, power line down, etc.) it would have been obvious to send a notification “work order” to perform maintenance on the infrastructure. With regards to motivation to combine, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize outputting a work order based on the determining the condition of a utility object would allow for preemptive maintenance and thus reduction of failures within the infrastructure object. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of the combination of Lautala, Zhu, and Uljanovs with the work orders of Saha, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs, and the work orders of Saha to obtain the invention as specified in claim 9.
26. Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2022/0414362 to Lautala in view of “View-Dependent Dynamic Point Cloud Compression” to Zhu, in view of U.S. Publication No. 2023/0419659 to Uljanovs, and further in view of “
P
3
-LOAM: PPP/LiDAR Loosely Coupled SLAM With Accurate Covariance Estimation and Robust RAIM in Urban Canyon Environment” Li et al. (hereinafter Li).
27. Regarding Claim 15, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala does not specifically disclose performing precise point positioning (PPP) around identified regions of interest, though one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Lautala does utilize GNSS data analogous to the data used for PPP to perform decimation and identify object/regions of interest ([par. 0054, ln. 1-9], [par. 0062, ln. 8-13] “…the method comprises using the first image data comprising… the GNSS data and/or the IMU data to identify the multiple objects in the area of interest.”, [par. 0073, ln. 1-34]). Likewise, Zhu and Uljanovs do not specifically disclose performing precise point positioning (PPP) around identified regions of interest.
However, Li teaches wherein a coupled LiDAR/PPP system ([pg. 6663, Fig. 1] see GNSS and PPP Algorithm, [pg. 6661, col. 1, par. 3, ln. 20-28] “…PPP technology is more suitable for unmanned systems as PPP is independent of base stations. Single-frequency PPP (SF-PPP) and Dual-frequency PPP (DF-PPP) are two mainstream PPP technologies. Both of them need precise corrections which can be acquired from International GNSS Service Real-Time Service (IGS-RTS) [25] for the satellite orbits and satellite clocks. In autonomous driving, SF-PPP receives considerable attention because of its lower cost.”, [pg. 6663, Fig. 2], [pg. 6662, col. 2, par. 3, ln. 1-10] “The coordinate frames and notations involved in this article will be clarified below. The Earth-centered Earth-fixed coordinate system (ECEF, Frame E , see Fig. 2) rotates with the Earth, taking the Earth’s centroid as the origin. The X-axis of Frame E points to the intersection of the equator and prime meridian. The Earth’s rotation axis is taken as Z-axis, and the North Pole is the positive direction. Then, the Y-axis is perpendicular to the X-Z plane, forming a right-handed coordinate system. The GNSS receiver generally outputs positioning results in Frame E.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Lautala, Zhu, and Li as within the same field of image processing for dynamically captured 3D point clouds, and Lautala and Li as further as within the same field of LiDAR GNSS integrated point cloud imaging systems, and as analogous to the claimed invention. Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize the GNSS system of the method of the combination of Lautala and Zhu as being directly analogous to the GNSS system of Li. The motivation to combine the PPP of Li with the GNSS system of the method of the combination of Lautala and Zhu is disclosed in Li, wherein it allows for a reliable conversion of LiDAR local reference positions to a global reference position ([pg. 6661, col. 2, par. 2, ln. 10-28] “For these reasons, the coupling of GNSS will enable the LiDAR-SLAM system to provide more robust localization and mapping results with global earth coordinates in large-scale scenarios. In turn, GNSS is heavily influenced by multipath and NLOS in urban canyon environment, resulting in poor performance. LiDAR-SLAM always performs well in urban canyon environment and gets a relative positioning result. So it is meaningful to fuse GNSS and LiDAR-SLAM: LiDAR-SLAM can improve the reliability and availability of GNSS, on the other hand, GNSS can turn the positioning result of LiDAR-SLAM in a local coordinate system to the global such as World Geodetic System 1984 (WGS-84). But in conventional GNSS and LiDAR-SLAM fusion systems, few works have paid attention to either the calculation of LiDAR-SLAM positioning covariance, or the use of PPP in autonomous driving. In conclusion, our motivation for this paper is complementing PPP and LiDAR-SLAM to obtain more reliable and available positioning results in global coordinate system.”). In combining the PPP of Li, it would have been obvious to one of ordinary skill in the art to perform PPP around identified regions of interest using the GNSS of the method of the combination of Lautala, Zhu, and Uljanovs analogous to the region of interest determination already performed in the method of the combination of Lautala, Zhu, and Uljanovs using the GNSS system. One of ordinary skill in the art, before the effective filing date of the claimed invention, would have combined the method the combination of Lautala, Zhu, and Uljanovs with the PPP of Li through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs, and the PPP of Li to obtain the invention as specified in claim 15.
28.. Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2022/0414362 to Lautala in view of “View-Dependent Dynamic Point Cloud Compression” to Zhu, in view of U.S. Publication No. 2023/0419659 to Uljanovs, and further in view of “An Easy-to-Use Airborne LiDAR Data Filtering Method Based on Cloth Simulation” to Zhang et al. (hereinafter Zhang).
29. Regarding Claim 17, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala does not specifically disclose wherein the method comprises one or more of storing the processed LIDAR data in a .las format; converting 64-bit LIDAR data points near a grid center to 32-bit data; storing the processed LIDAR data as a binary large object (BLOB); removing ground from images using a cloth simulation filter (CSF) algorithm; scaling the images to a predefined resolution; utilizing grayscale or HSV color space with color mapping to represent a z-position in the images; and employing one or more Yolo models for classification and object detection. Specifically, Lautala only discloses removing the ground from images, and does not specifically disclose this is done using a CSF ([par. 0068, ln. 1-20]). Likewise, Zhu and Uljanovs do not specifically disclose the limitations of claim 17.
However, Zhang discloses wherein the ground can be removed using a CSF ([pg. 3, par. 2, ln. 7-14] “To cope with these problems, this paper proposes a novel filtering algorithm which is capable of approximating the ground surface with a few parameters. Different from other algorithms, the proposed method filters the ground points by simulating a physical process that an virtual cloth drops down to an inverted (upside-down) point cloud. Compared to existing filtering algorithms, the proposed filtering method has some advantages: (1) few parameters are used in the proposed algorithm, and these parameters are easy to understand and set; (2) the proposed algorithm can be applied to various landscapes without determining elaborate filtering parameters; and (3) this method works on raw LiDAR data.”, [pg. 3, 2. Method, par. 1, ln. 1-11] “Our method is based on the simulation of a simple physical process. Imagine a piece of cloth is placed above a terrain, and then this cloth drops because of gravity. Assuming that the cloth is soft enough to stick to the surface, the final shape of the cloth is the DSM (digital surface model). However, if the terrain is firstly turned upside down and the cloth is defined with rigidness, then the final shape of the cloth is the DTM. To simulate this physical process, we employ a technique that is called cloth simulation [26]. Based on this technique, we developed our cloth simulation filtering (CSF) algorithm to extract ground points from LiDAR points. The overview of the proposed algorithm is illustrated in Figure 1. First, the original point cloud is turned upside down, and then a cloth drops to the inverted surface from above. By analyzing the interactions between the nodes of the cloth and the corresponding LiDAR points, the final shape of the cloth can be determined and used as a base to classify the original points into ground and non-ground parts.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Lautala, Zhu, Uljanovs, and Zhang as within the same field of rendering models using LiDAR data, and as analogous to the claimed invention. The motivation to combine the CSF filter of Zhang with the method of the combination of Lautala, Zhu, and Uljanovs is disclosed in Zhang, wherein it offers a low complexity determination of the ground points ([pg. 20, 5. Conclusions, par. 1, ln. 3-11] “Compared to conventional filtering algorithms, the parameters are less numerous and are easy to set. Regardless of the complexity of ground objects, the samples were divided into three categories according to the shape of the terrain. Few parameters are needed, and these parameters hardly changed among the three sample categories; only an integer parameter rigidness and a Boolean parameter ST are required to be set by the user. These three groups of parameters exhibit relatively high accuracies for all fifteen samples of the ISPRS benchmark datasets. Another benefit of the CSF algorithm is that the simulated cloth can be directly treated as the final generated DTM for some circumstances, which avoids the interpolation of ground points, and can also recover areas of missing data”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that you could identify and decimate the ground, analogous to the process already disclosed in Lautala ([par. 0068, ln. 1-20]), using the ground points as identified by the CSF of Zhang. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of the combination of Lautala, Zhu, and Uljanovs with the CSF filter for ground point identification of Zhang through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. The examiner specifically notes that the claim states “…one or more of…”, and therefore, only one of the select groups of limitations is required given the BRI of the claim, and as such, a combination of Lautala, Zhu, Uljanovs, and Zhang would teach claim 17.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to have combined the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating and point per-grid square limitation of Uljanovs, and the CSF filter of Zhang to obtain the invention as specified in claim 17.
30. Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2022/0414362 to Lautala “View-Dependent Dynamic Point Cloud Compression” to Zhu, in view of U.S. Publication No. 2023/0419659 to Uljanovs, and further in view of “3D YOLO: End-to-End 3D Object Detection Using Point Clouds” to Al Hakim (hereinafter Al Hakim).
31. Regarding Claim 18, a combination of Lautala, Zhu, and Uljanovs teaches the method of claim 1. Lautala further discloses processing data from regions {in a 3-section grid using a Yolo} model to identify a presence of the utility object ([par. 0062, ln. 1-13] “…for effective monitoring of the powerline, the method comprises identifying the multiple objects in the area of interest from the first image dataset. Typically, the multiple objects are identified using a detection and/or identification algorithm that may or may not be assisted by a human expert or user. Herein, the detection and/or identification algorithm is a machine learning algorithm configured to identify the multiple objects based on the information provided from the capture image data or the first image dataset. Typically, the method comprises using the first image data comprising the LiDAR data, the HSI data, the GNSS data and/or the IMU data to identify the multiple objects in the area of interest”); applying object detection to determine a precise location of the utility object ([par. 0062, ln. 1-13]); and creating a full 3D model of each utility object for assessment ([par. 0071, ln. 4-21], [par. 0074, ln. 1-10] “In an example, such as in cases of a flat terrain or a known flat forest and/or trees, instead of processing each of the pixels in the captured image data (location data for that group of pixels is coming from a digital twin model), a group of pixels based on the attribute information are processed to enable the method to process the image data much faster and efficiently. For instance, consider a whole Digital elevation model (DEM) for an area of 100 m×100 m that is represented by 1000×1000 pixels in the first image dataset.”, [par. 0079, ln. 19-25] “Thus, by joining or combining the second image dataset slice, the method or system is enabled to manage the overlapping data on the fly. Optionally, the DEM based on the second image dataset is turned into a 3D model via at least one tinning algorithm.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize the DEM of Lautala to comprise the utility objects within an area of interest, and therefore represent a full 3D model of each utility object. Lautala does not specifically disclose wherein the machine learning model for detecting the utility object is a YOLO based model or wherein the regions are in a 3d-section grid. Likewise, Zhu does not specifically disclose processing data from regions in a 3-section grid using a Yolo model.
However, Uljanovs specifically teaches wherein the regions are in a 3d-section grid ([Fig. 3D], [par. 0065, ln. 6-30], [par. 0068, ln. 1-26], [par. 0070, ln. 1-23], [par. 0072, ln. 1-32]). The motivation to combine remains analogous to claim 1 ([par. 0065, ln. 6-30], [par. 0068, ln. 1-26], [par. 0070, ln. 1-23] [par. 0072, ln. 1-32]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, and further combined the method of the combination of Lautala and Zhu with the decimating, 3d-section gird, and point per-grid square limitation of Uljanovs, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the method of the combination of Lautala and Zhu such that the method further decimated the LIDAR extending around the utility object with a point per-grid square limitation analogous to Uljanovs. Uljanovs does not specifically disclose a yolo model.
However, Al Hakim teaches processing data from regions in a 3-section grid using a YOLO model to identify a presence of an object ([pg. 34, Fig. 4.1], [pg. 34, Chapter 4, 3D YOLO, par. 1, ln. 1-5] “This chapter presents extension of YOLO for end-to-end trainable 3D object detection network that takes point cloud as input and yields 3D bounding boxes with class scores without using any hand-crafted features. The proposed model will be denoted 3D YOLO.”, [pg. 35, 4.1 Architecture, par. 1, ln. 1-6] “Briefly, 3D YOLO consists of two networks, the Feature Learning Network (FLN) and a CNN based on You Only Look Once v2 (YOLOv2), as shown in Figure 4.1. The FLN is the same network as used in VoxelNet, which transforms the input point cloud to a new feature space. YOLO takes this new representation of the point cloud as input and outputs class scores and bounding box coordinates.”, [pg. 35, 4.1.1 Feature Learning Network, Network, par. 1, ln. 1-5] “The FLN takes a non-empty voxel
V
i
n
(
i
)
=
{
p
^
1
i
,
…
,
p
^
T
i
}
∈
R
T
×
7
as input and outputs a feature vector with fixed dimension C, denoted as
V
o
u
t
(
i
)
∈
R
C
. We also convert an empty voxel (containing no points) to the zero feature vector. The architecture of FLN is a chain of connected Voxel Feature Encoding (VFE) layers, as shown in Figure 3.15.”, [pg. 36, par. 5, ln. 1-4] “By feeding all non-empty voxels
V
i
n
(
1
)
,
…
,
V
i
n
(
R
)
to the FLN, a list of feature vectors will be obtained
V
o
u
t
(
1
)
,
…
,
V
o
u
t
(
R
)
. This list can be represented as a sparse 4D tensor of size (C × D’ × H’ × W’). After reshaping it to a 3D tensor of size (H’ × W’ × C · D’), it passes through the YOLO network.”, [pg. 37, Fig. 4.3], [pg. 37, 4.1.2 YOLO Network, par. 1, ln. 1-2 and par. 1-9] “We design a new CNN architecture base on YOLOv2 [39] to detect 3D objects in real-time, called 3DNet…The output of 3DNet is a tensor of size (H’/8 × W’/8 × B · (8 + K)), where B is the number of the anchors and K is the number of classes. Each cell in the feature map grid (H’/8 × W’/8) predicts B bounding boxes, confidence scores for each of them and K class scores p1, ..., pK (see Figure 4.3). Predicted bounding boxes are parameterized as refined anchors.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize Lautala, Zhu, Uljanovs, and Al Hakim as within the same field of image processing for 3D point clouds, and Lautala, Uljanovs, and Al Hakim as further within the same field of object detecting using machine learning on 3D point cloud data, and as analogous to the claimed invention. The motivation to substituted the machine learning based utility object detection of the method of the combination of Lautala, Zhu, and Uljanovs with the 3-section grid-based YOLO object detection of Al Hakim is disclosed in Al Hakim, wherein 3D YOLO achieves significant accuracy while being significantly faster than comparably accurate models ([pg. 45, 5.2 Runtime, par. 1, ln. 1-5] “We compare runtime of 3D YOLO with MV3D [15] and VoxelNet [20], presented in Table 5.1. Since VoxelNet has been tested on a Nvidia Titan X GPU and source code is unavailable, we compared our runtime with an unofficial Tensorflow implementation [43] of VoxelNet on a Nvidia Tesla P100 GPU. The table shows our model is 1.64× faster than VoxelNet.”, [pg. 46, Table 5.1], [pg. 46, 5.3 KITTI Evaluation on Validation Set, par. 1, ln. 1-5] “We evaluate our model only on the Car class across all three difficulty levels (easy, moderate and hard) and compare it with several state-of-the-art 3D object detectors, including Mono3D [18] and 3DOP [19] which use image data only, VeloFCN [17] and VoxelNet [20] which use LiDAR data only and MV3D [15] which was both images and LiDAR data.”, [pg. 46, Table 5.2], [pg. 47, Table 5.3], [pg. 47, par. 1, ln. 1-4] “On the validation set (see Table 5.3), our model outperformed all LiDARbased methods except VoxelNet on both easy and moderate difficulty levels. We see in the tables and Figure 5.1 what is expected, the average position for bird-eye-view has higher value than the 3D world.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of the combination Lautala, Zhu, and Uljanovs with the 3-section grid based YOLO object detection of Al Hakim, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Lautala with the depth-encoded multi-perspective images from point cloud data of Zhu, the decimating, 3d-section gird, and point per-grid square limitation of Uljanovs, and the 3-section grid-based YOLO object detection of Al Hakim to obtain the invention as specified in claim 18.
Allowable Subject Matter
32. Claim 14 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
34. The following is an examiner’s statement of reasons for allowance:
With regard to claim 14, previously applied reference, alone or in combination, fail to specifically the claimed invention as a whole as described in claim 14. Specifically, while references of record teach individual elements of claim 14 (see Alismail), they fail to specifically teach or suggest a motivation to have combined the dynamic acquisition with an octree spatial index and a-prior knowledge to optimize point selection. The examiner further notes that combining Alismail with the method of the combination of Lautala, Zhu, Uljanovs, and Lee would require further modification of the voxel methodologies already used in Uljanovs beyond the scope of one of ordinary skill in the art in view of modification already relating to dynamic acquisition as taught in Lee.
Conclusion
35. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Conclusion
36. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights