DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on/after Mar. 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Allowable Subject Matter
Claims 7-9 and 13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including ALL of the limitations of the base claim AND any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 7, the prior art of record does not teach, suggest, or disclose the claim limitation “retraining, while freezing the code map and the decoder model, the encoder model with a training sparse LiDAR image” in combination with the other recited claim limitations.
Regarding claim 8, the prior art of record does not teach, suggest, or disclose the claim limitation “generating a loss based on the reconstructed dense LiDAR data” in combination with the other recited claim limitations. Claim 9, depending from claim 8, would be allowable for the same reason.
Regarding claim 13, the prior art of record does not teach, suggest, or disclose the claim limitation “determining a subset of the … codes as unused based on failing to satisfy the threshold to obtain the unused set of codes”, in combination with the claim limitation “the threshold is on … one(s) selected from a group consisting of an elapsed time a code is since last used and an amount of continuous space mapped to the code”, and further in combination with the other recited claim limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 15-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Spurek et al. (“HyperPocket: Generative Point Cloud Completion”, ‘SPUREK’) in view of Fei et al. ("VQ-DcTr: Vector Quantized Autoencoder With Dual-channel Transformer Points Splitting for 3D Point Cloud Completion", 'FEI') | {both presented Oct. 2022}.
Regarding claim 1, SPUREK discloses a method comprising:
generating a … (3D) LiDAR image from LiDAR input data (SPUREK; p. 6848; § I. Intro.; “The development of 3D … LIDARs … offers an … ability to capture complex 3D scenes and represent them with the point clouds [‘(3D) LiDAR image’]. One of the desired properties of this digitization process is the possibility to decompose real-life scenes into a set of distinct 3D objects to be manipulated or modified.”);
PNG
media_image1.png
546
719
media_image1.png
Greyscale
encoding, by an encoder model, the 3D LiDAR image to a continuous embedding in continuous space (SPUREK; FIG. 2; p. 6850; § 3. ‘HyperPocket …’; “Generative autoencoder model: A classical autoencoder architecture is designed to represent high-dimensional data (e.g., images) by their low-dimensional code vectors. Specifically, this architecture consists of encoder … and decoder … networks, acting on a data space X and a latent (code) space …, respectively, which are trained simultaneously to minimize distance between a given real data set … and the set of reconstructions … We can also interpret this objective in the context of increasing the similarity between two probability distributions on X that represents the real and reconstructed data. … To obtain a generative framework, we should additionally ensure that the distribution of encoded data … is similar to a given prior (noise) distribution PZ (typically a Gaussian one) on the latent Z. … we use Variational Autoencoders (VAE) …”; [The Examiner asserts that the VAEs map inputs to a ‘continuous embedding in continuous space’.]).
SPUREK does not explicitly disclose the following limitations, which FEI discloses:
performing, using a code map, a vector quantization of the continuous embedding to generate a discrete embedding (FEI; p. 4769; Abstract; “Existing point cloud completion methods mainly utilize the global shape representation to recover the missing regions of the 3D shape from the partial point cloud. However, these methods learn the global shape representations with continuous features against the inherently discrete nature of point cloud, hardly resulting in a high-quality structure for points. To address this challenge, we concentrate on discrete representations, which are potentially a more natural fit for the modalities of the point cloud. Therefore, we propose to employ Vector Quantization (VQ) Auto-Encoder and Dual-channel Transformer for point cloud completion (VQ-DcTr). The VQ-DcTr is apt to use discrete global features and exploit them in a well-structured generation process. … the vector quantization auto-encoder is integrated to learn a discrete latent representation along with inductive biases inherent in the transformer-based auto-encoder. By using the decoded seeds from the auto-encoder, the dual-channel transformer leverages pointwise and channel-wise attention to learn the splitting patterns in the previous Dual-channel Transformer Points Splitting (DCTPS) layer to perform the points splitting in the current DCTPS layer. In this way, we can obtain the locally compact and structured point cloud by capturing the structure characteristic of 3D shape in local patches.”)
decoding, by a decoder model, the discrete embedding to generate modified LiDAR data (FEI; p. 4773; § 3.5 ‘Gated Cross-attention Connection’; “The VQ-DcTr employs the attention mechanism to learn the relationship between the point jointly features by the DCTPS-based decoder and the features extracted from PSA and effectively integrates them … With the help of attention, the GCAC module can guide the DCTPS-based decoder to generate more realistic geometries when these regions exist in the original input point clouds by incorporating features from the PSA and DCTPS-based decoder. Otherwise, the GCAC can find similar regions of the partial inputs with symmetry instruction, enabling the DCTPS-based decoder to infer the missing regions.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of SPUREK to include the performing, using a code map, a vector quantization of the continuous embedding to generate a discrete embedding and the decoding, by a decoder model, the discrete embedding to generate modified LiDAR data of FEI. The motivation for this modification is to overcome the bottleneck of continuous global representations. A vector quantization auto-encoder to learn robust discrete global features from the partial input point cloud is proposed. The vector quantization auto-encoder is achieved by extending the vector quantization idea, which learns a codebook of context-rich visual parts and a model of their global compositions. Such a codebook can be regarded as a composition of perceptually abundant constituents and help the model understand the global features, enabling it to generate locally realistic as well as globally consistent patterns (FEI; p. 4770, 1st paragraph of left column).
PNG
media_image2.png
345
720
media_image2.png
Greyscale
SPUREK-FEI continue to disclose outputting the modified LiDAR data (SPUREK; FIG. 2; p. 6850; [See screen capture above, highlighted emphasis by the Examiner.]).
Independent claim 16 exhibits similar scope/limitations when compared to independent claim 1; therefore, the same motivation(s) to combine references will be maintained.
Regarding claim 16, SPUREK-FEI disclose a system comprising:
… computer processor(s); and
a non-transitory computer readable medium comprising computer readable program code for causing the … computer processor(s) to perform operations comprising: … ([The remaining limitations are repeated verbatim from independent claim 1. See the rationale for the rejection in the Office action above.]).
SPUREK (“HyperPocket …”) does not explicitly mention computer processor(s) or computer readable media; however, SPUREK does state “… we introduce a novel autoencoder-based architecture called HyperPocket that disentangles latent representations and, as a result, enables the generation of multiple variants of the completed 3D point clouds”. The implementation of the auto-encoder architecture implies use of a computing system which implicitly has a processor and software or memory equating to the “non-transitory computer readable medium”.
Independent claim 20 exhibits similar scope/limitations when compared to independent claim 1; therefore, the same motivation(s) to combine references will be maintained.
Regarding claim 20, SPUREK-FEI disclose a non-transitory computer readable medium comprising computer readable program code for causing a computer system to perform operations comprising: … ([The remaining limitations are repeated verbatim from independent claim 1. See the rationale for the rejection in the Office action above.]).
SPUREK (“HyperPocket …”) does not explicitly mention a computer system or computer readable media; however, SPUREK does state “… we introduce a novel autoencoder-based architecture called HyperPocket that disentangles latent representations and, as a result, enables the generation of multiple variants of the completed 3D point clouds”. The implementation of the auto-encoder architecture implies use of a computing system which implicitly has a computer system and software or memory equating to the “non-transitory computer readable medium”.
Regarding claim 15, SPUREK-FEI disclose the method of claim 1, wherein
the LiDAR input data is real sensor data (SPUREK; p. 6848; § I. Intro.; “The development of 3D registration devices, such as LIDARs or depth cameras, offers an unprecedented ability to capture complex 3D scenes and represent them with the point clouds [‘real sensor data’]. One of the desired properties of this digitization process is the possibility to decompose real-life scenes into a set of distinct 3D objects to be manipulated or modified. Such ability is essential for manipulating robots that struggle to operate autonomously in human-optimized environments. … the typical approach is for the robot to explore the surroundings and generate a 3D model of each object in its vicinity. The main obstacles that prohibit currently used methods from solving this problem are limited sensor resolution, difficulty in covering all scanning positions, and 3D occlusions, resulting in incomplete point clouds.”), and wherein
the modified LiDAR data is generated by a simulator to modify a scene for training a virtual driver of an autonomous system (SPUREK; p. 6853; § V. Conclusion; “This paper presents a new look at a point cloud completion problem and considers its generative extension to the object hallucination task. Our goal is to produce multiple possibilities of completing a given point cloud [‘modified LiDAR data’] instead of providing a single reconstruction. To solve such a problem, we proposed a novel generative HyperPocket architecture based on an autoencoder [‘simulator’] designed using the hypernetwork paradigm. Such a solution can produce many different completion outputs hallucinating unseen parts of objects in a scene [‘modify a scene’], as intuitively done by humans when decomposing a complex scene containing occluded objects. This ability is also crucial for mobile manipulating robots where our method can be used to support them in mapping 3D objects in their surroundings [‘training a virtual driver of an autonomous system’].”).
Claims 2, 11, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over SPUREK in view of FEI as applied to claims 1 and 16 above, respectively, and further in view of Mittal et al. ("AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation", presented in June 2022, 'MITTAL').
Regarding claim 2 and claim 17, SPUREK-FEI disclose the method of claim 1 and the system of claim 16; however, SPUREK-FEI do not explicitly disclose that the method of claim 1 and the system of claim 16 further comprise:
transforming, by a transformer model, the discrete embedding to generate a modified vector embedding prior to the decoding, which MITTAL discloses (MITTAL; p. 308; § 3. Approach; “We propose an autoregressive method to learn the distribution p(X) over possible 3D shapes X. Our method uses a volumetric Truncated-Signed Distance Field (T-SDF) for representing a 3D shape and learns a Transformer-based neural autoregressive model. However, as the computational complexity of transformers increases quadratically with the input dimension, we first map the high dimensional 3D shape to a corresponding low dimensional, discretized latent space [‘discrete embedding’]. We then learn a ‘non-sequential’ autoregressive prior over this compressed discrete representation [‘modified vector embedding’], and show that this learned prior can be leveraged across diverse conditional generation tasks.”), wherein the decoding uses the modified vector embedding (MITTAL; p. 308; § 3.1.; “To learn an effective autoregressive model, we aim to reduce the high-dimensional continuous 3D shape representation to a lower-dimensional discrete latent space. … we adapt the VQ-VAE framework and learn a 3D VQ-VAE whose encoder Eψ can compute the desired low dimensional representation, and the decoder Dψ can map this latent space back to 3D shapes.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 and the system of claim 16 of SPUREK-FEI to include the transforming the discrete embedding to generate a modified vector embedding prior to the decoding, wherein the decoding uses the modified vector embedding of MITTAL. The motivation for this modification is to implement an autoregressive prior for 3D shapes to solve multimodal 3D tasks such as shape completion, reconstruction, and generation. The distribution is modeled over 3D shapes as a non-sequential autoregressive distribution over a discretized, low-dimensional, symbolic grid-like latent representation of 3D shapes. This enables the representation of distributions over 3D shapes conditioned on information from an arbitrary set of spatially anchored query locations and thus perform shape completion in such arbitrary settings (MITTAL; Abstract).
PNG
media_image3.png
722
1517
media_image3.png
Greyscale
PNG
media_image4.png
707
724
media_image4.png
Greyscale
Regarding claim 11, SPUREK-FEI disclose the method of claim 1; however, SPUREK-FEI do not explicitly disclose that the method of claim 1 further comprises:
gradually, through several training iterations, changing the code map from mapping to continuous space to mapping to discrete space, which MITTAL discloses (MITTAL; FIG. 2; p. 308; § 3.1 ‘Discretized Latent Space for 3D Shapes’; [See screen-captured text highlighted above.]).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of SPUREK-FEI to include the gradually, through several training iterations, changing the code map from mapping to continuous space to mapping to discrete space of MITTAL. The motivation for this modification is to mitigate the computational intractability of learning a generic shape prior directly over the continuous and high-dimensional space of 3D shapes. Using the inspiration of recent approaches that overcome similar challenges for image synthesis, discrete representation learning is leveraged to compute discretized and low-dimensional representations for 3D shapes. This not only preserves the essential information for decoding high-quality outputs but also makes the training of autoregressive models tractable (MITTAL, p. 307).
Claims 3, 6, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over SPUREK in view of FEI as applied to claims 1 and 16 above, respectively, and further in view of Liu et al. ("Morphing and Sampling Network for Dense Point Cloud Completion", presented in 2020, 'LIU').
PNG
media_image5.png
691
1544
media_image5.png
Greyscale
Regarding claim 3 and claim 18, SPUREK-FEI disclose the method of claim 1 and the system of claim 16; however, SPUREK-FEI do not explicitly disclose the following limitations, which are disclosed by LIU:
the LiDAR input data is sparse LiDAR data (LIU; FIG. 2, [See ‘Input’ element in upper-left portion feeding into the ‘Encoder’ and ‘Minimum Density Sampling’ elements.]) and wherein
the modified LiDAR data is dense LiDAR data (LIU; FIG. 2, [See ‘Final Output’ element in lower-right portion]), wherein
the encoder model and the decoder model are a sparse to dense converter (LIU; FIG. 2; pp. 11597-11598; ‘Approach’; “Given a point cloud lying on the partial surface of an object, our approach is expected to predict a point cloud indicating the complete shape of the object. The output point cloud should be dense enough [‘modified LiDAR data is dense LiDAR data’] and evenly distributed so that it can capture the details of the shape. Our approach leverages supervised learning and is trained end-to-end. … our approach takes a partial point cloud as input [‘LiDAR input data is sparse LiDAR data’] and completes it in two stages. In the first stage, the auto-encoder predicts a complete point cloud by morphing the unit squares into a collection of surface elements. The expansion penalty is proposed to prevent the overlaps between the surface elements. In the second stage, we merge the coarse output with the input point cloud. Through a special sampling algorithm, we obtain an evenly distributed subset point cloud from the combination, and then feed it into a residual network for point-wise residual prediction. By adding the residual, our approach outputs the final point cloud. Unlike many existing approaches, we employ EMD for dense point cloud comparison.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 and the system of claim 16 of SPUREK-FEI to include the disclosure that the LiDAR input data is sparse LiDAR data, the disclosure that the modified LiDAR data is dense LiDAR data, and the disclosure that the encoder model and the decoder model are a sparse to dense converter of LIU. The motivation for this modification is to implement a spanning-tree based method such that those sparsely-distributed points are penalized (e.g., those sparsely-distributed points on the boundary of each surface element) instead of all the points. The expansion penalty allows each surface element to generate more flexible shapes (LIU; p. 11599).
Regarding claim 6, SPUREK-FEI disclose the method of claim 1; however, SPUREK-FEI do not explicitly disclose that the method of claim 1 further comprises:
simulating a scene to obtain dense LiDAR data for the scene, which LIU discloses (LIU; FIG. 2; ‘Approach’; pp. 11597-11598; “Given a point cloud lying on the partial surface of an object, our approach is expected to predict a point cloud indicating the complete shape of the object. The output point cloud should be dense enough and evenly distributed so that it can capture the details of the shape. Our approach leverages supervised learning [‘simulating a scene’] and is trained end-to-end.”); and
training the encoder model, the code map, and the decoder model using the dense LiDAR data (LIU; FIG. 2; [The Examiner notes the architecture depicted in FIG. 2 clearly shows an ‘Encoder’ element, a ‘Morphing-Based Decoder’ element, and a ‘GFV’ or ‘generalized feature vector’ element, which the Examiner regards as a ‘code map’ as is instantly recited. The Examiner notes that the ‘Final Output’ point cloud, resembling the ‘Ground Truth’, is depicted as much denser than the ‘Input’ point cloud.]).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of SPUREK-FEI to include the simulating a scene to obtain dense LiDAR data for the scene and the training the encoder model, the code map, and the decoder model using the dense LiDAR data of LIU. The motivation for this modification is to implement 3-D point cloud completion, which is the task of inferring a complete geometric shape from a partial point cloud. This implementation would achieve acquiring high-fidelity dense point clouds and avoiding uneven distribution, blurred details, or structural loss of point cloud data using the two-stage approach disclosed by Liu et al. (LIU, Abstract). Further, complete point cloud models of 3-D data enable robust and accurate understanding of a 3-D scene that may be autonomously navigated.
Claims 4-5 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over SPUREK in view of FEI as applied to claims 1 and 16 above, respectively, and further in view of Yang et al. (U.S. PG-PUB 2023/0222618, 'YANG').
Regarding claim 4, SPUREK-FEI disclose the method of claim 1; however, SPUREK-FEI do not explicitly disclose that the method of claim 1 further comprises:
performing, prior to the outputting, a denoising process on the modified LiDAR data, which YANG discloses (YANG; FIG. 4; ¶ 0050; “In step S420, the occluded point cloud data is removed [‘denoising process on the modified LiDAR data’] from the fused point cloud data to obtain the to-be-detected sample. The occluded point cloud data may include coordinate information of … occluded point clouds, and with the occluded point cloud data, the occluded point cloud may be removed from the fused point cloud data to obtain the to-be-detected sample. … a point cloud that does not include occluded point cloud data may be obtained as the to-be-detected sample.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of SPUREK-FEI to include the performing, prior to the outputting, a denoising process on the modified LiDAR data of YANG. The motivation for this modification is to remove occluded points in a captured point cloud that would not be visible to a human/robot/AV and may represent spurious data and/or noise related to imperfect LIDAR.
Dependent claim 19 is essentially the concatenation of dependent claims 4-5; therefore, the same motivation(s) to combine references as stated in the Office action above will be maintained.
Regarding claim 5 and claim 19, SPUREK-FEI-YANG discloses the method of claim 4 and the system of claim 16, wherein the operations further comprise:
performing, prior to the outputting, a denoising process on the modified LiDAR data (YANG; FIG. 4; ¶ 0050), wherein the performing the denoising process comprises:
projecting the modified LiDAR data from 3D space to a range image space to identify a set of obfuscated LiDAR points in the 3D space (YANG; FIG. 5; ¶ 0046-47; “When a scene is scanned to obtain a point cloud, there is an occlusion between the scanned point and an object, and the object relatively close to the scanned point may occlude the object far from the scanned point. … the occluded point cloud data may include an occluded point cloud. To calculate the occluded point cloud data [‘to identify a set of obfuscated LiDAR points in the 3D space’], the point cloud data may be converted into a spherical coordinate system [‘projecting the modified LiDAR data from 3D space to a range image space’]. … the method of calculating the occluded point cloud data may include the following steps S510 and S520, … In step S510, the fused point cloud data is projected into a spherical coordinate system to obtain a corresponding spherical projection. The fused point cloud data is first subjected to a coordinate transformation from a Cartesian coordinate system to a spherical coordinate system, and then the spherical projection of the fused point cloud data may be obtained.”); and
filtering the set of obfuscated LiDAR points from the modified LiDAR data (YANG; FIGS. 4-5; ¶ 0048-50; “In step S420, the occluded point cloud data is removed from the fused point cloud data to obtain the to-be-detected sample. The occluded point cloud data may include coordinate information of … occluded point clouds, and with the occluded point cloud data, the occluded point cloud may be removed from the fused point cloud data to obtain the to-be-detected sample. The spherical projection in step S410 allows for the removal of the occluded point cloud data. … in step S410, if it is determined that for a sample point, the first distance is not greater than the second distance, the pixel point corresponding to the sample point may be updated to update, in the spherical projection, the distance between the pixel point and the origin to the first distance. After the updating of the spherical projection …, the distance of each pixel point in the spherical projection is the distance of the nearest point cloud to the origin, i.e., the pixel points in the spherical projection are all un-occluded point clouds, thus by using the spherical projection to perform inverse transformation, the point cloud may be transformed into a point cloud in a Cartesian coordinate system, so that a point cloud that does not include occluded point cloud data may be obtained as the to-be-detected sample.”).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over SPUREK in view of FEI as applied to claim 1 above, and further in view of Cao et al. ("Compression of Sparse and Dense Dynamic Point Clouds—Methods and Standards", published in 2021, 'CAO').
Regarding claim 10, SPUREK-FEI disclose the method of claim 1; however, SPUREK-FEI do not explicitly disclose that the method of claim 1 further comprises:
generating, during training, a binary cross entropy loss as at least a part of a vector quantization loss for training the code map, which CAO discloses (CAO; p. 1542, right column, first paragraph; “A learning-based point cloud geometry compression (PCGC) method, so-called learned-PCGC, was presented …, which comprised a set of stacked 3-D convolutions, dedicated to the extraction of latent features [‘code map’] and hyperpriors; a variational autoencoder model was proposed for accurate entropy modeling of latent features. … a weighted binary cross-entropy loss was used for training, and an adaptive thresholding scheme in inference was used for correct voxel classification.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of SPUREK-FEI to include the generating, during training, a binary cross entropy loss as at least a part of a vector quantization loss for training the code map of CAO. The motivation for this modification is to implement deep learning-based methods to achieve efficient compression for point clouds (CAO; p. 1542).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over SPUREK in view of FEI as applied to claim 1 above, and further in view of Dhariwal et al. ("Jukebox: A Generative Model for Music", published 2020, 'DHARIWAL').
Regarding claim 12, SPUREK-FEI disclose the method of claim 1; however, SPUREK-FEI do not explicitly disclose that the method of claim 1 further comprises:
detecting an unused set of codes that are unused during a training process of learning the code map; and
reactivating the unused set of codes during the training process of learning the code map, which DHARIWAL discloses (DHARIWAL; FIG. 5; pp. 2-3; § 3.1; “VQ-VAEs are known to suffer from codebook collapse, wherein all encodings get mapped to a single or few embedding vectors while the other embedding vectors in the codebook are not used [‘detecting an unused set of codes that are unused during a training process of learning the code map’], reducing the information capacity of the bottleneck. To prevent this, we use random restarts: when the mean usage of a codebook vector falls below a threshold, we randomly reset it to one of the encoder outputs from the current batch. This ensures all vectors in the codebook are being used and thus have a gradient to learn from, mitigating codebook collapse.” p. 9; § 5.4; “To mitigate codebook collapse, we restart dead codes [‘reactivating the unused set of codes during the training process of learning the code map’] near random encoder embeddings. … we see that this yields higher codebook usage even from early on in training.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of SPUREK-FEI to include the detecting an unused set of codes that are unused during a training process of learning the code map and the reactivating the unused set of codes during the training process of learning the code map of DHARIWAL. The motivation for this modification is to mitigate codebook collapse by restarting (reactivating) dead (unused) codes near random encoder embeddings. It is seen that this yields higher codebook usage even from early on in training. Models trained without random restarts can converge to the same test error and codebook usage but require more training steps. With poor initialization, these models sometimes end up with suboptimal codes hurting reconstruction fidelity (DHARIWAL, p. 9, left col.).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over SPUREK in view of FEI as applied to claim 1 above, and further in view of Thatte (U.S. PG-PUB 2023/0092198, 'THATTE').
Regarding claim 14, SPUREK-FEI disclose the method of claim 1; however, SPUREK-FEI do not explicitly disclose the following limitations, which THATTE discloses:
outputting the modified LiDAR data is to a component of a virtual driver of an autonomous system (THATTE; FIG. 7; ¶ 0091; “At 720, the system generates a second ground plane corresponding to the environment. The system can generate the second ground plane based on the machine learning system of the plane generator 450, and can generate the second ground plane [‘modified LiDAR data’] in accordance with the ground plane 340. The machine learning system can obtain … the foreground point(s) 320, and can generate the second ground plane based on fitting at least a portion of the ground plane to at least a portion of the foreground points 320. The second ground plane is not limited to an exact geometric plane, and can include variations, curvatures, and the like in accordance with a model corresponding to generating the first ground plane from the points. … present implementations can advantageously generate at least one second plane, and potentially multiple planes, within the time required to capture a single frame including a complete point cloud. This increases the responsiveness of an at least partially autonomous vehicle navigation or control system without increasing cost or expense of requiring sensor input and scanning output hardware with a framerate faster than the plane generation rate. … At 722, the system generates the second ground plane aligned to … point(s) satisfying the residual thresholds. At 724, the system generates the second ground plane based on the subset of the points detected from the environment and associated with a partial sweep. … At 730, the system calculates the geometry of the second ground plane.”), and wherein the method further comprises:
determining, by the virtual driver, an action of the autonomous system using the modified LiDAR data (THATTE; FIG. 7; ¶ 0092; “At 740, the system transmits … vehicle operation instruction(s) [which] can be based at least partially on a geometry of the second ground plane. … At 742, the system transmits … vehicle operation instruction(s) … [which] can include … instruction(s) to modify a path of the vehicle within the environment based on the ground plane. … the vehicle navigation instruction can include an instruction to modify a heading of the vehicle. At 744, the system transmits … vehicle operation instruction(s) [which] can include an environmental alarm including a proximity alarm, for example, and can include an instruction to modify a speed or heading of the vehicle, for example, in response to or in conjunction with the alarm instruction.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of SPUREK-FEI to include the outputting the modified LiDAR data to a component of a virtual driver of an autonomous system and the determining, by the virtual driver, an action of the autonomous system using the modified LiDAR data of THATTE. The motivation for this modification is to provide implementations that can advantageously generate second plane(s), and potentially multiple planes, within the time required to capture a single frame including a complete point cloud. This increases the responsiveness of an autonomous vehicle navigation system without increasing the cost or expense of requiring sensor input and scanning output hardware with a framerate faster than the plane generation rate (THATTE; ¶ [0091]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN M COFINO whose telephone number is (303) 297-4268. The examiner can normally be reached Monday-Friday 10A-4P MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN M COFINO/ Examiner, Art Unit 2614
/TERRELL M ROBINSON/ Primary Examiner, Art Unit 2614
Prosecution Insights