DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Claims 3 and 15-20 have been canceled and as such all rejections to the canceled claims are withdrawn as moot. Claims 1-2 and 4-14, and new claims 21-27 are currently under consideration.
Response to Arguments
Applicant's arguments filed July 31, 2026, have been fully considered but they are not persuasive.
Regarding claim 1, applicant argues that Wang fails to teach “processing the BEV map using a BEV spatial prior model to generate one or more priors; and generating a refined BEV map based on a detector score and a prior score, wherein the detector score is based on the BEV map and wherein the prior score is based on the one or more priors” (see pages 8-9 of applicant’s correspondence filed July 31, 2026). Examiner respectfully disagrees.
First, Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. Second, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., any particular meaning of “prior score” and/or “detector score”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). As the claim is currently drafted, the claim fails to clarify what the “prior score” or “detector score” refer to or how the particular scores are calculated or used by the claimed invention. Instead, the terms are merely included in the claim without any explicit definition, either in the claim or the specification. Without any language defining the different scores or their particular use, the claim is at most merely reciting that values related to any prior or any detector, whether merely identifying points for calculations or otherwise, are intended to be read on by the claims. Wang teaches a birds-eye-view map refining technique that builds on existing technologies for generating a BEVmap (namely the Lift-Splat technique of projecting 2D pixel data into a 3D space with estimated possible depths and then projecting the 3D values into a BEVmap – see generally Introduction of Wang). From the generated BEV map, Wang then discloses a technique to refine the generated BEV map by reprojecting the BEV map into a 3D frustrum space and performing refinement on the reprojected data. In particular, Wang teaches as part of generating a refined BEV map using a detector score and a prior score, wherein the detector score is based on the BEV map and wherein the prior score is based on the one or more priors (Wang, p.15098, equation (5) obtains a reprojected image feature by summing the weightings at each pixel, i.e. “prior score”; Wang, p. 1598, left column, “2) Efficient Reprojection With TopK Depth Selection” discloses reducing the reprojection to only frustum points having top K refined depth distribution weights – where K is interpreted as “detector score”, which is based on the BEV map due to the BEV reprojection discussed on p.15097, right column, section “C. Camera-View Feature Refinement”). As there is no clarification of the individual scoring or what priors are used, instead reciting the terms very broadly and leaving up to the broadest reasonable interpretation, the values provided by Wang for refinement to generate a refined BEV map are read on by the claims. Accordingly, applicant’s arguments are not persuasive.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 5 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 5 recites the limitation "the BEV detector” in line 3. There is insufficient antecedent basis for this limitation in the claim. Moreover, the claim is indefinite as to whether the BEV detector generates the detector score, or merely provides probabilities, and furthermore what the probabilities are related to ad determined by the BEV detector. Without further clarification, the claim is indefinite.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7-10, 4-14, and 21-27 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Wang et al. (B. Wang et al., "BEVRefiner: Improving 3D Object Detection in Bird’s-Eye-View via Dual Refinement," in IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 10, pp. 15094-15105, Oct. 2024, doi: 10.1109/TITS.2024.3394550, Date of Publication May 10, 2024) in view of
Liu et al. (US 2021/0150203 A1).
Regarding claim 1, Wang discloses:
An apparatus for generating map data (Wang, Abstract: transform image feature into BEV; p. 15095, “II Related Work”, Section A. discloses camera based 3D object detection for autonomous vehicles), the apparatus comprising:
at least one memory (Wang, p. 15103, GPU memory); and
process sensor data representative of a scene using a bird’s-eye-view (BEV) detector to generate a BEV map of the scene (Wang, p. 15094, Introduction, ¶2: view transformation models transform image features to BEV, lifting 2D camera-view features into 3D pseudo point cloud space and “splat” 3D features into a rasterized BEV grid; p. 15096, Fig. 2 shows image features as input to lift and splat to generate BEV feature grid; p. 15096, left col., lines 6-10: “LSS [9] is a milestone in depth-based approaches. It ‘lifts’ each image features individually into a frustum by explicitly predicting depth distribution for each pixel first, and then “splats” multiple frustums onto a unified BEV plane.”; pp. 15096-15097, “III. Methodology” section, ¶1 discloses BEVDet extracting camera-view features, using an LSS pipeline to predict image depth distribution for each pixel and projecting the multi-view image features into 3D frustrum space and construct the BEV feature; Also 15097, section “A.” discloses obtaining the BEV representation B created from Lift-Splat algorithm);
process the BEV map using a BEV spatial prior model to generate one or more priors; (Wang, p. 15095, left column, lines 6-8: “we propose to reproject the BEV grid feature back to each camera view for refining the original 2D image features”; p. 15097, right column, section C, ¶1: “To this end, we propose to reproject the BEV feature B ∈ RC×X×Y back to each camera view to further enhance its 2D image feature, to obtain reprojected image feature Fip, where p.15098, 1st paragraph discloses that to reproject features back to the 2D image plane, each pixel is weighted using Drj depth distribution values which indicates the contribution value of the BEV feature – i.e. “priors”) and
generate a refined BEV map (Wang, p. 15095, left column, lines 20-24: “the refined depth distribution and camera-view features can be used to perform the second-stage view transformation and obtain refined BEV feature, hence leading to more accurate 3D object detection”; p. 15096, end of left column, “1)” discloses BEVRefiner improving quality of BEV features by refining both depth distribution and camera-view features’; p. 15097, left column, lines 10-17 discloses employing dual refinement techniques based on the BEV reprojection, and after that utilizing the refined depth and camera features to refactor the BEV feature – See “Refined BEV feature” in Fig. 2 on page 15096; p. 15098, 3rd full paragraph “As for the reprojected image feature Fpi , we use it to refine the original image feature Fi and obtain the refined image feature Fir ∈ RC×HF×WF ; and Finally, we perform the second-stage view transformation with the refined depth distributions {Dir}i=1N and the refined image features {Fir}i=1N to obtain the refined BEV feature Br ∈ RC×X×Y for final 3D object detection”) based on a detector score and a prior score, wherein the detector score is based on the BEV map and wherein the prior score is based on the one or more priors (Wang, p.15098, equation (5) obtains a reprojected image feature by summing the weightings at each pixel, i.e. “prior score”; Wang, p. 1598, left column, “2) Efficient Reprojection With TopK Depth Selection” discloses reducing the reprojection to only frustum points having top K refined depth distribution weights – where K is interpreted as “detector score”, which is based on the BEV map due to the BEV reprojection discussed on p.15097, right column, section “C. Camera-View Feature Refinement”)
The only limitations not explicitly taught by Wang are directed to the particular computer architecture claimed as implementing the computer system. Although these are obvious, if not inherently required limitations for implementing the computer software system as taught by Wang, the Office additionally relies upon Liu as teaching the additional elements.
Liu discloses:
An apparatus comprising at least one memory; and at least one processor coupled to the at least one memory and configured to process sensor data representative of a scene using a bird’s-eye-view (BEV) detector to generate a BEV map of the scene (Liu, ¶6: system including one or more processor devices and memory in communication with at least one or more processor devices, for converting digital images into top-down view image; Fig. 6 and ¶154: processing system 600 including at least one processor and memory)
Both Wang and Liu are directed to BEV map generation systems. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, by utilizing the common computer architecture for implementing an image processing system as provided by Liu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing common computer components for easier and/or less expensive implementation, as well as allowing integration of the system into a computer for actual use by a consumer or other type of user.
Regarding claim 26, the apparatus of claim 1 performs the method of claim 26 and as such claim 26 is rejected based on the same rationale as claim 26 set forth above.
Regarding claim 27, the claim recites a non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform the same operations as the method recited in claim 26. As such, the recited operations are taught by Wang modified by Liu for the same rationale as claim 26 set forth above. Further regarding claim 27, Liu further discloses:
A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform operations (Liu, ¶7 and Fig. 6 and ¶¶153-155, ¶170 and ¶174)
Both Wang and Liu are directed to BEV map generation systems. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, by utilizing the common computer architecture for implementing an image processing system as provided by Liu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing common computer components for easier and/or less expensive implementation, as well as allowing integration of the system into a computer for actual use by a consumer or other type of user.
Regarding claim 7, Wang further discloses:
wherein, to generate the BEV map, the BEV detector is configured to: encode the sensor data to generate features (Wang, p. 15097, section A, “1) Lift” Given the camera-view image features Fi ∈ RC×HF×WF N i=1 , they cannot be directly projected to 3D space without depth information. LSS-based methods first estimate depth distribution Di ∈ RD×HF×WF for each image from the corresponding 2D image feature Fi using Eq. 1, where D represents the number of depth bins: Di =ψ(Fi). (1) Here, ψ is a lightweight DepthNet which consists of one 1 ×1 convolution layer. Then, the frustum feature Gi ∈ RC×D×HF×WF is con structed by associating image features to the depth distribution as Eq. 2: Gi =Fi ⊗Di,);
project the features into BEV space to generate projected features (Wang, p. 15097, section A, 2nd to last paragraph: camera intrinsic and extrinsic matrices are used to project the frustum features Gi to a 3D voxel representation Vi ∈ RC×X×Y×Z, tied to Lift-Splat steps of projecting features into 3D space in Lift-Splat algorithm); and
decode the projected features to generate the BEV map (Wang, p. 15097, section A, last paragraph: Splat: BEV pooling [1] is used to collapse the voxel features to a single height plane to create the BEV representation B ∈RC×X×Y. Concretely, the voxel features within the same “pillar” are summed up as the pooled BEV feature)
Regarding claim 8, Wang further discloses:
wherein the BEV map comprises a plurality of pixels, and wherein each pixel of the plurality of pixels comprises a value indicative of a probability that the respective pixel represents a class (Wang, p. 15097, Section A Review of LSS-Based View Transformation, 1st paragraph: “Lift” projects 2D camera-view features into the 3D voxel space according to the depth distribution probability. Next, “Splat” uses BEV pooling to transform the 3D features into a BEV grid via compressing the height dimension.”; Further note right column of p. 15097 discusses the refinement resulting in a refined depth distribution probability:
The overall depth refinement can be written in Eq. 3: Dr i = Softmax(FC(Conv(Conv(Conv(Di))))), (3) where softmax normalization is used to generate the refined depth distribution probability. Our proposed depth refinement module can aggregate plentiful structural information to correct those inaccurately predicted depth distribution. On the one hand, by stacking 3D convolution modules, depth prediction is equipped with a large receptive field, theoretically enabling the learning of local 3D relationship among adjacent pixels and depth bins. On the other hand, the final FC layer introduces the global relationship learning across all D different depth bins. More importantly, instead of performing refinement on the 3D frustum feature space with shape C × D × HF × WF like BEVDepth [2], our design conducts refinement on the depth distribution space with shape.
In other words, refinement of the BEV depth distribution based on depth bins or class of a pixel)
Regarding claim 9, Wang modified by Liu further discloses:
wherein the BEV map comprises a plurality of pixels, wherein each pixel of the plurality of pixels comprises a respective vector of values, and wherein a vector of values of a pixel of the plurality of pixels indicates probabilities that the pixel represents classes (Liu, ¶73: for each pixel in perspective image space, a depth value is provided representing the predicted absolute depth of the pixel; ¶75: segmentation mask based on foreground classes; ¶76:
A top-view image of the road can be generated by (1) back-projecting all the road pixels into a 3D point cloud and (2) projecting all the 3D points onto the x-y plane. This mapping can be used to transfer the semantic class probability distribution of each pixel from perspective-view into top-view (bird's eye view (“bev”)), and is referred to as bev and denote as x.sup.r∈custom-character.sup.H×W×C, where C is the number of semantic classes, and in various embodiments H=128 and W=64 pixels, for example, which can relate to 60×30 meters in the point cloud in a non-limiting exemplary embodiment.
¶80: “Given the depth map and the intrinsic camera parameters, each coordinate of the perspective view can be mapped into the 3D space. The z-coordinate (height axis) can be dropped for each 3D point, and assign x and y coordinates to the closest integer, which gives us a mapping into bird's eye view (bev) representation. Each pixel can be mapped to some geometrically meaningful top-view space. We use this mapping to transfer the class probability distribution of each pixel in the perspective view into the bird's eye view.”; Also note ¶104 discloses neural network changing the input image to its own representation for each pixel having a vector of length C, and ¶110 discloses the vector used for neural network using weighted combination function that combines features from a current frame with previous transformed frames with applied weights)
Both Wang and Liu are directed to BEV map generation systems. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, by further using the data values for mapping data points in converting images to BEV as further provided by Liu, using known electronic interfacing and programming techniques. The modification merely substitutes one known type of mapping data for transforming images to BEV for another, yielding predictable results of using depth class for differentiating elements within images. The modification results in an improved system for BEV map generation by classifying depth data top better account for differentiation between image objects and improving the results of the BEV map with more relevant information for a user.
Regarding claim 10, Wang further discloses:
wherein the refined BEV map comprises a plurality of pixels, wherein each pixel of the plurality of pixels comprises a respective indication of a class. (Wang, p. 15097, Section A Review of LSS-Based View Transformation, 1st paragraph: “Lift” projects 2D camera-view features into the 3D voxel space according to the depth distribution probability. Next, “Splat” uses BEV pooling to transform the 3D features into a BEV grid via compressing the height dimension.”; Further note right column of p. 15097 discusses the refinement resulting in a refined depth distribution probability:
The overall depth refinement can be written in Eq. 3: Dr i = Softmax(FC(Conv(Conv(Conv(Di))))), (3) where softmax normalization is used to generate the refined depth distribution probability. Our proposed depth refinement module can aggregate plentiful structural information to correct those inaccurately predicted depth distribution. On the one hand, by stacking 3D convolution modules, depth prediction is equipped with a large receptive field, theoretically enabling the learning of local 3D relationship among adjacent pixels and depth bins. On the other hand, the final FC layer introduces the global relationship learning across all D different depth bins. More importantly, instead of performing refinement on the 3D frustum feature space with shape C × D × HF × WF like BEVDepth [2], our design conducts refinement on the depth distribution space with shape.
In other words, refinement of the BEV depth distribution based on depth bins or class of a pixel)
Regarding claim 14, Wang further discloses:
wherein the sensor data comprises at least one of: image data; light detection and ranging (LIDAR) data; or radio detection and ranging (RADAR) data (Wang, Abstract and Fig. 2 on p. 15096 discloses image data)
Regarding claim 21, Wang modified by Liu further discloses:
Wherein the BEV map comprises class values indicative of probabilities that pixels of the BEV map represent various classes (Liu, ¶76: top-view image of the road can be generated by (1) back-projecting all the road pixels into a 3D point cloud and (2) projecting all the 3D points onto the x-y plane. This mapping can be used to transfer the semantic class probability distribution of each pixel from perspective-view into top-view (bird's eye view (“bev”)), and is referred to as bev and denote as xr, where C is the number of semantic classes, and in various embodiments H=128 and W=64 pixels, for example, which can relate to 60×30 meters in the point cloud; Also ¶¶79-80 discloses context cues for priors and constraints for layout estimation in top view, including prior probability distribution of context cues and transferring the class probability distribution of each pixel into perspective view into the bird’s eye view)
Both Wang and Liu are directed to BEV map generation systems. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, and by further utilizing class probability for pixels when generating the BEV map as further provided by Liu using known electronic interfacing and programming techniques. The modification merely substitutes one known technique for generating a BEV map for refinement in place of another, yielding predictable results of using a known probability classification technique for generating BEV map data. Furthermore, the modification results in an improved system for BEV map generation by further using statistical weightings to better estimate the characteristics of elements to better define resulting components instead of strict one-to-one correspondence that includes mismatched or improper data lost in the 2D to 3D conversion.
Regarding claim 22, Wang modified by Liu further discloses:
Wherein the at least one processor is configured to generate the detector score based on the class values of the BEV map (Liu, ¶80: “Each pixel can be mapped to some geometrically meaningful top-view space. We use this mapping to transfer the class probability distribution of each pixel in the perspective view into the bird's eye view”)
Examiner notes that it is this teaching of the creation of BEV map initially that uses class values within the BEV map, as taught by Liu, combined with the refinement of such a map that reprojects the data for refinement (See, Wang, p.15098, equation (5) obtains a reprojected image feature by summing the weightings at each pixel, i.e. “prior score”; Wang, p. 1598, left column, “2) Efficient Reprojection With TopK Depth Selection” discloses reducing the reprojection to only frustum points having top K refined depth distribution weights – where K which is top K depth of a range provided by the frustrum points is interpreted as “detector score”, which is based on the BEV map due to the BEV reprojection discussed on p.15097, right column, section “C. Camera-View Feature Refinement”) that teaches the limitation of the claim (i.e. the detector score of top weights within range generated by reprojection of BEV map pixels, which would are based on original class probabilities of Liu).
Both Wang and Liu are directed to BEV map generation systems. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, and by further utilizing class probability for pixels when generating the BEV map as further provided by Liu using known electronic interfacing and programming techniques. The modification merely substitutes one known technique for generating a BEV map for refinement in place of another, yielding predictable results of using a known probability classification technique for generating BEV map data. Furthermore, the modification results in an improved system for BEV map generation by further using statistical weightings to better estimate the characteristics of elements to better define resulting components instead of strict one-to-one correspondence that includes mismatched or improper data lost in the 2D to 3D conversion.
Regarding claim 23, Wang modified by Liu further discloses:
Wherein the BEV spatial prior model comprises a trained machine learning model (Wang, Abstract: BEVRefiner model for performing refinement; p.15099, left column, “Implementation Details”: “We implement BEV Refiner using Pytorch and build upon the MMDetection3D framework” where BEVRefiner is a trained model)
Regarding claim 24, Wang modified by Zhu further discloses:
Wherein the BEV spatial prior model is trained based on ground-truth BEV maps to predict whether input BEV maps reflect real scenes (Wang, p. 15095, Fig. 1 discloses comparing ground truth and predictions of BEV refiner; p. 15101, Fig. 4: “Object detection visualization results of the baseline [1]model and our proposed BEVRefiner on the nuScenes [24] val dataset. We show the predicted and ground-truth boxes on surrounding images from six perspectives (left) and the LiDAR-topview (right). The predicted and ground-truth objects are shown in red and green boxes.”)
Examiner notes that Wang clearly teaches obtaining ground-truth BEV maps and a comparison of the generated BEV maps to the ground-truth BEV maps to determine accuracy of the model. The details of the training are not completely clear from Wang alone. The use of ground-truth training, however, is a well-known and conventional technique for machine learning (see Zhu ¶143).
Liu further discloses:
Wherein the prior model is trained based on ground-truth maps to predict whether input reflect maps real scenes (Liu ¶24: a deep learning framework can leverage training data from both domains, real and simulated, to infer the parameters of a scene model, including gathering ground truth maps for training; ¶100: neural network trained to predict road features; ¶¶141-143: given data sets of real data, use a loss function to compare predictions of the scene attributes/parameters with ground truth, and minimizing error/loss for standard supervised learning of neural networks)
Both Wang and Liu are directed to BEV map generation systems. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, by utilizing the common computer architecture for implementing an image processing system as well as the technique for ground truth supervised training for machine learning as provided by Liu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing common computer components for easier and/or less expensive implementation, as well as allowing integration of the system into a computer for actual use by a consumer or other type of user. In addition, the utilizing of the supervised training allows for more accurate results by ensuring that the machine learning develops more accurate results over time and has measurable differences to ensure the machine learning is guided to more accurate results.
Regarding claim 25, Wang further discloses:
wherein the one or more priors are indicative of a likelihood that the BEV map is accurate (Wang, p.15098, 1st paragraph: Here, we weight the reprojected feature with the corresponding refined depth distribution value Dr i (dj,uj,vj). A large/small depth weight means this frustum point has large/small contribution to this BEV feature.)
Claim(s) 2, 4-5, and 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Wang et al. (B. Wang et al., "BEVRefiner: Improving 3D Object Detection in Bird’s-Eye-View via Dual Refinement," in IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 10, pp. 15094-15105, Oct. 2024, doi: 10.1109/TITS.2024.3394550, Date of Publication May 10, 2024) in view of
Liu et al. (US 2021/0150203 A1) and in further view of
Zhu et al. (ZHU X., et al., "MapPrior: Bird's-Eye View Map Layout Estimation with Generative Models", 2023 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, 1 October 2023, pp. 8194-8205, XP034515342 – Applicant provided IDS reference, filed 3/12/2026).
Regarding claim 2, the limitations included from claim 1 are rejected based on the same rationale as claim 1 set forth above. Further regarding claim 2, Zhu discloses:
wherein, to generate the refined BEV map, the at least one processor is configured to optimize an objective including the detector score and the prior score (Zhu, p. 8194, last paragraph: initial layout estimate from sensory input, using MapPrior and initial estimate to sample one or multiple refined layouts; p. 8195, “Generative Models”: generative BEV perception involves learning a map prior and performing conditional sampling using deep generative models, using vector quantized generative models; p. 8196 Fig. 2 and section 3.2 Inference: generative stage takes noisy estimate y’ to encode as latent code z’ and generate realistic samples; p. 8197, “Generative Stage” ¶¶1-2: incorporate generative map prior built using VQGAN architecture, where autoregressive scheme progressively samples latent code z’ outputting token’s conditional probability of encoded latent space z’, and used finally to form final layout estimation y(k)).
Examiner notes that the combination of Wang with Zhu teaches the use of the detector score and prior score which are built into the map refinement of Wang included within the refined map generated by Wang and utilizing the optimization of Zhu, as the claim does not provide any guidance as to how the objective includes the scorings, other than the data being used in any manner, such as within the map data itself which is optimized by a process provided by Zhu.
Both Wang and Zhu are directed to BEV map generation and refinement. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, by further utilizing the generative modeling technique of Zhu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing a more accurate realistic results by utilizing machine learning that using an improved generative model for processing data (see e.g. Zhu Abstract).
Regarding claim 4, Wang modified by Liu and Zhu further discloses:
wherein, to optimize the objective, the at least one processor is configured to solve an optimization problem based on the detector score and the prior score (Zhu, p. 8197, “Generative Stage” ¶¶1-2: incorporate generative map prior built using VQGAN architecture, where autoregressive scheme progressively samples laten code z’ outputting token’s conditional probability of encoded latent space z’, and used finally to form final layout estimation y(k))
Both Wang and Zhu are directed to BEV map generation and refinement. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, by further utilizing the generative modeling technique of Zhu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing a more accurate realistic results by utilizing machine learning that using an improved generative model for processing data (see e.g. Zhu Abstract)
Regarding claim 5, Wang modified by Liu and Zhu further discloses:
wherein the detector score comprises a likelihood that the BEV map is accurate given probabilities determined by the BEV detector (Zhu, p. 8194, last paragraph: initial layout estimate from sensory input, using MapPrior and initial estimate to sample one or multiple refined layouts; p. 8195, “Generative Models”: generative BEV perception involves learning a map prior and performing conditional sampling using deep generative models, using vector quantized generative models; p. 8196 Fig. 2 and section 3.2 Inference: generative stage takes noisy estimate y’ to encode as laten code z’ and generate realistic samples; p. 8197, “Generative Stage” ¶¶1-2: incorporate generative map prior built using VQGAN architecture, where autoregressive scheme progressively samples latent code z’ outputting token’s conditional probability of encoded latent space z’, and used finally to form final layout estimation y(k) – note samples decoded as estimation)
Both Wang and Zhu are directed to BEV map generation and refinement. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, by further utilizing the generative modeling technique of Zhu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing a more accurate realistic results by utilizing machine learning that using an improved generative model for processing data (see e.g. Zhu Abstract)
Regarding claim 11, the limitations included from claim 1 are rejected based on the same rationale as claim 1 set forth above. Further regarding claim 11, Zhu discloses:
wherein the BEV spatial prior model comprises a generative likelihood-based machine-learning model (Examiner notes that Applicant admits that the type of machine learning model recited is well-known in the art, citing to publications that explain such machine learning models – see e.g. ¶74 of PG-Pub 2026/0073577 A1 of applicant’s specification; However, Zhu p. 8195, “Generative Models” section discloses using variational autoencoders and/or flow-based models for use in generative BEV perception infloving learning map prior)
Both Wang and Zhu are directed to BEV map generation and refinement. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, by further utilizing the generative modeling technique of Zhu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing a more accurate realistic results by utilizing machine learning that using an improved generative model for processing data (see e.g. Zhu Abstract)
Regarding claim 12, the limitations included from claim 1 are rejected based on the same rationale as claim 1 set forth above. Further regarding claim 12, Zhu discloses:
wherein the BEV spatial prior model comprises at least one of: a normalizing flow machine-learning model; a variational autoencoder machine-learning model; or an autoregressive machine-learning model. (Examiner notes that Applicant admits that the type of machine learning model recited is well-known in the art, citing to publications that explain such machine learning models – see e.g. ¶74 of PG-Pub 2026/0073577 A1 of applicant’s specification; However, Zhu p. 8195, “Generative Models” section discloses using variational autoencoders and/or flow-based models for use in generative BEV perception infloving learning map prior)
Both Wang and Zhu are directed to BEV map generation and refinement. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, by further utilizing the generative modeling technique of Zhu, using known electronic interfacing and programming techniques. The modification results in an improved system for BEV map generation by utilizing a more accurate realistic results by utilizing machine learning that using an improved generative model for processing data (see e.g. Zhu Abstract)
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Wang et al. (B. Wang et al., "BEVRefiner: Improving 3D Object Detection in Bird’s-Eye-View via Dual Refinement," in IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 10, pp. 15094-15105, Oct. 2024, doi: 10.1109/TITS.2024.3394550, Date of Publication May 10, 2024) in view of
Liu et al. (US 2021/0150203 A1) and in further view of
Choi et al. (US 2024/0312219 A1).
Regarding claim 13, the limitations included from claim 1 are rejected based on the same rationale as claim 1 set forth above. Further regarding claim 12, Wang discloses use of ground truth for predictions (e.g. Fig. 1 on page 15095). However, Wang does not explicitly teach the use of unsupervised training for machine learning models.
Choi discloses:
wherein the BEV spatial prior model is trained using an unsupervised training process based on ground-truth BEV maps (Choi, ¶63: machine learning models trained using ground truth data 605 represetning BEV map indicating the locations of objects, where ground truth can be synthetically produced, machine automated, etc.; ¶69: The machine learning model(s) 106 may then use one or more layers (e.g., one or more layers associated with the transformer component 112) to transform the feature map(s) 110 to a BEV feature map(s) 116; ¶177: machine learning models training may be executed using unsupervised training) Abstract: generate a BEV from feature values corresponding to a plurality of views;
Both Wang and Choi are directed to BEV map generation. It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the system and technique for generating a refined BEV map as provided by Wang, utilizing the common computer architecture for implementing an image processing system as provided by Liu, by further utilizing the unsupervised generative modeling technique of Choi, using known electronic interfacing and programming techniques. The modification merely substitutes one type of machine learning for another, yielding predictable results of using commonly used and known unsupervised machine learning. The modification results in an improved system for BEV map generation by utilizing a more accurate realistic results by utilizing machine learning that using an improved generative model for processing data while requiring less user input or human intervention for ease-of-use.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Wang et al. (B. Wang et al., "BEVRefiner: Improving 3D Object Detection in Bird’s-Eye-View via Dual Refinement," in IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 10, pp. 15094-15105, Oct. 2024, doi: 10.1109/TITS.2024.3394550, Date of Publication May 10, 2024) in view of
Liu et al. (US 2021/0150203 A1) and
cited as evidentiary support, Anunay et al. (Anunay, Pankaj and C. Dhiman, "DepthNet: A Monocular Depth Estimation Framework," 2021 International Conference on Engineering and Emerging Technologies (ICEET), Istanbul, Turkey, 2021, pp. 1-6)
Regarding claim 6, the limitations included from claim 1 are rejected based on the same rationale as claim 1 set forth above. Wang further discloses:
wherein the BEV detector is trained to generate BEV maps based on sensor data (Wang, p. 15094, right column, 1st full paragraph: “LSS (Lift-Splat-Shoot) [9] is one of the most straightforward and widely used approaches. Concretely, it first utilizes a DepthNet to predict latent depth distribution for each image pixel and then “lifts” 2D camera-view features into a 3D pseudo point cloud space according to the depth distribution probability.”)
As evidentiary support, Anunay discloses that DepthNet is a trained deep learning method to generate depth maps from RGB image – see Abstract of Anunay)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM A BEUTEL whose telephone number is (571)272-3132. The examiner can normally be reached Monday-Friday 9:00 AM - 5:00 PM (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DANIEL HAJNIK can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM A BEUTEL/Primary Examiner, Art Unit 2616
