Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response
The RCE is entered.
The applicant’s IDS is considered.
A new rejection is made based on the IDS.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 11 and 18 are rejected under 35 U.S.C. sec. 102(a(2) as being anticipated by NPL, Ku, Jason et al., Joint 3d Proposal Generation and object detection from view aggregation, 2010 IEEE
PNG
media_image1.png
628
1556
media_image1.png
Greyscale
In regard to claim 1 and 11 and 18, Ku discloses “...1. One or more processors comprising processing circuitry to:
generate, based at least on one or more Neural Networks (NNs) processing data representing” (see FIG. 2 where the neural network can receive a 3d view from one or more neural networks to process the scene around the vehicle)
“... “multiple views of LiDAR data generated using at least one LiDAR sensor of an ego-machine, (see page 3-5 and table II where different LIDAR views and slices can be processed by the multiple different neural networks to render the scene and identify the pedestrians and parked cards and then the moving vehicles )
First extracted feature data in a first view of the multiple views and second extracted feature data in a
second view of the multiple views; ( See table III where the neural networks can provide different views of the scene to detect different features of the scene and fig. 1 where the vehicle can provide data from a first view and a second view and slices of the views to identify the pedestrians and cyclists and vehicles and parked cars with millions of points of data in the point cloud)
generate combined extracted feature data combining the first extracted feature data in the first
view with a projected representation of the second extracted feature data projected into the first view; (see Table III where the different architectures can provide different extracted feature data representations that are protected on the same view such as non ordered corners and feature extraction and a pyramid scheme and see To extract feature crops for every anchor from the view specific feature maps, we use the
crop and resize operation [18]. Given an anchor in 3D, two regions of interest are obtained by projecting the anchor onto the BEV and image feature maps. The corresponding regions are then used to extract feature map crops from each view, which are then bilinearly resized to 3 × 3 to
obtain equal-length feature vectors. This extraction method results in feature crops that abide by the aspect ratio of the projected anchor in both views, providing a more reliable feature crop than the 3 × 3 convolution used originally by
Faster-RCNN )
generate, based at least on the one or more NN s processing the combined extracted feature data, one or more outputs; and (see page 4-5 where the features of the scene and provided to determine and classify the target features using the BMV mapping and the image maps which are cropped and resized and then fused to detect the features)
“..control one or more operations of the ego-machine based at least on the one or more outputs”. ( see page 5 where the vehicle can move I the environment).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2-4 and 10, 12-14 and 17 are rejected under 35 U.S.C. sec. 103 as being unpatentable as obvious in view of by Ku and in view of NPL, Hu, Fangchao, Continuous Point Cloud Stitch Based on Image Feature Matching Constraint and Score, IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL. 4, NO. 3, SEPTEMBER 2019 363
(https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8723434) which is prior to 9-13-24 (hereinafter “HU”) .
PNG
media_image2.png
482
1282
media_image2.png
Greyscale
In regard to claim 2 and 12, Hu discloses “...2. The one or more processors of claim 1, wherein the data representing at least one of
the first view or the second view that is processed using the one or more NN s comprises a number of channels corresponding to a number of returns supported by the at least one LiDAR sensor”. (see FIG. 4 where the first point cloud is formed and a second point cloud is formed and at page 1-2 and 10 where the scene is captured by a first LIDAR sensor and then a second LIDAR sensor for a first and a second point cloud and then these are merged via a 3d point cloud stitching and see footnote 27 where machine learning can be used which includes channels )
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings Hu since Hu teaches that the point cloud may use different channels for point cloud stitching of the different features. A first LIDAR sensor can capture a first scene and a second can capture a second scene and then the machine learning can aggregate via a 3d stitching operation to recognize different occluded features.
In regard to claim 3 and 13, Hu discloses “...3. The one or more processors of claim 1, wherein the processing circuitry is further to
generate the combined extracted feature data based at least on projecting multiple frames of the
second extracted feature data representing sequential LiDAR scans from the second view into the first view. (see FIG. 4 where the first point cloud is formed and a second point cloud is formed and at page 1-2 and 10 where the successive frames are provided and a scene is captured by a first LIDAR sensor and then a second LIDAR sensor for a first and a second point cloud and then these are merged via a 3d point cloud stitching and see footnote 27 where machine learning can be used which includes channels )
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings Hu since Hu teaches that the point cloud may use different channels for point cloud stitching of the different features. A first LIDAR sensor can capture a first scene and a second can capture a second scene and then the machine learning can aggregate via a 3d stitching operation to recognize different occluded features.
In regard to claim 4 and 14, Hu discloses “...4. The one or more processors of claim 1, wherein the second extracted feature data comprises an intermediate representation extracted using the one or more NNs, wherein the
processing circuitry is further to project the intermediate representation from the second view to the
first view. (see FIG. 1 where the first and the second view provides an intermediate representation that is down sampled and then aligned and see FIG. 4 where the first point cloud is formed and a second point cloud is formed and at page 1-2 and 10 where the successive frames are provided and a scene is captured by a first LIDAR sensor and then a second LIDAR sensor for a first and a second point cloud and then these are merged via a 3d point cloud stitching and see footnote 27 where machine learning can be used which includes channels ) It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings Hu since Hu teaches that the point cloud may use different channels for point cloud stitching of the different features. A first LIDAR sensor can capture a first scene and a second can capture a second scene and then the machine learning can aggregate via a 3d stitching operation to recognize different occluded features.
Claims 5 and 15 are rejected under 35 U.S.C. sec. 103 as being unpatentable as obvious in view of by Ku and in view of NPL, Hu, Fangchao, Continuous Point Cloud Stitch Based on Image Feature Matching Constraint and Score, IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL. 4, NO. 3, SEPTEMBER 2019 363
(https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8723434) which is prior to 9-13-24 (hereinafter “HU”) and in view of
United States Patent Application Pub. No.: US20230063828A1 to Bao that was filed in 2020.
In regard to claim 5 and 15, Bao teaches “...5. The one or more processors of claim 1, wherein the processing circuitry is further to
generate, based at least on the one or more NN s processing the second extracted feature data, artifact
data comprising a number of channels of artifact classification data corresponding to a number of supported LiDAR returns encoded by the second view”. (see claim 7 and paragraph 52-59).
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings of He with a reasonable expectation of success since BAO of BAIDU™ teaches that an artifact can be provided with the neural network to determine a number of artifacts for a degree of image quality. A number of artifacts can be applied into different types. This can be a bar, ring, motion and then the model can be trained for that artifact and an image quality can be evaluated. See paragraph 52-55 and claims 1-8. Poor image quality can be excluded.
Claims 6-7 are rejected under 35 U.S.C. sec. 103 as being unpatentable as obvious in view of by Ku and in view of NPL, Hu, Fangchao, Continuous Point Cloud Stitch Based on Image Feature Matching Constraint and Score, IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL. 4, NO. 3, SEPTEMBER 2019 363
(https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8723434) which is prior to 9-13-24 (hereinafter “HU”) and in view of U.S. Patent No.: 11,608,078 B2 to He et al. that was filed in 2019.
He teaches “...6. The one or more processors of claim 1, wherein the one or more outputs represent
one or more detected obstacles extracted based at least on the combined extracted feature data, and
the one or more operations of the ego-machine comprise path planning based at least on the one or
more detected obstacles”. (see col. 7, line 48 to col. 8, lines 45 and this is based on the spliced point clouds in col. 1, lines 55-60).
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings of He with a reasonable expectation of success since He of BAIDU™ teaches that a number of multiple point clouds can be generated with multiple poses. This is from a LIDAR sensor. A pose graph algorithm can be provided and a stitching of the point clouds can be provided for the autonomous vehicle. This can provide a sharing of the point clouds that are combined via a pose graph to provide a safe operation for the vehicle to move through in a safe manner. See claims 1-3 and the abstract of He.
PNG
media_image3.png
478
1370
media_image3.png
Greyscale
Hu discloses “...7. The one or more processors of claim 1, wherein the one or more outputs represent a detected navigable space extracted based at least on the combined extracted feature data”, (see FIG. 4 where the first point cloud is formed and a second point cloud is formed and at page 1-2 and 10 where the scene is captured by a first LIDAR sensor and then a second LIDAR sensor for a first and a second point cloud and then these are merged via a 3d point cloud stitching and see footnote 27 where machine learning can be used which includes channels ) (see page 1-2 and FIG. 1 where the point cloud can be generated from the multiple views and by triangulating the target and then aligning and down sampling the point clouds to match them and then merge them by feature mapping) It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings Hu since Hu teaches that the point cloud may use different channels for point cloud stitching of the different features. A first LIDAR sensor can capture a first scene and a second can capture a second scene and then the machine learning can aggregate via a 3d stitching operation to recognize different occluded features.
He teaches “...and the one or more operations of the ego-machine comprise path planning based at least on the detected navigable space. (see col. 7, line 48 to col. 8, lines 45 and this is based on the spliced point clouds in col. 1, lines 55-60)”.
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings of He with a reasonable expectation of success since He of BAIDU™ teaches that a number of multiple point clouds can be generated with multiple poses. This is from a LIDAR sensor. A pose graph algorithm can be provided and a stitching of the point clouds can be provided for the autonomous vehicle. This can provide a sharing of the point clouds that are combined via a pose graph to provide a safe operation for the vehicle to move through in a safe manner. See claims 1-3 and the abstract of He.
Claim 8 is rejected under 35 U.S.C. sec. 103 as being unpatentable as obvious in view of by Ku
in view of U.S. Patent Pub. No.: US11173918B1 to Fields.
Fields teaches “...8. The one or more processors of claim 1, wherein the one or more outputs represent one or more detected weather or surface conditions extracted based at least on the combined
extracted feature data, and the one or more operations of the ego-machine comprise controlling speed of the ego-machine based at least on the one or more detected weather or surface conditions”. (see col. 11, line 30-21 and col. 27, line 25 to col. 28, lines 10 where the autonomous vehicle can determine a high risk based on the weather conditions and the heavy rain ahead and the manual risk of driving also may be high and where the vehicle can then pull off the road until the heavy rain and weather passes and where the weather can cause failures of the operational features of the av)
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings of FIELDS with a reasonable expectation of success since FIELDS of STATE FARM™ teaches that a weather condition can be provided in the infrastructure component. This weather and traffic and provide items in the map together with the road to indicate a high risk area. The server 140 may access data stored in the database 146 when executing various functions and tasks associated with the evaluating feature effectiveness or assessing risk relating to an autonomous vehicle.
Claims 9 and 16 are rejected under 35 U.S.C. sec. 103 as being unpatentable as obvious in view of by Ku and in view of
NPL, Poudel, Pravin, Advanced Caching and Streaming for Large Scale Point Cloud Data Visualization on the Web, Utah State University, (https://digitalcommons.usu.edu/cgi/viewcontent.cgi?params=/context/etd2023/article/1002/&path_info=COMSetd2023Dec_Poudel_Pravin.pdf) (2023)(hereinafter “Poudel”).
In regard to claim 9 and 16, Poudel teaches “...9. The one or more processors of claim 1, wherein the processing circuitry is further to generate the combined extracted feature data based at least on projecting, from the second view into
the first view, the second extracted feature data representing a current time slice and one or more cached instances of the second extracted feature data representing one or more previous time slices”. (see page 20-26 where the device can store the slices of the data in a caching operation and this can be prefetched for an improved operation speed)
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings of POUDEL with a reasonable expectation of success since POUDEL teaches that slices of the data for the vehicle mapping can be pre-fetched to improve the speed and decrease a latency when fetching the map or data. See page 24-26.
In regard to claim 10 and 17, Hu discloses “...10. The one or more processors of claim 1, wherein the one or more processors are
comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual
reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more language models;
a system implementing one or more large language models (LLMs);
a system implementing one or more vision language models (VLMs);
a system implementing one or more multi-modal language models;
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system for performing one or more generative AI operations;
a system incorporating one or more virtual machines (VMs );
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources”. (see abstract and page 1 where this is used as a cloud computing resource for a vehicle collision avoidance point cloud stitching operation).
Claims 19-20 are rejected under 35 U.S.C. sec. 103 as being unpatentable as obvious in view of by Ku and in view NPL, Unknown, All about industries, What's happening around the digital twin
2024-05-03 Source: IDTA, Contact Software, VDMA, Nvidia | Translated by AI 9 min Reading Time , Hannover Messe 2024 (https://www.all-about-industries.com/hannover-messe-2024-ai-digital-twin-administration-shell-a-6ba2a80328cd9608685ec3eab2fd3f9c/)(hereinafter “All about”).
All about teaches “...19. The system of claim 18, wherein the simulation is generated, at least in part, using a three-dimensional (3D) content collaboration platform for 3D assets. (see omniverse API for self driving vehicles that uses a collaborative environment using an open usd and The five new Omniverse Cloud APIs, which can be used individually or together, include:
USD Render: creates fully ray-traced Nvidia RTX renderings from Open USD data
USD Write: allows users to modify and interact with Open USD data
USD Query: enables scene queries and interactive scenarios
USD Notify: tracks USD changes and provides updates
Omniverse Channel: connects users, tools, and worlds to enable cross-scene collaboration.)”.
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Hu with the teachings of ALL ABOUT Publication with a reasonable expectation of success since The ALL ABOUT publication teaches that an OPEN USD application used by PIXAR can be used as an API for a collaboration of the data via a query to render the autonomous vehicle data and 3d model.
All about teaches “...20. The system of claim 18, wherein the 3D content collaboration platform for 3D assets uses OpenUSD”. (see omniverse API for self driving vehicles that uses a collaborative environment using an open usd and The five new Omniverse Cloud APIs, which can be used individually or together, include:
USD Render: creates fully ray-traced Nvidia RTX renderings from Open USD data
USD Write: allows users to modify and interact with Open USD data
USD Query: enables scene queries and interactive scenarios
USD Notify: tracks USD changes and provides updates
Omniverse Channel: connects users, tools, and worlds to enable cross-scene collaboration.)”.
It would have been obvious for one of ordinary skill in the art before the effective filing date to combine the disclosure of Ku with the teachings of ALL ABOUT Publication with a reasonable expectation of success since The ALL ABOUT publication teaches that an OPEN USD application used by PIXAR can be used as an API for a collaboration of the data via a query to render the autonomous vehicle data and 3d model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEAN PAUL CASS whose telephone number is (571)270-1934. The examiner can normally be reached Monday to Friday 7 am to 7 pm; Saturday 10 am to 12 noon.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott A. Browne can be reached at 571-270-0151. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEAN PAUL CASS/Primary Examiner, Art Unit 3666
Prosecution Insights