DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20 are currently pending in the present application, with claims 1, 10, and 19 being independent.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 26 March 2025 and 30 May 2025 have been considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With respect to claim 1, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, it is not immediately clear as to:
What constitutes the one or more physical ways through the physical environment of the 3D model and how the metrics are indicative of boundaries of the physical ways. The disclosure in paragraph 92 sets forth “For example, the boundaries can correspond to one or more points, vectors, planes, 15 or volumes that define edges and surfaces topology of one or more roadway surfaces. For example, the boundaries can be obtained or generated from detection of surface topology of a physical environment via one or more sensors. For example, the boundaries can define a road and curb mesh”. However, taking the specification into account, it remains unclear as to what comprises a physical way in the 3D model and how the metrics are indicative of boundaries of the physical ways. If the boundaries are points or define a road/curb, then what would constitute the metric that is indicative of the boundary? What does indicative mean in this instance? How does a first surface correspond to multiple physical ways through the physical environment?
What is meant by the second environment metrics indicative of the one or physical ways and how are the geometric objects generated on the first surface according to the metrics? How are the metrics indicative of one or more physical ways? The disclosure sets forth “For example, the 2D objects can correspond to one or more points, vectors, planes, textures, images, or patterns that define two-dimensional objects on one or more roadway surfaces. For example, the 2D objects can be indicative of lane markings, directions markings, or any combination thereof. For example, the 2D objects can be obtained or generated from detection of surface imagery of a physical environment via one or more sensors. For example, the 2D objects can define lane paint decals and directional road markings”. However, taking the disclosure into account, it remains unclear as to how the geometric objects (e.g., lane markings) are generated according to metrics and how the metrics are indicative of one or more physical ways.
How one or more viewpoints oriented to capture corresponding portions of the 3D model are identified according to one or more viewpoint metrices indicative of cameras of a physical object configured to move along the one or more physical ways. The disclosure sets forth in paragraph 94 “For example, the cameras of a physical object can be simulated viewpoints in a virtual environment and capture rendered video objects. Viewpoints can include a front view, a right-front view, a right-rear view, a left-front view, and a left-rear view. Each of the viewpoints can correspond to 2D images or 2D video captured by a simulated ego vehicle traveling through a virtual environment”. However, taking the disclosure into account, it remains unclear as to how the one or more viewpoints oriented to capture corresponding portions of the 3D model are identified according to one or more viewpoint metrices indicative of cameras of a physical object configured to move along the one or more physical ways. What are the viewpoint metrics and how are they are indicative of cameras of a physical object? How do the cameras of a physical object relate to the camera feed of an ego object?
The examiner respectfully requests the applicant clarify the scope of the claimed invention.
Claims 10 and 19 recite substantially similar subject matter as to that of claim 1 and are accordingly also rejected using substantially similar rationale as to that set forth for claim 1.
Claims depending thereon do not cure the noted deficiencies and are accordingly rejected using substantially similar rationale as to that set forth for the claims from which they depend.
With respect to claim 2, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, it is not immediately clear as to how a portion of the first surface corresponding to a portion of a physical way among the physical ways is modified according to one or more third environment metrics. What constitutes the metrics, which are indicative of a condition of the physical way. What is meant by condition of the physical way? The disclosure in paragraph 48 sets forth “For example, the system can modify a virtual environment to modify particular roadway markings to change indications of traffic flow, change levels of wear on a roadway surface, levels of water on a roadway surface, roadway markings, or traffic signs, or change one more objects surrounding a roadway corresponding to weather, a biome or a level of urban density. The system can include dynamic objects that can be captured by one or more viewpoints. For example, dynamic objects can include dynamic traffic objects including traffic lights or gates. In another example, dynamic objects can include dynamic environment objects including trees, branches, traffic cones, or other obstructions in a roadway or that can affect traffic patterns or vehicles, pedestrians, or any combination thereof. Thus, this technical solution can provide at least a technical improvement to create numerous permutations of a real-world environment that would otherwise not be available and that could not be detected or drawn manually”. Based on the given examples, is physical way roads/pathways through the environment for a vehicle? However, it remains unclear as to how a portion of the first surface corresponding to a portion of a physical way among the physical ways is modified according to one or more third environment metrics and what constitutes the metrics, which are indicative of a condition of the physical way. How would traffic signs reflect a condition of the physical way? The examiner respectfully requests the applicant clarify the scope of the claimed limitation.
Claims 3 and 4 also recite third environment metrics and are rejected using similar rationale as to that set forth with respect to claim 2. In addition, with respect to claim 4, it is unclear as to how the one or more third environmental metrics are used to modify an opacity of at least a portion of a geometric 2D object with the 2D object being located at the portion of first surface. It is also unclear as to how the portion in the claim is both a portion of a 3D object (e.g., part of the 3D object) and a location of the first surface.
Claims 11, 12, 13, and 20 recite similar limitations as to that discussed with respect to claims 2-4 are accordingly rejected using substantially similar rationale as to that set forth with respect to claims 2-4.
With respect to claim 7, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, it is not immediately clear as to what is meant by generate, according to an environment heuristic indicative of an atmospheric condition, indicative of weather in the physical environment, one or more 3D objects that satisfy the environment heuristic at one or more corresponding positions in the 3D model. To which portion of the claim limitation does “indicative of weather in the physical environment belong? Is it referencing back to the atmospheric condition, the environmental heuristic or the generate step? As drafted, it would appear to go to the generate step.. In addition it is not immediately clear as to how the environment heuristic is indicative of an atmospheric condition and how the 3D objects generated satisfy the heuristic at corresponding positions in the model. Is the heuristic a conditional expression of some type. The examiner respectfully requests the applicant clarify the scope of the claimed limitation.
Claim 16 recites similar limitations as to that discussed with respect to claim 7 and is accordingly rejected using substantially similar rationale as to that set forth with respect to claim 7.
With respect to claim 8, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, it is not immediately clear as to a) how claim 8 fits within the scope of claim 1. Claim 1 generates the 3D model from the camera feed based on metrics, while in claim 8 the 3D model is corresponds to a region segment among the region segments, where the region model is segmented into different region segments. Also, what constitutes the block heuristic indicative of an amount of computational resources and how is the segmentation performed according? The examiner respectfully requests the applicant clarify the scope of the claimed limitation.
Claim 17 recites similar limitations as to that discussed with respect to claim 8 and is accordingly rejected using substantially similar rationale as to that set forth with respect to claim 8.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 5, 6,10-12, 14, 15, 19, and 20 is/are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Theverapperuma et al. (US PG Publication 2022/0024485).
Regarding claim 1, Theverapperuma teaches a system, comprising: a non-transitory memory and one or more processors (see for instance, paragraph 4) configured to: retrieve a camera feed of an ego object navigating within a physical environment (see for instance, paragraphs 9à-92 and fig. 4);
generate, according to one or more first environment metrics based on the camera feed, a three-dimensional (3D) model including a first surface corresponding to one or more physical ways through the physical environment, the one or more first environment metrics indicative of boundaries of the one or more physical ways (see for instance, paragraphs 83, 84, 87, 104, and 107. The DSES may combine all of the above listed outputs of the data processing subsystem to determine the boundaries of the road surface and which areas of the road surface are drivable, see for instance, paragraph 104. The output of the ground plane estimator may provide the DSES with a rough approximation of the 3D contours of the physical environment, including the approximate shape and boundaries of objects and surfaces, see for instance, paragraph 107. A 3D model can be generated from the camera feed, see for instance, paragraphs 83, 84, 87, 104, and 107);
generate, according to one or more second environment metrics, one or more geometric two-dimensional (2D) objects on the first surface, the second environment metrics indicative of the one or more physical ways (see for instance, paragraphs 83-87, 103, 111, and figs. 3 and 5. For instance, surface identification subsystem can be configured to detect objects that render an otherwise drivable surface unsuitable for driving on (e.g., buildings, other vehicles, cone markers, poles, pools of liquid, cracks, and the like), see for instance, paragraph 85. In fig. 5 boxes are shown around the border of a pole 502, a pole 504, a traffic sign 506, and a traffic sign 508, see for instance, paragraph 111);
identify, according to one or more viewpoint metrics indicative of cameras of a physical object configured to move along the one or more physical ways, one or more viewpoints oriented to capture corresponding portions of the 3D model (see for instance, paragraphs 53, 70, 73, 92, 108-113 and fig. 4. Localization subsystem 202 is also configured to
perform visual odometry, which involves determining the orientation and position of the ego vehicle based upon sensor data, such as by analyzing images captured by one or more cameras, see for instance, paragraph 53. To combine the outputs of the data processing subsystem 420, the DSES 430 may perform geometric transformations or calculations that map data from different sensors onto each other. For instance, the DSES 430 may generate the output representation 450 taking into account differences between the orientation and positions of the LIDAR sensor 460 and the cameras 402, 404, see for instance, paragraph 108. “The sensor data used for generating any particular set of extracted features can be obtained using one or more temporal views and/or one or more spatial views. For instance, as indicated above, there can be multiple pairs of cameras or LIDAR sensors in different positions. A set of extracted features from which an output representation of a physical environment is generated can be the result of processing sensor data obtained over a period of time, e.g., sensor data collected over several image frames or over a course of vehicle movement”, see for instance, paragraph 92);
render, from the one or more corresponding portions of the 3D model of the physical environment, one or more simulated environment 2D images each corresponding to respective ones of the viewpoints (see for instance, paragraphs 93, 107, 108, 110, 112, 146, and figs. 4-6. The output representation 450 can be a "true" 3D representation or quasi-3D representation indicating whether there are any drivable surfaces present in the environment, see for instance, paragraph 93 Fig. 5 illustrates an example of the results of object detection performed on a camera image, see for instance, paragraph 110); and
training, by the processor, an artificial intelligence model in accordance with the camera feed and the one or more simulated environment 2D images (see for instance, paragraphs 59, 133, 135and 137).
Regarding claim 2, Theverapperuma teaches the system of claim 1 and further teaches the processors to: modify, according to one or more third environment metrics, a portion of the first surface corresponding to a portion of a physical way among the physical ways, the one or more third environment metrics indicative of a condition of the physical way (see for instance, paragraphs 44, 57, 65, 103, and figs 4-6).
Regarding claim 3, Theverapperuma teaches the system of claim 1 and further teaches the processors to: modify, according to one or more third environment metrics, a topology of the portion of the first surface (see for instance, paragraphs 54, and 103, and fig. 6).
Regarding claim 5, Theverapperuma teaches the system of claim 1 and further teaches the processors to: generate, according to a localization heuristic indicative of a type of physical objects in the physical environment, one or more 3D objects that satisfy the localization heuristic at one or more corresponding positions in a second surface of the 3D model excluding the first surface (see for instance, paragraphs 57, and 83-88).
Regarding claim 6, Theverapperuma teaches the system of claim 5 and further teaches the type of physical objects corresponding to at least one of a type of geography, a type of climate, or a type of architecture (see for instance, paragraph 85, 104, and 127).
Regarding claims 10 and 19, claim 10 is a method claim and claim 19 is a non-transitory computer readable medium claim recites substantially similar subject matter as to that set forth in claim 1. Claims 10 and 19 are rejected using substantially similar rationale as to that set forth with respect to claim 1. In addition Theverapperuma teaches a method and a non-transitory computer readable medium including one or more instructions stored thereon and executable by a processor to perform actions (paragraphs 4 and 133).
Regarding claims 11 and 20, claim 11 is a method claim and claim 20 is a non-transitory computer readable medium claim recites substantially similar subject matter as to that set forth in claim 2. Claims 11 and 20are rejected using substantially similar rationale as to that set forth with respect to claim 2.
Regarding claim 12, claim 12 recites substantially similar limitations as to that set forth in claim 3 and is rejected using substantially similar rationale as to that which was set forth in the rejection of claim 3.
Regarding claim 14, claim 14 recites substantially similar limitations as to that set forth in claim 5 and is rejected using substantially similar rationale as to that which was set forth in the rejection of claim 5.
Regarding claim 15, claim 15 recites substantially similar limitations as to that set forth in claim 6 and is rejected using substantially similar rationale as to that which was set forth in the rejection of claim 6.
Allowable Subject Matter
Since no prior art is currently being applied and based on their current scope, claims 4, 7-9, 13, 16-18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US PG Publication 2021038616 to Rong et al. teaches generating a 3D environment and a target viewpoint, see for instance, paragraph 45-47 and 113.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J COBB whose telephone number is (571)270-3875. The examiner can normally be reached Monday - Friday, 11am - 7pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J COBB/Primary Examiner, Art Unit 2615
Prosecution Insights