Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1, 5-6, 9-10, 14, and 18-19 have been amended. 1-20 are pending.
Response to Arguments
The objection to the drawings have been withdrawn, as each number used in labeling is found in the specification.
The arguments made regarding the rejection of claims 1-20 have been found convincing due to the amendment, specifically the recitation of 3 separate neural networks, 1 for the background, and a second a third for the first and second objects respectively. New art has been provided for those limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-9, 14, and 17-20 rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (“Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering” Hereinafter “Yang”) in view of Mildenhall et al. (“NeRF: Representing Scenes as
Neural Radiance Fields for View Synthesis” Hereinafter “Mildenhall”) in further view of Kundu et al. (“Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation” Hereinafter “Kundu”).
Regarding claim 1, Yang teaches a system, comprising: a computer that includes a
processor and a memory (Page 4, section 3.4: “We jointly optimize the scene branch and object branch at the training stage”. If the branches can be trained, a processor has to be present to perform the training operations and other operations of the proposed neural radiance field for editable scene rendering algorithm, and instructions for performing the training operations would have to be stored on a memory. The memory and processor would be part of a computer since the system is performing computation operations), the memory including instructions executable by the processor to:
generate background pixels and object pixels, wherein the object pixels include first object pixels based on a first object and second object pixels based on a second object (Page 2, paragraph 3: “Firstly, we propose the first editable neural scene rendering system given a collection of posed images and 2D instance masks, which supports high-quality novel view rendering as well as object manipulation”. These 2D instance masks provide 2D segmentation for the objects in the posed images, separating the objects and the background (scene), this can be seen when everything not segmented as objects are included in the scene “The scene branch also renders the contents that are not labeled by the instance segmentation to provide a seamless whole scene rendering”(Page 2, paragraph 2). Fig. 1 also shows two separate objects being obtained, a first object comprising first object pixels (square furniture), and a second object comprising second object pixels (chair);
generate background pixel ray data based on the background pixels and first object pixel ray data based on the first object pixels and second object pixel ray data based on the second object pixels (Fig. 2, Fig. 3: “As shown in Fig. 2, our framework adopts two separate branches for scene rendering and object rendering. We take the advantages both from the voxelized representation [12] and the coordinate-based positional encoding [17], and propose a hybrid space embedding as network input. Practically, for each point x sampled along the camera ray, we apply positional encoding γ(·) [17] on both of the scene voxel feature fscn interpolated from 8 nearest vertices and space coordinate x to get the hybrid space embedding”. Since they apply position coding for each sample along a ray, they generate background and object pixel ray data. The object pixel ray data would be generated for each of the first (square furniture) and second (chair) objects).
input the background pixel ray data to a (Fig. 2: scene pixel ray data (fscn+x)) is input into a branch function (Fscn) to generate background neural radiance fields ((C, σ)scn));
input the first object pixel ray data to a (Fig. 2: object pixel ray data (fobj+x)) is input into a branch function (Fobj) to generate object neural radiance fields ((C, σ)obj). One of the NeRFs would be for the first object (square furniture)) and second object pixel ray data into a (Fig. 2: object pixel ray data (fobj+x)) is input into a branch function (Fobj) to generate object neural radiance fields ((C, σ)obj), One of the NeRFs would be for the second object (square furniture)); and
render an output image based on the background NeRFs, the first object NeRFs and the second object NeRFs (Fig. 1, page 4, section 3.5: “Thanks to the object-compositional NeRF, we can readily obtain radiance fields for each annotated object by simply switching the applied optimized object activation code, making it easy to realize the editable scene rendering”. Fig. 1 shows examples of the scenes they can render using the scene and both object NeRFs).
Yang does not expressly disclose using neural network to generate NeRF data.
However, Mildenhall teaches using neural networks to generate NeRF image data (Pages 1-2, last and first paragraphs: “To render this neural radiance field (NeRF) from a particular viewpoint we: 1) march camera rays through the scene to generate a sampled set of 3D points, 2) use those points and their corresponding 2D viewing directions as input to the neural network to produce an output set of colors and densities, and 3) use classical volume rendering techniques to accumulate those colors and densities into a 2D image”).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to substitute Yang’s function that generates NeRFs with Mildenhall’s neural network for generating NeRFs because such a modification is the result of simple substitution of one known element for another producing a predictable result. More specifically, with Mildenhall’s neural network for generating NeRFs teaches that it’s known to use neural networks for NeRF generation, and one of ordinary skill in the art would expect similar effects if substituted for Yang’s function that generates NeRFs.
The combination of Yang and Mildenhall does not expressly disclose using 3 separate neural networks for generating the NeRFs for the background, first, and second objects.
However, Kundu teaches using separate neural networks to generate NeRFs for the background, first, and second objects (Page 2, Col. 1: “We address these issues in our proposed Panoptic Neural Fields (PNF), an object-aware neural scene representation that explicitly decomposes a scene into a set of objects(things) and amorphous stuff background. Each object instance is represented by a separate MLP to evaluate the radiance field within the local domain of a potentially moving and semantically labeled 3D bounding box. The semantic radiance field of the stuff background is also represented by a MLP which includes an additional semantic head”. Each object has its own neural network (MLP) for generating NeRFs”. The objects and the background each have their own MLP for generating NeRFs, an MLP is a type of neural network).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the combination of Yang and Mildenhall’s NeRF generation system to include Kundu’s use of multiple neural networks for each of the objects and the background because such a modification is taught, suggested, or motivated by the art. More specifically, the motivation to modify the combination of Yang and Mildenhall to include Kundu is expressly provided by Kundu, stating that using object specific MLP’s leads to smaller and faster networks for the process (Abstract: “Each object MLPs are instance-specific and thus can be smaller and faster than previous object-aware approaches, while still leveraging category-specific priors incorporated via meta-learned initialization”). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify the combination of Yang and Mildenhall’s NeRF generation system to include Kundu’s use of multiple neural networks for each of the objects and the background with the motivation of more efficient processing due to smaller networks and increased speed. The person of ordinary skill in the art would have recognized the benefit of more efficient processing.
Regarding claim 4, the combination of Yang, Mildenhall, and Kundu teaches the system
of claim 1, in addition, Yang further teaches wherein the image segmentor is a
third neural network (Page 8, section 4.7: “This shed light on distilling fined-grained 3D
segmentation only from the knowledge of 2D segmentation networks by the proposed learning pipeline”. The proposed pipeline they use uses segmentation networks; segmentation networks are a neural network trained to segment. This neural network would be the “third”, since we substituted two into Yang from Mildenhall which can be seen in the claim 1 combination).
Regarding claim 5, the combination of Yang, Mildenhall, and Kundu teaches the system of claim 1, in addition, Kundu further teaches wherein the first neural network, the second neural network, and the fourth neural network include fully connected layers (Page 6, Col. 1: “Furthermore, for all MLPs, we use the softplus activation for the fully connected layer predicting
the density outputs [69].).
The rationale for this combination is similar to the rationale for the Kundu combination for the claim 1 rejection due to the similar methods of combinations (using neural networks for NeRF generation of single objects and the background) and similar benefits (improved processing efficiency).
Regarding claim 6, the combination of Yang, Mildenhall, and Kundu teaches the system
of claim 1, in addition, Yang further teaches wherein the background NeRFs, first object NeRFs and the second object NeRFs are five-dimensional (5D) radiance functions that include the radiance at multiple directions (θ,φ) at a three-dimensional (3D) point (x, y, z), wherein the radiance functions include color, intensity and opacity (Page 3, paragraph 1: “Since the framework is built upon NeRF, we refer to Mildenhall et al. [17] for the technical Background”. Mildenhall describes the radiance functions containing color and density information “We represent a continuous scene as a 5D vector-valued function whose input is a 3D location x = (x, y, z) and 2D viewing direction (θ, φ), and whose output is an emitted color c = (r, g, b) and volume density σ.”. So Yang teaches the claimed limitation, it simply relies on Mildenhall to describe the technical background since it’s built off their framework. The radiance functions would also contain intensity values for their current brightens levels. These would be true for the background, first object, and second object NeRFs).
Regarding claim 7, the combination of Yang, Mildenhall, and Kundu teaches the system
of claim 6, in addition, Yang further teaches wherein rendering the output image includes determining the 5D radiance functions along rays to a selected point of view (Page 3, paragraph 1: “Since the framework is built upon NeRF, we refer to Mildenhall et al. [17] for the technical Background”. Mildenhall describes the 5D radiance functions along rays “We synthesize images by sampling 5D coordinates (location and viewing direction) along camera rays” (Page 5, Fig. 2). So Yang teaches the claimed limitation, it simply relies on Mildenhall to describe the technical background since it’s built off their framework).
Regarding claim 8, the combination of Yang, Mildenhall, and Kundu teaches the system
of claim 7, in addition, Yang further teaches wherein the radiance functions include a location, a direction, an intensity and a color of a selected point (Page 3, paragraph 1: “Since the framework is built upon NeRF, we refer to Mildenhall et al. [17] for the technical Background”. Mildenhall describes the radiance functions containing a location, direction, and color information “We represent a continuous scene as a 5D vector-valued function whose input is a 3D location x = (x, y, z) and 2D viewing direction (θ, φ), and whose output is an emitted color c = (r, g, b) and volume density σ”. The NeRF generated would have a location, direction, and light intensity. So Yang teaches the claimed limitation, it simply relies on Mildenhall to describe the technical background since it’s built off their framework).
Regarding claim 9, the combination of Yang, Mildenhall, and Kundu teaches the system
of claim 1, in addition, Yang further teaches wherein rendering the output image includes selecting the first object to be included in the output image which includes selecting a first object location in x, y, and z location coordinates and direction in θ,φ rotational coordinates, selecting a first object colors, and selecting first object illumination (Page 3, paragraph 1: “Since the framework is built upon NeRF, we refer to Mildenhall et al. [17] for the technical Background”. Mildenhall describes the selecting these attributes for the output image “We represent a continuous scene as a 5D vector-valued function whose input is a 3D location x = (x, y, z) and 2D viewing direction (θ, φ), and whose output is an emitted color c = (r, g, b) and volume density σ”. The NeRF generated for the object in the output image would have a location, direction, color, and illumination. These would all be selected by virtue of the system selecting these attributes for the objects placement in the editable rendering scene. So Yang teaches the claimed limitation, it simply relies on Mildenhall to describe the technical background since it’s built off their framework. This is for the first object (square furniture), since this process is done for all objects NeRFs are generated for); and
selecting the second object to be included in the output image which includes selecting a second object location in x, y, and z location coordinates and direction in θ,φ rotational coordinates, selecting a second object colors, and selecting second object illumination (Page 3, paragraph 1: “Since the framework is built upon NeRF, we refer to Mildenhall et al. [17] for the technical Background”. Mildenhall describes the selecting these attributes for the output image “We represent a continuous scene as a 5D vector-valued function whose input is a 3D location x = (x, y, z) and 2D viewing direction (θ, φ), and whose output is an emitted color c = (r, g, b) and volume density σ”. The NeRF generated for the object in the output image would have a location, direction, color, and illumination. These would all be selected by virtue of the system selecting these attributes for the objects placement in the editable rendering scene. So Yang teaches the claimed limitation, it simply relies on Mildenhall to describe the technical background since it’s built off their framework. This is for the second object (chair), since this process is done for all objects NeRFs are generated for).
Regarding claim 14, the content of claim 14 is similar to the content of claim 1, therefore it is rejected for the same reasons of obviousness as claim 1.
Regarding claim 17, the content of claim 17 is similar to the content of claim 4, therefore it is rejected for the same reasons of obviousness as claim 4.
Regarding claim 18, the content of claim 18 is similar to the content of claim 5, therefore it is rejected for the same reasons of obviousness as claim 5.
Regarding claim 19, the content of claim 19 is similar to the content of claim 6, therefore it is rejected for the same reasons of obviousness as claim 6.
Regarding claim 20, the content of claim 20 is similar to the content of claim 7, therefore it is rejected for the same reasons of obviousness as claim 7.
Claims 2-3 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (“Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering” Hereinafter “Yang”) in view of Mildenhall et al. (“NeRF: Representing Scenes as
Neural Radiance Fields for View Synthesis” Hereinafter “Mildenhall”) in further view of Kundu et al. (“Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation” Hereinafter “Kundu”) in further view of Martin-Brualla et al. (“NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections” Hereinafter “Martin-Brualla”).
Regarding claim 2, the combination of Yang, Mildenhall, and Kundu teaches the system
of claim 1, in addition, Yang further teaches the system of claim 1, the instructions including further instructions to render the output image based on a selected point of view, (Fig. 1, page 4, section 3.5: “Thanks to the object-compositional NeRF, we can readily obtain radiance fields for each annotated object by simply switching the applied optimized object activation code, making it easy to realize the editable scene rendering”. Fig. 1 shows examples of the scenes they can render, these scenes were rendered based on selected point of views).
The combination of Yang, Mildenhall, and Kundu does not expressly disclose rendering the output image based on an illumination and a weather condition.
However, Martin-Brualla teaches rendering an image base on an illumination and a weather condition (Page 7211, Fig. 3, paragraph 2: “First, we model per-image appearance variations such as exposure, lighting, weather, and post-processing in a learned low-dimensional latent space”(Emphasis added). These variations are modeled using appearance embeddings which influence the rendered NeRF images).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Yang, Mildenhall, and Kundu’s image rendering to include Martin-Brualla’s use of weather because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Martin-Brualla’s use of weather for NeRF generation permits the ability to generate variational NeRFs with respect to weather conditions. This known benefit in Martin-Brualla is applicable to Yang, Mildenhall, and Kundu’s image rendering as they both share characteristics and capabilities, namely, they are directed to generating realistic synthetic NeRF data. If a scenario arose where the editable scene was located outside and affected by weather conditions, being able to generate images while considering weather like Martin-Brualla would be beneficial for obtaining realistic synthetic data. Therefore, it would have been recognized that the combination of Yang, Mildenhall, and Kundu’s image rendering to include Martin-Brualla’s use of weather for NeRF generation would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Martin-Brualla’s use of weather for NeRF generation in generating realistic synthetic NeRF data and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Regarding claim 3, the combination of Yang, Mildenhall, Kundu, and Martin-Brualla
teaches the system of claim 2, in addition, Yang further teaches wherein the point of view selected to render the output image includes a 3D viewing location in x, y, and z location coordinates and direction in θ,φ rotational coordinates (Page 3, paragraph 1: “Since the framework is built upon NeRF, we refer to Mildenhall et al. [17] for the technical Background”. Mildenhall describes the point of view to render the image containing the 3D coordinates and rotation coordinates “We represent a continuous scene as a 5D vector-valued function whose input is a 3D location x = (x, y, z) and 2D viewing direction (θ, φ), and whose output is an emitted color c = (r, g, b) and volume density σ.”. So Yang teaches the claimed limitation, it simply relies on Mildenhall to describe the technical background since it’s built off their framework).
Regarding claim 15, the content of claim 15 is similar to the content of claim 2, therefore it is rejected for the same reasons of obviousness as claim 2.
Regarding claim 16, the content of claim 16 is similar to the content of claim 3, therefore it is rejected for the same reasons of obviousness as claim 3.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (“Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering” Hereinafter “Yang”) in view of Mildenhall et al. (“NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis” Hereinafter “Mildenhall”) in further view of Kundu et al. (“Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation” Hereinafter “Kundu”) in further view of Toschi et al. (“ReLight My NeRF: A Dataset for Novel View Synthesis and Relighting of Real World Objects” Hereinafter “Toschi”).
Regarding claim 10, the combination of Yang, Mildenhall, and Kundu teaches the
system of claim 9, wherein rendering the output image includes rendering the first object illumination and the second object illumination (Fig. 1, page 4, section 3.5: “Thanks to the object-compositional NeRF, we can readily obtain radiance fields for each annotated object by simply switching the applied optimized object activation code, making it easy to realize the editable scene rendering”. Fig. 1 shows examples of the scenes they can render using the scene and object NeRFs. This would include both the first and second object NeRFs).
The combination of Yang, Mildenhall, and Kundu does not expressly disclose rendering the object illumination to match the background illumination.
However, Toschi teaches rendering the object illumination to better match the illumination of the scene at different points of the object (Pages 6-7, Section 5.2, V1-V5: “We avoid this cumbersome procedure by introducing a neural approximation of Li in the form of an MLP aimed at predicting a scalar value o = Ψvis(h(x), ζ(l)), which allows us to efficiently query the point-to-light visibility. The final pixel color is then obtained with c⋆ = o · c”. Li is the light intensity of the object, so the light intensity of the object at a location is dependent on where the light is in the background “Hence, our key idea is to modify the input to the Ψrgb injecting light position and let the model learn the interaction between the geometry and light”. So the object’s illumination will better match the backgrounds illumination at multiple points on the object. This would be true for the different locations of the first object since this is being done for objects in the scene).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the combination of Yang, Mildenhall, and Kundu’s rendered image to include Toschi’s ability to change the illumination of objects because such a modification is taught, suggested, or motivated by the art. More specifically, the motivation to modify the combination of Yang, Mildenhall, and Kundu to include Toschi is implicitly provided by Yang, stating that “Besides, to achieve more realistic scene editing, it is also promising to integrate the scene lighting model into the framework in the future work”. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify the combination of Yang, Mildenhall, and Kundu’s rendered image to include Toschi’s ability to change the illumination of objects with the motivation of more realistic scene editing. The person of ordinary skill in the art would have recognized the benefit of a more realistic scene.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (“Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering” Hereinafter “Yang”) in view of Mildenhall et al. (“NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis” Hereinafter “Mildenhall”) in further view of Kundu et al. (“Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation” Hereinafter “Kundu”) in further view of Ge et al. (“Neural-Sim: Learning to Generate Training Data with NeRF” Hereinafter “Ge”).
Regarding claim 11, the combination of Yang, Mildenhall, and Kundu teaches the
system of claim 1, in addition, Yang further teaches wherein the output images are output (Fig. 1, page 4, section 3.5: “Thanks to the object-compositional NeRF, we can readily obtain radiance fields for each annotated object by simply switching the applied optimized object activation code, making it easy to realize the editable scene rendering”. Fig. 1 shows examples of the scenes they can render using the scene and object NeRFs).
The combination of Yang, Mildenhall, and Kundu does not expressly disclose using
generated data to train a separate neural network.
However, Ge teaches using synthesized NeRF data for training object detection models (Page 480, section 3: “Furthermore, in recent times, NeRF and its variants (NeRFs) have been used to synthesize high-resolution photorealistic images for complex scenes [5,29,33,43,50]. This motivates us to explore NeRFs as potential sources of generating training data for computer vision models. We propose a technique to optimize rendering parameters of NeRFs to generate the optimal set of images for training object detection models”).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Yang, Mildenhall, and Kundu’s output NeRF data to include Ge’s ability to use NeRF data to train object detection models because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Ge’s ability to use NeRF data to train object detection models permits improved training for object detection models due to higher quality training data. This known benefit in Ge is applicable to the combination of Yang, Mildenhall, and Kundu’s output NeRF data as they both share characteristics and capabilities, namely, they are directed to generation of NeRF data in an attempt to create more realistic images. Therefore, it would have been recognized that modifying the combination Yang, Mildenhall, and Kundu’s output NeRF data to include Ge’s ability to use NeRF data to train object detection models would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Ge’s ability to use NeRF data to train object detection models in generation of NeRF data in an attempt to create more realistic images and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (“Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering” Hereinafter “Yang”) in view of Mildenhall et al. (“NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis” Hereinafter “Mildenhall”) in further view of Kundu et al. (“Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation” Hereinafter “Kundu”) in further view of Ge et al. (“Neural-Sim: Learning to Generate Training Data with NeRF” Hereinafter “Ge”) in further view of Bharadwaj et al. (US 20240273913 A1 “Bharadwaj”).
Regarding claim 12, the combination of Yang, Mildenhall, Kundu, and Ge teaches the
system of claim 11,
The combination of Yang, Mildenhall, and Ge does not expressly disclose wherein the
trained neural network is output to a third computing system in a vehicle.
However, Bharadwaj teaches using a trained object detection system in a vehicle ([0005]: ““In certain cases, the learnable decoders can comprise a series of neural network layers on the backend computing system that perform the scene reconstruction tasks, scene understanding tasks, object detection tasks, and other downstream tasks”. These decoders are learnable which means they can be trained).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to substitute Bharadwaj’s trained object detection model with The combination of Yang, Mildenhall, Kundu, and Ge’s trained object detection model because such a modification is the result of simple substitution of one known element for another producing a predictable result. More specifically, Yang, Mildenhall, Kundu, and Ge’s trained object detection model teaches that an object detection model that has been trained to detect objects, and one of ordinary skill in the art would expect similar effects if substituted for Bharadwaj’s trained object detection model.
Regarding claim 13, the combination of Yang, Mildenhall, Kundu, Ge, and Bharadwaj
teaches the third computing system of claim 12, in addition, Bharadwaj further teaches wherein memory included in the third computing system ([0024]: “These instructions can be stored in one or more memory resources of the computing device. A programmatically performed step may or may not be automatic”) includes instructions that are used to operate the vehicle by determining a vehicle path ([0045]: “The autonomous vehicle can include one or more autonomy maps 352 that the computing system of the autonomous vehicle dynamically references to perform localization, pose, object detection and classification, change detection, and motion planning operations in order to safely travel along a route autonomously”).
The rationale of this combination is the same as the combination of the previous claim due to the use of a trained model in the previous claim being used for operating a vehicle by determining a path.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Nguyen et al. (“Semantically-aware Neural Radiance Fields for Visual Scene Understanding”) teaches using NeRFs for visual scene understanding
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEFANO A DARDANO whose telephone number is (703)756-4543. The examiner can normally be reached Monday - Friday 11:00 - 7:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEFANO ANTHONY DARDANO/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698