DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Terminal Disclaimer
The terminal disclaimer filed on 4/14/2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of co-pending Application 18670373 has been reviewed and is accepted. The terminal disclaimer has been recorded.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-18 is/are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Benedek et al1 (“Benedek”).
Regarding claim 1, Benedek teaches one or more processors comprising: processing circuitry to (see Benedek, paragraphs 0042-0044 teaching “a system for generating a three-dimensional model, said system comprising a scanning device adapted for generating a point set corresponding to a scene comprising at least one object shape, a point set dividing module adapted for dividing the point set corresponding to the scene into a foreground point set corresponding to the foreground of the scene, and comprising a subset corresponding to the at least one object shape of the point set corresponding to the scene, and into a background point set corresponding to the background of the scene, an object shape subset dividing module adapted for dividing the foreground point into each of at least one object shape subset corresponding to the at least one object shape, a background modelling module adapted for generating a background three-dimensional model on the basis of the background point set, an optical model-generating module adapted for generating from the optical recordings a three-dimensional model of at least one substituting object shape assignable to each of the at least one object shape, and a model combining module adapted for generating a combined three-dimensional model on the basis of the background three-dimensional model and the three-dimensional model of at least one substituting object shape substituting each of the at least one object shape subset, respectively” where such modules of such a system performing processing on the data as below are processors comprising processing circuitry to carry out the complex operations only possible using a processor with processing circuitry and where for example paragraph 0150 teaches “combination of various formats of data coming from different sources is performed by software developed for this purpose, by which the data are brought to a common format” and for example “Displaying is preferably carried out by a programme based on a VTK Visualisation Kit” such that here since the software has been developed to perform the functions, the modules of the system are processors with processing circuitry to perform the techniques as explained below):
compute, based at least on sensor data generated using one or more sensors of an ego-machine in an environment (see Benedek, paragraphs 0082-0086 teaching “generating a three-dimensional model, a so-called combined three-dimensional model. The combined three-dimensional model comprises parts reconstructed on the basis of a point set corresponding to a scene generated by a scanning device, and parts modelled by three-dimensional models generated on the basis of optical recordings” where “a point set corresponding to a scene is generated by means of a scanning device where the scene comprises at least one object shape” and “a combined three-dimensional model is generated on the basis of the background three-dimensional model and the three-dimensional model of at least one substituting object shape substituting each of the at least one object shape subset, respectively” such that here the computing is based on the point set generated by a scanning device sensor of an ego-machine in an environment as the camera may be for example attached to a moving ego-machine where “LIDAR device converts the distance data into a point set corresponding to a scene in a known manner, and the LIDAR device is located in the centre of the point set” such that this device functions egocentrically with respect to the scene as the perspective is from the egocentric perspective of the device capturing the perspective of the scene and not outside cameras imaging the device and scene ), and based at least on one or more three-dimensional (3D) representations of one or more detected dynamic objects in the environment, a 3D surface topology of the environment (note that a “detected dynamic object” comprises either an object that has dynamic characteristics or that is capable of being dynamic such that it is not required that an object necessarily is moving dynamically in the environment when detected; further note that “(3D) representations” of such objects in the environment comprises any type of 3D representation such as their natural 3D representation as existing in the real world or could be some other computed representation of such objects where any form of data representing the 3D objects would be a 3D representation of the objects; see Benedek, paragraphs 0082-0086 teaching “generating a three-dimensional model, a so-called combined three-dimensional model. The combined three-dimensional model comprises parts reconstructed on the basis of a point set corresponding to a scene generated by a scanning device, and parts modelled by three-dimensional models generated on the basis of optical recordings” and “a point set corresponding to a scene is generated by means of a scanning device where the scene comprises at least one object shape. Then, in an operational step S120, the point set corresponding to a scene is divided into a foreground point set comprising a subset of at least one object shape corresponding to the foreground of the scene, and into a background point set corresponding to a scene background. In an operational step S130, from the foreground point set, at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set. In an operational step S140, the three-dimensional model of the scene background is generated on the basis of the background point set. Then, in an operational step S150, a three-dimensional model of at least one substituting object shape assignable to each of the at least one object shape, respectively, is generated from optical recordings. And finally, in an operational step S160, a combined three-dimensional model is generated on the basis of the background three-dimensional model and the three-dimensional model of at least one substituting object shape substituting each of the at least one object shape subset, respectively” such that here at least one object is represented in the 3D dimensions in the environment and its 3D shape is determined such that these point cloud points of the foreground dynamic objects and/or the “at least one object shape subset corresponding to each of the at least one object shape…separated from the foreground point set” are 3D representations of one or more detected dynamic objects in the environment which is used along with the 3D shape of the background environment such that the 3D model generated is a computed 3D surface topology of the environment and note for example as in paragraph 0012 “object of the invention is to provide a method and a system by which the three-dimensional model of a scene can be generated substantially, that is almost, in real time in a way that some object shapes, for example, moving people, cars or other significant shapes, significant from the aspect of the scene and located in the foreground of the scene, and the three-dimensional model of some static objects are processed on the basis of optical recordings” such that here the objects comprise dynamic objects; see also paragraphs 0016-0019 teaching “a topographic model is generated by modelling the topographic features of the scene, after the division of the point set corresponding to the scene into a foreground point set and a background point set” and “topographic model is made on the basis of an approximating plane fitted onto the topographic features of the scene” and “topographic model is made on the basis of a parameterised surface which is fitted onto the topographic features of the scene and follows the unevenness of the topographic features” where this topographic 3D surface is based on sensing the environment and dynamic objects in order to determine the background topology); and
generate a visualization of a representation of the sensor data projected onto note that the claim requires “a visualization of a representation of the sensor data” such that what is “projected onto the one or more 3D representations in the 3D surface topology” need not be the sensor data itself, but rather can be any “representation of the sensor data” such that for example if a different type of data besides the sensor data functions to represent the sensor data or describe the sensor data in some way then this would be a representation of the sensor data; see Benedek, paragraphs 0082-0086 as explained above where the purpose as explained above is “generating a three-dimensional model” which is displayed and visualized and “combined three-dimensional model comprises parts reconstructed on the basis of a point set corresponding to a scene generated by a scanning device, and parts modelled by three-dimensional models generated on the basis of optical recordings” and “point set corresponding to a scene is divided into a foreground point set comprising a subset of at least one object shape corresponding to the foreground of the scene” and “from the foreground point set, at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set” such that this combined 3D model combines the 3D topology determined with a representation of the sensor data which is projected onto the one or more 3D representations of the one or more detected dynamic objects in the 3D surface topology, where such representation of the sensor data projected onto the one or more 3D representations of the dynamic object in the 3D surface topology is taught as in paragraphs 0030-0033 disclosing “point sets preferably made by a moving car may also be modelled, and the three-dimensional models of the substituting object shapes may be fitted into the registered point set to substitute the appropriate object shape subsets” and “a dynamic three-dimensional model of the at least one substituting object shape is generated on the basis of the optical recordings, and the combined three-dimensional model is generated on the basis of the background three-dimensional model and the dynamic three-dimensional model of at least one substituting object shape substituting each members of the series of object shape subsets corresponding to the at least one object shape” and as in paragraph 0039 “the three-dimensional model of at least one substituting object shape and/or the three-dimensional model of the scene background are provided with a texture. The parts of the combined three-dimensional model are preferably provided with a texture” where paragraphs 0139-0150 further explain generating of visualization of a representation of the sensor data where the “subsets corresponding to the object shapes are substituted by such three-dimensional models” by “substituting object shapes that may be assigned to the object shapes” where these are “detailed and preferable textured, static or dynamic three-dimensional models” and “model shown in the figure is generated by a method according to the invention in a way that the three-dimensional model of the substituting object shape is textured” and “the textured three-dimensional model is placed in front of a reconstructed background” and as in paragraphs 0167-0168 “for vehicles the three-dimensional model of their substituting object shapes can be made, by which the appropriate vehicle associated object shape subsets may be substituted. In such a way, a combined three-dimensional model of the scene can be generated, in which object shape subsets corresponding to vehicles and/or object shape subsets corresponding to human figures are substituted by substituting three-dimensional models” such that the “substituting object shape” that is generated and projected to the topographic model when generating the combined 3D model is a visualization of a representation of the sensor data given that its purpose is to represent the detected 3D dynamic object in the sensor data, such that when substituting the detailed 3D model in place of the detected 3D dynamic object representation, this constitutes that representation of the sensor data projected onto the 3D representation in the 3D surface topology such that as in paragraphs 0147-0149, “last step of the method according to the invention is the integration of the point set corresponding to the scene and the three-dimensional models obtained on the basis of the optical recordings, i.e. generating a combined three-dimensional model by means of the three-dimensional model of the background and the three-dimensional model of at least one substituting object shape. The combined three-dimensional model obtained by the method of the invention is preferably displayed” and “the combined three-dimensional model is generated on the basis of the background three-dimensional model and the dynamic three-dimensional model multiplied to a length corresponding to the movement along a trajectory corresponding to the object shape of at least one substituting object shape substituting each of the series of object shape subsets corresponding to at least one object shape, respectively. Therefore in this embodiment, the moving dynamic three-dimensional models are placed into the reconstructed background in a way that the dynamic three-dimensional model tracks the trajectory obtained on the basis of the point set corresponding to the scene. It can be exemplary assumed that the object shape proceeds along the trajectory in one direction. The orientation of the object shape placed on the trajectory in the course of moving along the trajectory is determined subject to the shape of the trajectory, i.e. in each moment of time the direction of the tangent of the trajectory defines the orientation of the three-dimensional model substituting the object shape” where for example “a dynamic three-dimensional model of the at least one substituting object shape is generated, and the combined three-dimensional model is generated on the basis of the background three-dimensional model and the dynamic three-dimensional model multiplied to a length corresponding to the movement along a trajectory corresponding to the object shape of at least one substituting object shape substituting each of the series of object shape subsets corresponding to at least one object shape, respectively. Therefore in this embodiment, the moving dynamic three-dimensional models are placed into the reconstructed background in a way that the dynamic three-dimensional model tracks the trajectory obtained on the basis of the point set corresponding to the scene. It can be exemplary assumed that the object shape proceeds along the trajectory in one direction. The orientation of the object shape placed on the trajectory in the course of moving along the trajectory is determined subject to the shape of the trajectory, i.e. in each moment of time the direction of the tangent of the trajectory defines the orientation of the three-dimensional model substituting the object shape” such that here this placing of the 3D models into the reconstructed background “in a way that the dynamic three-dimensional models tracks the trajectory obtained on the basis of the point set corresponding to the scene” is the generation of a visualization of a representation of the sensor data which projected through this placement onto the 3D surface representations corresponding to the identified dynamic 3D objects and is projected onto the location of the 3D representation in the 3D surface topology to generate the combined model).
Regarding claim 2, Benedek teaches all that is required as applied to claim 1 above and further teaches the processing circuitry further to mask the one or more detected dynamic objects from the sensor data during a first pass of generating the visualization (note that to mask such objects is interpreted as any manner of setting such data apart, logically, functionally, or otherwise, from other data where the selection of such data from the full set may be considered a mask of such data when used and for example the masked object may be processed or the data in which the mask objects have been masked may be processed; see Benedek, paragraphs 0082-0086 teaching “in an operational step S120, the point set corresponding to a scene is divided into a foreground point set comprising a subset of at least one object shape corresponding to the foreground of the scene, and into a background point set corresponding to a scene background. In an operational step S130, from the foreground point set, at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set. In an operational step S140, the three-dimensional model of the scene background is generated on the basis of the background point set” such that here this dividing out of the foreground points of the scene functions to mask the detected dynamic objects which is during a first pass of generating the visualization where a pass is some attempt at processing relating to the method or is a passing of such data used in generating the visualization, and for example also “at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set” such that this also functions to mask the object through such separation giving the ability to process that masked object, such as for generating a visualization of the masked object or background).
Regarding claim 3, Benedek teaches all that is required as applied to claim 1 above and further teaches the processing circuitry further to: compute a first 3D surface topology of the environment representing a static portion of the environment (see Benedek, paragraphs 0082-0086 teaching “a point set corresponding to a scene is generated by means of a scanning device where the scene comprises at least one object shape. Then, in an operational step S120, the point set corresponding to a scene is divided into a foreground point set comprising a subset of at least one object shape corresponding to the foreground of the scene, and into a background point set corresponding to a scene background. In an operational step S130, from the foreground point set, at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set. In an operational step S140, the three-dimensional model of the scene background is generated on the basis of the background point set. Then, in an operational step S150, a three-dimensional model of at least one substituting object shape assignable to each of the at least one object shape, respectively, is generated from optical recordings. And finally, in an operational step S160, a combined three-dimensional model is generated on the basis of the background three-dimensional model and the three-dimensional model of at least one substituting object shape substituting each of the at least one object shape subset, respectively” such that a first 3D surface topology of the environment representing a static portion of the environment corresponds to the background point set which is modeled as a background three-dimensional model where such model represents the 3D surface topology of the environment; see also paragraphs 0016-0019 teaching a “a topographic model is generated by modelling the topographic features of the scene, after the division of the point set corresponding to the scene into a foreground point set and a background point set” and “topographic model is made on the basis of an approximating plane fitted onto the topographic features of the scene” and “topographic model is made on the basis of a parameterised surface which is fitted onto the topographic features of the scene and follows the unevenness of the topographic features” such that the background is a static portion of the environment and its 3D surface topology is computed where this is part of the background 3D modeling as in paragraphs 0082-0086 explained above where “the three-dimensional model of the scene background is generated on the basis of the background point set”); and update the first 3D surface topology based at least on inserting the one or more 3D representations of the one or more detected dynamic objects into the first 3D surface topology (see Benedek, paragraphs 0016-0019 as explained above where “a projected foreground point set is generated by projecting the foreground point set to the topographic model, at least one projected object shape subset corresponding to each of the at least one object shape, respectively, is generated by dividing the projected foreground point set by means of shape filtering and/or dimensional fitting, and the at least one object shape subset is determined on the basis of the at least one projected object shape subset. In the present embodiment of the method, the object shape subsets are generated on the basis of projection to a topographic model” such that this projecting is such inserting of the 3D representations of the dynamic objects into the first 3D surface topology such that this creates the combined 3D representation as in paragraphs 0082-0086 teaching “in an operational step S150, a three-dimensional model of at least one substituting object shape assignable to each of the at least one object shape, respectively, is generated from optical recordings. And finally, in an operational step S160, a combined three-dimensional model is generated on the basis of the background three-dimensional model and the three-dimensional model of at least one substituting object shape substituting each of the at least one object shape subset, respectively” such that this inserts the 3D representation of the dynamic object into the first 3D surface topology).
Regarding claim 4, Benedek teaches all that is required as applied to claim 1 above and further teaches wherein the one or more detected dynamic objects includes at least one rigid object of one or more classes of rigid objects (note that this requires at least one rigid object of one class of rigid objects where such classes need not be generated nor objects actually classified so long as the object could be considered a rigid object which would put it in a class of rigid objects, and note that a rigid object is considered an object that is rigid in some way where rigid is taken to mean something is unable to bend or be forced out of shape or is not flexible, without action by some outside force where for example a vehicle or building or tree or other unmoveable objects are examples of rigid objects; see Benedek, paragraphs 0152-0156 teaching vehicles detected as dynamic objects which are rigid objects of a class of rigid objects where “the invention described below is covered in association with such a scene, where the object shapes in the foreground may be human figures and also vehicles. These foreground object shapes may be stationary and also mobile similarly to the discussion above”), wherein the processing circuitry is further to generate one or more 3D representations of the at least one rigid object based at least on warping one or more detected depth values corresponding to the at least one rigid object using one or more detected trajectories corresponding to the at least one rigid object (note that “warping” is considered any spatial transformation, distortion, shifting, re-projection, or the like of data elements from one coordinate state or coordinate system to another state or system, where such could comprise geometric transformations applied to data points to align them, correct for motion, or map them onto a different surface or reference frame and warping a depth value in the context of the claims could involve capturing depth data points for tracked objects at different times and locations and mathematically transforming and thus warping such points to align them into a single, coherent 3D representation; see Benedek, paragraphs 0152-0156 as explained above teaching vehicles detected as dynamic objects which are rigid objects of a class of rigid objects and paragraphs 0163-0178 teaching that to generate the 3D representation of such objects this involves a warping of detected depth values through registering such depth values of tracked objects over time in the “point set registration” where “the time series of the point set corresponding to a scene is generated by a scanning device, which has been in different places when recording the various point sets, i.e. the time series of the point set corresponding to a scene is generated in a way that the scanning device is moved. Such a situation can be conceived for example if the scanning device is fitted on top of a vehicle, and during the movement of the vehicle the members of the time series of the point set are recorded on an ongoing basis. In such cases, at least in one part of the members of the time series of the point set corresponding to a scene, a so-called point set registration is carried out, i.e. the point sets coming from various points are transformed to a common co-ordinate system, and through this a registered point set is established” and “After projection to the common co-ordinate system, a dense point set is obtained about the scene” and for example “by the registration of the point sets, the quality of information obtainable from each object shape has been substantially improved, and the registered point set subset corresponding to the static type of object shapes is much denser than the object shape subsets shown on the left hand side. This means that to the static object shapes, in such a way, a very high resolution combined object shape subset may be assigned in the point set” such that here as the trajectory of the points are detected as belonging to the same object these points may be warped in order to register them such that a 3D representation relating to the object and where it appears in the warped depth data can be determined).
Regarding claim 5, Benedek teaches all that is required as applied to claim 1 above and further teaches the processing circuitry further to fuse, into the 3D surface topology, at least a first 3D representation of at least a first detected dynamic object of the one or more detected dynamic objects generated based at least on (see Benedek, paragraph 0082 teaching “In an operational step S130, from the foreground point set, at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set. In an operational step S140, the three-dimensional model of the scene background is generated on the basis of the background point set. Then, in an operational step S150, a three-dimensional model of at least one substituting object shape assignable to each of the at least one object shape, respectively, is generated from optical recordings. And finally, in an operational step S160, a combined three-dimensional model is generated on the basis of the background three-dimensional model and the three-dimensional model of at least one substituting object shape substituting each of the at least one object shape subset, respectively” and see paragraph 0148 teaching “last step of the method according to the invention is the integration of the point set corresponding to the scene and the three-dimensional models obtained on the basis of the optical recordings, i.e. generating a combined three-dimensional model by means of the three-dimensional model of the background and the three-dimensional model of at least one substituting object shape. The combined three-dimensional model obtained by the method of the invention is preferably displayed” where the generating of the combined 3D model fuses into the surface topology a 3D version of the object that is substituted and fused into the representation where detected dynamic objects are treated as explained further below): tracking a trajectory of the first detected dynamic object (see Benedek, paragraphs 0103-0110 teaching “to track the object shapes, i.e. to determine their trajectories will be disclosed” where “It is attempted with the STA module to fit each actually detected object shape candidate to the considered trajectories on the basis of the positions of the weighted centres of the projected object shapes” and “On the basis of these two data, i.e. in accordance with the earlier movement of an object shape, the estimate given for the new position and according to the actually measured position, a distance matrix D is written” and “on the basis of the location points, consequently at least one trajectory is determined by performing the following steps cyclically. A next location point in sequence is assigned for the at least one object shape, the assigned location point is corrected after examination with a Kalman-filter, and finalising the corrected location point, and a proposal is made by means of a Kalman-filter, for the next location point in sequence, for the at least one object shape” such that “the input of the procedural steps described above is the time series of the point set corresponding to the scene, where each point is marked by a foreground or a background tag, i.e. each level of the point set corresponding to a scene time series is divided into a foreground point set and a background point set. As the output of this step, the object shape subsets of the foreground point set are obtained, and the object shape subsets corresponding to the same object shape preferably maintain the very same tag in the complete time series of the point set corresponding to a scene, i.e. in the time series of the point set corresponding to a scene, the object shape subsets corresponding to each object shape can be tracked” and “for each element of the time series of the point set corresponding to a scene, the projected object shape subsets corresponding to the object shapes can be obtained, i.e. again a time series of the projected object shape subsets are generated. On the basis of the at least one projected object shape subset, the location points series of at least one object shape on the topographic model is determined, and on the basis of the series of location points, at least one trajectory is defined for each of the at least one object shape. In the embodiments where the trajectory is determined, as the location point of at least one object shape—in the case of human figures principally the locations of the feet of the object shape on the ground plane—preferably the weighted centre of the projected object shape subset corresponding to the given object shape is selected” such that here this tracks a trajectory of the first and all relevant detected dynamic objects); identifying one or more detected depth values representing the first detected object in a previous time slice, and warping the one or more detected depth values using the trajectory (see Benedek, paragraphs 0103-0110 as explained above where the detected depth values correspond to the 3D point sets of the objects above and depth values from previous time slices are identified as where “on the basis of the location points, consequently at least one trajectory is determined by performing the following steps cyclically. A next location point in sequence is assigned for the at least one object shape, the assigned location point is corrected after examination with a Kalman-filter, and finalising the corrected location point, and a proposal is made by means of a Kalman-filter, for the next location point in sequence, for the at least one object shape” and as in paragraphs 0163-0178 teaching that to generate the 3D representation of such objects this involves a warping of detected depth values through registering such depth values of tracked objects over time in the “point set registration” where “the time series of the point set corresponding to a scene is generated by a scanning device, which has been in different places when recording the various point sets, i.e. the time series of the point set corresponding to a scene is generated in a way that the scanning device is moved. Such a situation can be conceived for example if the scanning device is fitted on top of a vehicle, and during the movement of the vehicle the members of the time series of the point set are recorded on an ongoing basis. In such cases, at least in one part of the members of the time series of the point set corresponding to a scene, a so-called point set registration is carried out, i.e. the point sets coming from various points are transformed to a common co-ordinate system, and through this a registered point set is established” and “After projection to the common co-ordinate system, a dense point set is obtained about the scene” and for example “by the registration of the point sets, the quality of information obtainable from each object shape has been substantially improved, and the registered point set subset corresponding to the static type of object shapes is much denser than the object shape subsets shown on the left hand side. This means that to the static object shapes, in such a way, a very high resolution combined object shape subset may be assigned in the point set” such that here as the trajectory of the points are detected as belonging to the same object these points may be warped in order to register them such that a 3D representation relating to the object and where it appears in the warped depth data can be determined).
Regarding claim 6, Benedek teaches all that is required as applied to claim 1 above and further teaches wherein the one or more detected dynamic objects includes at least one non-rigid object of one or more classes of non-rigid objects (see Benedek, paragraphs 0010-0012 teaching that detected dynamic objects can be non-rigid objects belonging to one or more classes of non-rigid objects such as “stationary or moving people” where people are considered non-rigid objects), and wherein the processing circuitry is further to generate one or more 3D representations of the at least one non-rigid object based at least on inserting, for the at least one non-rigid object, a 3D representation of a two-dimensional (2D) surface at a location in the 3D surface topology corresponding to a detected location of the at least one non-rigid object in the environment (note that a “3D representation of a two-dimensional (2D) surface” is extremely broad and would encompass the display of any 3D data on a 2D display surface as ultimately each 3D point is represented by a 2D pixel corresponding to the visible surface of the 3D point from whatever camera is responsible for capturing the scene, and for example would correspond to polygon facets of a 3D mesh model which are 2D triangle surfaces of a 3D representation for example and further note this would encompass a texture map applied to a surface representing a 3D object as well, and for example would also cover more explicitly defined or simple 2D shapes in a 3D environment such as an explicitly defined 2D billboard type object placed into a 3D space; see Benedek, paragraph 0152-0156 teaching “dynamic three-dimensional models may be multiplied not only in space, but also in time” and “a walking person can be displayed” and “the object shapes in the foreground may be human figures and also vehicles. These foreground object shapes may be stationary and also mobile similarly to the discussion above” and as in paragraphs 0165-0174 a 3D representation of the non-rigid object may be substituted for the point sets corresponding to the non-rigid object where “a combined three-dimensional model of the scene can be generated, in which object shape subsets corresponding to vehicles and/or object shape subsets corresponding to human figures are substituted by substituting three-dimensional models made on the basis of optical recordings” and “on the basis of the time stamps, at least one, stationary shape associated, static combined object shape subset and/or at least one, moving shape associated, dynamic object shape subset is separated in the registered point set” and “On the different time levels, object shape subsets 100 a, 100 b, 100 c are associated one by one with the given object shape” and “from the aspect of substituting the three-dimensional model of the substituting object shape, it does not have a significance how the trajectory serving as a basis for the fitting of the three-dimensional model was obtained, and it is only to be determined how the three-dimensional model should be fitted to the trajectory (with its point of contact on the ground or with the centre of its volume)” such that this 3D representation is of a 2D surface at a location in the 3D surface topology corresponding to a detected location of the object in the environment as the insertion or substituting is done at the locations of the detected dynamic object being tracked, and note that as in paragraphs 0141-0148 it is evidenced that the 3D representation may comprise a 3D representation of a 2D surface as the “triangular lattice” and “textured” triangular lattices of the 3D objects that are inserted are 3D representations of such 2D triangular textured lattices as appearing in the 3D space as generated).
Regarding claim 7, Benedek teaches all that is required as applied to claim 1 above and further teaches the processing circuitry further to fuse into the 3D surface topology at least a first 3D representation of a flat surface at a location corresponding to a detected centroid of a corresponding one of the one or more detected dynamic objects (note that a “3D surface representation of a flat surface” corresponds to a 3D surface representation of a 2D surface as a flat surface may be considered a 2D surface, though a flat surface may also be described in three-dimensions as well if for example the surface points all belong to only two of the described dimensions of the coordinate system, and thus as explained above this would encompass the display of any 3D data on a 2D display surface as ultimately each 3D point is represented by a 2D pixel corresponding to the visible surface of the 3D point from whatever camera is responsible for capturing the scene, and for example would correspond to polygon facets of a 3D mesh model which are 2D triangle surfaces of a 3D representation for example and further note this would encompass a texture map applied to a surface representing a 3D object as well, and for example would also cover more explicitly defined or simple 2D shapes in a 3D environment such as an explicitly defined 2D billboard type object placed into a 3D space; note that a centroid is considered any central point of a set of data where the centrality is not limited to any specific aspect but may be functionally a centroid if used as the center of some object or set of points or the like; see Benedek, paragraphs 0032-0033 teaching “he time series of at least one object shape subset is generated on the basis of the time stamps, from the at least one dynamic object shape subset, and a trajectory to the time series of the at least one object shape in the time series of the at least one object shape subset is assigned on the basis of the weighted centres of the at least one object shape subset. In the present embodiment of the invention, the trajectory of a dynamic object shape can be determined in a way other than that of the embodiments above, and the three-dimensional model of the substituting object shape can be substituted to this trajectory” such that here the “centres” are detected centroids and these positions are used to fuse or substitute the 3D models of the substituting object shape into the 3D surface topology where such representation is of a flat surface at a location corresponding to the centroid as the 3D model comprises 3D representations of flat surfaces paragraphs as in 0141-0148 where it is evidenced that the 3D representation may comprise a 3D representation of a 2D or flat surface as the “triangular lattice” and “textured” triangular lattices of the 3D objects that are inserted are 3D representations of such 2D, flat, triangular textured lattices as appearing in the 3D space as generated).
Regarding claim 8, Benedek teaches all that is required as applied to claim 1 above and further teaches, the processing circuitry further to generate the representation of the sensor data see Benedek, paragraphs 0082-0086 teaching “a point set corresponding to a scene is generated by means of a scanning device where the scene comprises at least one object shape. Then, in an operational step S120, the point set corresponding to a scene is divided into a foreground point set comprising a subset of at least one object shape corresponding to the foreground of the scene, and into a background point set corresponding to a scene background. In an operational step S130, from the foreground point set, at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set. In an operational step S140, the three-dimensional model of the scene background is generated on the basis of the background point set. Then, in an operational step S150, a three-dimensional model of at least one substituting object shape assignable to each of the at least one object shape, respectively, is generated from optical recordings. And finally, in an operational step S160, a combined three-dimensional model is generated on the basis of the background three-dimensional model and the three-dimensional model of at least one substituting object shape substituting each of the at least one object shape subset, respectively” such that here this division into different point sets of the sensor data is segmenting of the sensor data which is classified as background objects or dynamic foreground objects which are then processed accordingly and are used to generate the representation of the sensor data as these tracked and identified dynamic objects corresponding to the “object shape subset” in the sensor data are then replaced with representations of the sensor data such as the “substituting object shape”).
Regarding claim 9, Benedek teaches all that is required as applied to claim 1 above and further teaches wherein the one or more processors are comprised in at least one of (note that in each “system for” limitation below it is not required that the technique be actually used or being used in such a system so long as the system is capable of being used for such a purpose in any manner as such systems do not breathe any new life or meaning into the claim limitations; note that the method and processors can be considered to be comprised in numerous of the systems below, but as the claim is recited in the alternative only one limitation will be specifically addressed):
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content (see Benedek, paragraphs 0082-0086 as explained above where the “combined three-dimensional model” may be considered any or all of augmented reality content, virtual reality content, or mixed reality content as the real world data is used to generate and display virtual reality content and the virtual reality content is mixed with the real world data such that the output is augmented, virtual, and/or mixed reality content and is thus generated and displayed as well);
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Regarding claims 10-18, the instant claims correspond to a “system comprising one or more processors” where the system performs the same functions as recited with regard to the device of “One or more processors” as in claims 1-9, respectively. Such processors can already be considered a system for performing such functions. In light of this, the limitations of claims 10-18 correspond to the limitations of claims 1-9, respectively; thus they are rejected on the same grounds as claims 1-9, respectively.
Note that claim 11 contains a further recitation that the “first pass” is “a first pass of texturizing the detected 3D surface topology” where claim 2 only requires a “first pass of generating the visualization.” This limitation is also taught by Benedek as in the rejection of claim 2 where the first pass may be considered a first pass of texturizing the detected 3D surface topology (note that to mask such objects is interpreted as any manner of setting such data apart, logically, functionally, or otherwise, from other data where the selection of such data from the full set may be considered a mask of such data when used and for example the masked object may be processed or the data in which the mask objects have been masked may be processed; see Benedek, paragraphs 0082-0086 teaching “in an operational step S120, the point set corresponding to a scene is divided into a foreground point set comprising a subset of at least one object shape corresponding to the foreground of the scene, and into a background point set corresponding to a scene background. In an operational step S130, from the foreground point set, at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set. In an operational step S140, the three-dimensional model of the scene background is generated on the basis of the background point set” such that here this dividing out of the foreground points of the scene functions to mask the detected dynamic objects which is during a first pass of generating the visualization where a pass is some attempt at processing relating to the method or is a passing of such data used in generating the visualization, and for example also “at least one object shape subset corresponding to each of the at least one object shape, respectively, is separated from the foreground point set” such that this also functions to mask the object through such separation giving the ability to process that masked object, such as for generating a visualization of the masked object or background and such first pass may be considered a first pass of texturizing the detected 3D surface topology as this data is first passed in order to mask such objects in order to substitute them with a texturized version of the detected 3D surface topology).
Claim(s) 1, 10, and 19-20 is/are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Ren2.
Regarding claim 19, Ren teaches a method comprising:
computing, based at least on image data generated using one or more cameras of an ego-machine in an environment (see Ren, paragraph 0005, teaching “generating data corresponding to a plurality of photographs of the physical environment surrounding the vehicle taken in a plurality of directions extending outward from the vehicle, generating sensor data corresponding to a relative distance from the vehicle, and a direction from the vehicle of an object in the physical environment, generating a default three-dimensional projection surface centered around a virtual representation of the vehicle in a virtual environment, deforming the three-dimensional projection surface at a location in the virtual environment corresponding to the object in the sensor data, projecting the plurality of photographs onto the deformed three-dimensional projection surface, and displaying graphics corresponding to the deformed three-dimensional projection surface with the plurality of projected photographs with a display device” where the “vehicle” is an ego-machine in an environment with one or more cameras used in the computing explained further below, and see paragraphs 0034-0038 teaching “cameras 168 include one or more digital camera devices that generate photographic image data of the environment around the vehicle 102. An in-vehicle camera refers to a camera that is placed on the vehicle in a position that provides a view of at least a portion of the environment outside the vehicle. Common configurations of in-vehicle cameras including roof-mounted cameras and cameras that are mounted on other external locations of the vehicle including near the sides and bumpers of the vehicle. The cameras 168 include, but are not limited to, color, monochrome, low-light, and infrared cameras that generate image data of the environment around the vehicle 102 in a wide range of lighting conditions. In one configuration, the cameras are arranged around the vehicle 102 in substantially fixed positions to generate a mosaic of multiple photographs. The processor 108 merges the mosaic photographs to generate panoramic photographic image data of at least a portion of the environment around the vehicle 102, including objects that the vehicle could contact during operation” and “processor 108 stores the mosaic photographic data in the memory 120 as photographic texture data 136. As described below, the GPU 116 projects the photographic texture data onto a deformed projection surface in a virtual environment to form a visual representation of the environment around the vehicle 102” and “in-vehicle information system 104 is operatively connected to one or more range sensors 172. The range sensors 172 include one or more active and passive range finding devices” and “Using either passive or active sensors, the in-vehicle information system 104 typically receives the range data corresponding to objects in the environment around the vehicle 102 as a “point cloud.” As used herein, the term “point cloud” refers to a plurality of locations in a 3D space centered around the range finding sensor where a small portion of an object is detected. In one embodiment, the location of each point is identified with a set of 3D coordinate data using, for example, a Cartesian coordinate system with x, y, z coordinates corresponding to a relative width, height, and depth, respectively, of the location from the vehicle 102” and “in-vehicle information system 104 generates a large number of points in various locations throughout the environment around the vehicle 102 to identify the distance and approximate shape of objects in the environment around the vehicle 102” such that here the cameras correspond to the photographic cameras capturing images for texturing as well as any depth or rangefinding sensors that generate point cloud image data of the environment), and using one or more three-dimensional (3D) representations of one or more detected dynamic objects in the environment (see Ren as explained above in relation to paragraphs 0034-0038 where 3D representations of one or more detected objects in the environment are detected using the rangefinding sensors which “identify the distance and approximate shape of objects in the environment around the vehicle 102” where these objects may be detected dynamic objects as in paragraph 0041 teaching “the in-vehicle information system 104 generates photographic data of the environment around the vehicle 102, including both mobile and immobile objects and terrain feature objects, using the in-vehicle cameras 168 (block 208)” and as in paragraph 0043, “the in-vehicle information system 104 generates range data corresponding to objects around the vehicle 102 before, during, or after the generation of the photographic data of the objects around the vehicle 102 (block 212). In the in-vehicle information system 104, the depth sensors 172 generate range data for multiple objects that are within the fields of view of the cameras 168. As described above, the processor 108 generates a point cloud corresponding to objects in the environment around the vehicle with reference to the data received from the sensors” and “processor 108 identifies a point cloud 612 including a plurality of points, such as points 620 and 616, which correspond to portions of an object in the physical environment around the vehicle 102. As depicted in FIG. 6, the data point 620 in the point cloud 612 is located within the inverted hemisphere polygon mesh of the projection surface 604, while the data point 616 is located outside the inverted hemisphere polygon mesh of the projection surface 604. The points in the point cloud 612 form an approximation of the size and shape of an object in the physical environment around the vehicle 102 as detected by the depth sensors” and “the processor 108 deforms the shape of the projection surface 604 to enable the generation of a 3D graphical depiction of the object corresponding to the point cloud 612 with reduced distortion to the shape of the object” where paragraph 0020 teaches that ““object” refers to any physical entity that is visible to one or more cameras in a vehicle for display using a display device in the vehicle. Objects include, but are not limited to, terrain features, immobile bodies such as curbs, mailboxes, light posts, and parking meters, and mobile bodies including other vehicles, pedestrians, animals, and the like” and as in paragraph 0022 these representations of the detected dynamic objects are used in computing a 3D surface topology of the environment where ““projection surface” refers to an arrangement of polygons in a 3D virtual environment that form a surface for display of one or more photographs that are generated by cameras in the vehicle. In one embodiment, the projection surface is a continuous mesh formed from polygons that form a curved surface, such as a hemisphere, that extends outward from a location corresponding to the cameras in the vehicle that generate the photographic data. During operation of an in-vehicle information system, a display device depicts all or a portion of the projection surface with photographs taken by the cameras arranged on the polygons of the projection surface. The two-dimensional photograph data are projected into the 3D environment through the texturing process. As described below, an in-vehicle information system deforms the polygons in the projection surface to approximate the shapes of objects that are visible in the photographic data of the environment around the vehicle. The deformation of the 3D projection surface reduces distortion of the objects that are visible in the environment around the vehicle to provide occupants of the vehicle with a more realistic view of the 3D environment around the vehicle” such that this deformed 3D model of the surfaces that is generated based on the detected environment and dynamic objects detected as shapes is a 3D surface topology of the environment), a 3D surface topology of the environment (see Ren as explained above teaching image data generated using cameras of an ego-machine in an environment as well as 3D representations of one or more detected dynamic objects in the environment, where as explained in relation to paragraph 0022 the camera data as well as 3D data of detected dynamic objects in the environment are used to create a 3D surface topology of the environment in the form of a deformed projection surface based on camera data and 3D data about the objects in the environment where ““projection surface” refers to an arrangement of polygons in a 3D virtual environment that form a surface for display of one or more photographs that are generated by cameras in the vehicle. In one embodiment, the projection surface is a continuous mesh formed from polygons that form a curved surface, such as a hemisphere, that extends outward from a location corresponding to the cameras in the vehicle that generate the photographic data. During operation of an in-vehicle information system, a display device depicts all or a portion of the projection surface with photographs taken by the cameras arranged on the polygons of the projection surface. The two-dimensional photograph data are projected into the 3D environment through the texturing process. As described below, an in-vehicle information system deforms the polygons in the projection surface to approximate the shapes of objects that are visible in the photographic data of the environment around the vehicle. The deformation of the 3D projection surface reduces distortion of the objects that are visible in the environment around the vehicle to provide occupants of the vehicle with a more realistic view of the 3D environment around the vehicle”, ); and
generating a visualization of the image data projected onto the one or more 3D representations of the one or more detected dynamic objects in the 3D surface topology (see paragraph 0022 as explained above teaching ““projection surface” refers to an arrangement of polygons in a 3D virtual environment that form a surface for display of one or more photographs that are generated by cameras in the vehicle. In one embodiment, the projection surface is a continuous mesh formed from polygons that form a curved surface, such as a hemisphere, that extends outward from a location corresponding to the cameras in the vehicle that generate the photographic data. During operation of an in-vehicle information system, a display device depicts all or a portion of the projection surface with photographs taken by the cameras arranged on the polygons of the projection surface. The two-dimensional photograph data are projected into the 3D environment through the texturing process. As described below, an in-vehicle information system deforms the polygons in the projection surface to approximate the shapes of objects that are visible in the photographic data of the environment around the vehicle. The deformation of the 3D projection surface reduces distortion of the objects that are visible in the environment around the vehicle to provide occupants of the vehicle with a more realistic view of the 3D environment around the vehicle” such that here when generating the final visualization or “realistic view of the 3D environment around the vehicle” the actual image data captured with respect to the dynamic objects and environment around the vehicle is projected onto the 3D projection surface that describes the 3D surface topology of the environment and objects around the vehicle; see further paragraphs 0062-0063 teaching “the processor 108 projects the photographic image data onto the projection surface, including portions of the projection surface that are deformed to correspond to objects in the environment around the vehicle 102 (block 232). In the in-vehicle information system 104, one or more texture units in the GPU 116 project the photographic texture data 136 onto the polygons in the deformed projection surface. The shape of the deformed projection surface corresponds to the identified objects that are proximate to the vehicle 102. The stored photographic texture data 136 include the photographic representations of the objects, and the GPU 116 projects the photographs for the objects onto the corresponding deformed portions of the projection surface” and “Process 200 continues with generation of a visual depiction of at least a portion of the projection surface including a 3D proxy model corresponding to the vehicle 102 on a display device in the vehicle 102 (block 236). In the in-vehicle information system 104, the processor 108 generates a two-dimensional depiction of a portion of the 3D virtual environment through the display device 132” such that again there is generation of a visualization of the image data projected onto the one or more 3D representations of the one or more detected dynamic objects in the 3D surface topology as the photographic data is textured to match the deformed projection surface representing the topology of the environment).
Regarding claim 20, Ren teaches all that is required as applied to claim 19 above and further teaches wherein the method is performed by at least one of: (note that in each “system for” limitation below it is not required that the technique be actually used or being used in such a system so long as the system is capable of being used for such a purpose in any manner as such systems do not breathe any new life or meaning into the claim limitations; note that the method and processors can be considered to be comprised in numerous of the systems below, but as the claim is recited in the alternative only one limitation will be specifically addressed):
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content (see paragraphs 0062-0063 as explained above where the “generation of a visual depiction of at least a portion of the projection surface including a 3D proxy model corresponding to the vehicle 102 on a display device in the vehicle 102” such as a “two-dimensional depiction of a portion of the 3D virtual environment through the display device 132” may be considered any or all of augmented reality content, virtual reality content, or mixed reality content as the real world data is used to generate and display virtual reality content and the virtual reality content is mixed with the real world data such that the output is augmented, virtual, and/or mixed reality content and is thus generated and displayed as well);
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Regarding claims 1 and 10, the instant claims recite a device and system that perform the same functions as performed in the method of claim 19, but more broadly require “a representation of the sensor data projected onto…” instead of the more narrow requirement of claim 19 of “the image data projected onto…”. Thus as Ren anticipates the narrower version of the claim, Ren also anticipates the broader version of the claims as in claims 1 and 10. Thus additionally and alternatively, claims 1 and 10 are rejected on the same grounds as claim 19 above.
Response to Arguments
Applicant’s arguments, see “REMARKS”, filed 4/8/2026, with respect to the rejection(s) of claim(s) 19 and 20 under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Ren as explained above.
Applicant argues first on pages 11-12, in relation to claim 19, that Benedek does not teach “generates a visualization of the claimed image data projected onto these substitute 3D models”. Applicant notes that the “Office Action rejects claim 19 on the same basis as claim 1,” however, the Examiner notes that claims 1 and 10 as amended are broader than claim 19 as amended, as claims 1 and 10 recite differently to “generate a visualization of a representation of the sensor data”. Thus while Applicant’s arguments regarding claim 19 are persuasive, they are not persuasive with regard to claim 1.
Applicant's arguments filed 4/8/2026 have been fully considered but they are not persuasive with respect to claim 1. As noted above, claims 1, 10 and 19 do not recite the same claim limitations, thus Applicant’s arguments with respect to claim 19 do not specifically apply to the broader claim language of claims 1 and 10. As explained in the rejection of claims 1 and 10 above in view of the amendments to the claims, Benedek does teach, inter alia, generating a visualization of a representation of the sensor data, but does not teach the actual sensor data is projected. Thus claims 1 and 10 are rejected as explained above.
Applicant then argues with respect to claim 2 that in “Benedek, the removal of foreground points occurs during background reconstruction, and this background reconstruction is performed prior to substituting textured models of small foreground objects and prior to generating a visualization of the combined 3D model” and that allegedly “the cited removal of foreground points does not occur ‘during a first pass of generating the visualization’ of the representation of the sensor data projected onto the one or more 3D representations in the 3D surface topology.” The Examiner respectfully disagrees. As noted in the rejection of claim 2 above, a “first pass of generating the visualization” is extremely broad and could constitute any step or passing of processing data that will be used in generating the visualization. Regardless of whether masking occurs in an initial reconstruction step, this masking is specifically for the purpose of generating the visualization such that the masking is necessary to generate the visualization and constitutes a first pass of generating the visualization. Thus the claim and those identified as similar by Applicant such as claims 8, 11, and 17 are rejected as explained above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Binder (US PGPUB No. 2023/0025209) – teaching a system which images dynamic objects and the environment around a vehicle and generates a 3D surface topology where sensor data is projected onto the 3D surface topology as in paragraph 0028 teaching “a surroundings model 200 for displaying surroundings 150 of vehicle 100. Surroundings model 200 includes a close-range projection surface 210 as well as a vehicle model 250 which is centrally located on close-range projection surface 210. Vehicle model 250 is loaded advantageously from electronic memory 130 and corresponds to at least one type of vehicle 100. Close-range projection surface 210 represents a close range of surroundings 150 of vehicle 100 and preferably includes a grid that is fine-meshed, especially relative to the size of vehicle model 250. The surroundings model also includes a far-range projection surface 290. Far-range projection surface 290 in this exemplary embodiment is disposed essentially perpendicular to close-range projection surface 210. As an alternative to the exemplary embodiment shown in FIG. 2 , close-range projection surface 210 and far-range projection surface 290 may also include different sections of a tub-shape surface, so that close-range projection surface 210 and far-range projection surface 290 are situated directly next to each other, for example, and may to some extent have a curvature. Far-range projection surface 290 represents a far range of surroundings 150 of vehicle 100. In this exemplary embodiment, far-range projection surface 290 is in one piece and is formed as the inner surface of a cylindrical lateral surface. Alternatively, far-range projection surface 290 may also be implemented as part of a lateral surface, e.g., as a half shell, and disposed in the direction of travel, or multiple far-range projection surfaces 290 may be provided, positioned rectangularly around close-range projection surface 210, for example. In other words, textures or camera images 220, 230, which image or represent the close range of surroundings 150 of vehicle 100, are displayed on close-range projection surface 210 in the surroundings model. On far-range projection surface 290, on the other hand, textures or camera images 220, 230 are displayed which image or represent the far range, that is, a more remote surroundings area of surroundings 150. Close-range projection surface 210 is deformed spatially, that is, three-dimensionally depending on detected distances from objects 240, 241 and 242 and/or depending on loaded standard models for recognized objects 240, 241 and/or 242. Thus, close-range projection surface 210 advantageously forms an envelope curve around objects 240, 241 and 242 and a background of surroundings 150 in the close range of vehicle 100. For example, vehicle 100 is in motion in this exemplary embodiment. In other words, the surroundings model changes continuously, since, for example, the textures on close-range projection surface 210 and far-range projection surface 290 shift according to the present vehicle position or, e.g., new objects appear in close-range projection surface 210 or recognized objects 240, 241 and/or 242 exit from close-range projection surface 210. For instance, in a predetermined period of time prior to the present moment, the vehicle has moved from a first vehicle position 124 via a second-last vehicle position 123 and via a last vehicle position 122 to a present vehicle position 121. During this travel from first vehicle position 124 to present vehicle position 121, a sequence of camera images 220 was captured continuously by each of cameras 110 on the vehicle and individual camera images 230 were stored, in doing so, each stored camera image 230 was assigned the specific vehicle position 124, 123, 122 or 121 of vehicle 100 at the moment respective camera image 230 was captured”.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT E SONNERS whose telephone number is (571)270-7504. The examiner can normally be reached Mon-Friday 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT E SONNERS/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
1 US PGPUB No. 20160093101
2 US PGPUB No. 2014
Prosecution Insights