DETAILED ACTION
With regard to the amendment filed 06/16/2026, the applicant amended claims 1, 3, 4, 9, 11 and 17. Claims 2, 6 and 7 are cancelled. No new matter is added.
The Applicant sufficiently amended claim in order to overcome rejections under 35 U.S.C. 101 Abstract idea. Therefore, the rejections are withdrawn.
Response to Arguments
Applicant’s arguments, see pg. 6-8, filed 06/16/2026, with respect to the rejection(s) of claim(s) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Benemann et al. See sections below.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Claims 17-19 are interpreted to invoke 35 U.S.C. 112(f).
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a scanning device” and “a central station” in claim in claims 17-19.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. The original specification paragraph [0021-0022] and [0026] discloses hardware structure(s) or processor(s) are equivalent thereof to execute the functions by the device and station.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 8-19 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (DE 102019205474 A1, as provided) in view of Tang et al. (US Pub No. 20220101508 A1, as provided) and further in view of Benemann et al. (US Pub No. 20200099824 A1).
Regarding Claim 1,
Chen discloses A method comprising: scanning an environment of a motor vehicle; (Chen, Description, discloses in addition to the primary and secondary sensor setup 20th ,30th Another sensor device, such as a thermal camera, can be used to monitor the surroundings 60 of the vehicle 50 recorded and generated third environment data. In the case of a thermal camera, the third environmental data include thermal data from the surrounding area 60 located objects; object image is captured with camera sensor around the vehicle)
recognizing a scanned object in the environment; (Chen, Description, discloses object recognition plays a central role in automated driving. Objects that are in the vicinity of a self-driving or partially self-driving vehicle must be recognized with a high degree of accuracy and timeliness. This is used to take measures to avoid or at least reduce a collision with such objects, such as other vehicles, road users, people, animals, buildings, plantings, public and private transport facilities ... etc.; Compared to the punctual information generated by the primary sensor device such as radar or lidar sensors, which are only correlated with one another in a group assignment process (clustering), more relationships between the individual pixels can be detected in the second environment data without complex data processing. In particular, one can benefit from the surface texture information that is to be extracted from the second environment data. The second environment data are thus used to optimize or verify the object recognition by means of the first environment data. The object recognition is therefore based on a more extensive information content of the detected objects, which increases the reliability of the object recognition; object in captured image is recognized being other vehicles, people, animals, buildings etc.)
determining a texture of the scanned object; (Chen, Description, discloses secondary sensor device is, for example, a camera, for example an RGB camera, a stereo camera and / or a surround view camera. The secondary sensor device is preferably designed to detect a surface texture (e.g. color, surface normal, roughness) of detected objects. The surface texture is analyzed by the evaluation unit, from which material class data is extracted. The material class data contain one or more material classes of the objects detected by the secondary sensor device. The material class data are finally combined with the first surrounding data to form a common data record; texture of detected and recognized object is determined by image analysis of the object) and
Chen does not explicitly disclose creating a textured representation of the scanned object.
Tang discloses creating a textured representation of the scanned object. (Tang, [0006], discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Chen in view of Tang having a method of detecting texture regions in a scanned object image with the teachings of Tang having, by system, encoding texture detected from an object image and render by creating similar texture in an image to depict any changes in the object image or determining specific features by texture patterns in an object image to improve detection of object in applications including vehicle damage where texture patterns define extent of damage to the vehicle object image.
The combination of Chen and Tang does not explicitly disclose wherein a plurality of scans of the scanned object are collected; performing distortion rectification on the textured representation on the scans and based on a distortion of the scans which is based on a perspective of the motor vehicle relative to the scanned object;
Benemann discloses wherein a plurality of scans of the scanned object are collected; (Benemann, [0016], Fig. 2, discloses an image sensor of a vehicle (e.g., an autonomous vehicle) may capture an image of a scene within a first field of view of the image sensor. In some examples, the image capture may be performed by scanning across the scene (e.g., vertically or horizontally) to capture a portion of the scene (e.g., a first scan line) before capturing another portion of the scene (e.g., a second scan line), and so on, such as in a rolling shutter image capture. The rolling shutter image capture may produce scan lines of an image frame. A scan line may include scan line data (e.g., pixel data) and end of line data. The end of line data may indicate an end of the scan line; plurality of scans of the same scan object are collected as obtained as scan line 1, scan line 2 etc. as in in fig. 2, object vehicle scans)
performing distortion rectification on the textured representation on the scans and based on a distortion of the scans which is based on a perspective of the motor vehicle relative to the scanned object; (Benemann, [0019], [0088], [0110], discloses the computing system may use the time data added to one or more scan lines to modify a distorted image to produce a corrected image. For example, the distorted image may include one or more distortion effects caused by motion associated with a rolling shutter image capture. As non-limiting examples, the distortion effect(s) may include wobble, skew, spatial aliasing, and/or temporal aliasing, etc. Modifying the distorted image may include correcting or compensating for the distortion effect(s). at 906, the process 900 may include determining whether an image is distorted. If, at 906, it is determined that the image is distorted, then the process 900 may include modifying at least a portion of the image to correct or compensate for a distortion effect (at 908), e.g., as described herein with reference to Fig. 6. In various examples, the distortion effects may be caused by motion associated with a rolling shutter image capture (e.g., relative motion between an image sensor and an object in the environment). As non-limiting examples, the distortion effect(s) may include wobble, skew, spatial aliasing, and/or temporal aliasing. In some examples, the image may be modified based at least in part on the time data added to one or more scan lines; memory 1118 can further include one or more maps (not shown) that can be used by the vehicle 1102 to navigate within the environment. For the purpose of this discussion, a map can be any number of data structures modeled in two dimensions, three dimensions, or N-dimensions that are capable of providing information about an environment, such as, but not limited to, topologies (such as intersections), streets, mountain ranges, roads, terrain, and the environment in general. In some instances, a map can include, but is not limited to: texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV/HSL color information), and the like), intensity information (e.g., LIDAR information, RADAR information, and the like); spatial information (e.g., image data projected onto a mesh, individual surfels (e.g., polygons associated with individual color and/or intensity)), reflectivity information (e.g., specularity information, retroreflectivity information, BRDF information, BSSRDF information, and the like). In one example, a map can include a three-dimensional mesh of the environment. In some instances, the map can be stored in a tiled format, such that individual tiles of the map represent a discrete portion of an environment and can be loaded into working memory as needed. In at least one example, the one or more maps can include at least one map (e.g., images and/or a mesh). In some examples, the vehicle 1102 can be controlled based at least in part on the maps. That is, the maps can be used in connection with the localization component 1120, the perception component 1122, and/or the planning component 1124 to determine a location of the vehicle 1102, identify objects in an environment, and/or generate routes and/or trajectories to navigate within an environment; objects scan obtained and represented as the texture are corrected (distortion rectification) for the distortion introduced by the moving environment such as while in motion when capturing the image frames and the scans obtained are distorted due to rolling shutter motion effect)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Chen as modified by Tang and further in view of Benemann having a method of detecting texture regions in a scanned object image and representing the texture of the object scanned with the teachings of Benemann having, by system, correcting distortion introduced by when capturing image frame scans while in motion such as when in vehicle that are due to rolling shutter effect in order to accurately capture objects in the surroundings for applications including obstacle avoidance while automatic navigation of a vehicle.
Regarding Claim 3,
The combination of Chen, Tang and Benemann further discloses wherein the scans originate from different motor vehicles. (Tang, [0006], discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 4,
The combination of Chen, Tang and Benemann further discloses wherein the scans show different sections of the scanned object. (Tang, [0006], 0015], FIG. 1, discloses functional units of an apparatus for image-based anomaly detection for vehicle damage assessment in accordance with one embodiment of the present invention. The apparatus may comprise an image processor 101 configured for performing various imaging processing techniques such as image segmentation and augmentation on input images of vehicles; a texture encoder 102 configured for generating texture feature maps from vehicle images processed by the image processor 101; a frame extractor 103 configured for extracting vehicle frame images from vehicle images processed by the image processor 101; an IVF reconstructor 104 configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor 103; an image reconstructor 105 configured for reconstructing intact-vehicle images from the reconstructed IVF images generated by the IVF reconstructor 104 and texture feature maps generated by the texture encoder 102; and an anomaly locator 106 configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; IVF reconstructor 104 may include a frame encoder 1041 configured for mapping extracted vehicle frame images to generate IVF feature maps and a frame decoder 1402 configured for remapping IVF feature maps back to IVF images; the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image of different sections of the vehicles and details of sections of vehicles are obtained by edge segmentations). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 5,
The combination of Chen, Tang and Benemann further discloses wherein the scans show different sections of the scanned object. (Tang, [0006], discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image). Additionally, the rational and motivation to combine the references Chen and Tang as applied in rejection of claim 1 apply to this claim.
Regarding Claim 8,
The combination of Chen, Tang and Benemann further discloses wherein a geographical position of the motor vehicle is determined; Chen, Description, discloses vehicle can be a passenger vehicle and / or a utility vehicle, for example a land vehicle, an industrial vehicle, an industrial machine, a vehicle for a swap body, a mobile robot and / or an automated driverless transport system; primary sensor device is, for example, a distance sensor, for example a radar sensor, a lidar sensor and / or an ultrasonic sensor. The first environment data generated by the primary sensor device typically contain position data of one or more objects detected by the primary sensor device in the environment of the vehicle; For example, in the event that the primary sensor device comprises a radar sensor, a radio frequency signal (hereinafter: RF signal) is sent from a transmitter of the radar sensor. The RF signal is scattered back or reflected on an object in the vicinity of the vehicle and propagates back to a receiver of the radar sensor. The period in which the RF signal is propagated between the time of transmission and reception is generally known in the technical field as the time of flight (ToF). The position of the object reflecting the RF signal can typically be derived from the flight time in the form of determine an elevation angle, a side angle and a distance; if the primary sensor device comprises a lidar sensor, a light signal can be sent from a transmitter of the lidar sensor. The light signal is scattered or reflected back on an object located in the vicinity of the vehicle and propagates back to a receiver of the lidar sensor. The period of time in which the light signal is propagated between the time of transmission and reception is also known as the time of flight. The position data of detected objects can also be determined from the flight time; geographical position of vehicle in the environment is determined) and wherein the scanned object is recognized based on map data in a region of the geographical position. (Tang, [0006], discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 9,
The combination of Chen, Tang and Benemann further discloses wherein a geographical position of the motor vehicle is determined; (Chen, Description, discloses vehicle can be a passenger vehicle and / or a utility vehicle, for example a land vehicle, an industrial vehicle, an industrial machine, a vehicle for a swap body, a mobile robot and / or an automated driverless transport system; primary sensor device is, for example, a distance sensor, for example a radar sensor, a lidar sensor and / or an ultrasonic sensor. The first environment data generated by the primary sensor device typically contain position data of one or more objects detected by the primary sensor device in the environment of the vehicle; For example, in the event that the primary sensor device comprises a radar sensor, a radio frequency signal (hereinafter: RF signal) is sent from a transmitter of the radar sensor. The RF signal is scattered back or reflected on an object in the vicinity of the vehicle and propagates back to a receiver of the radar sensor. The period in which the RF signal is propagated between the time of transmission and reception is generally known in the technical field as the time of flight (ToF). The position of the object reflecting the RF signal can typically be derived from the flight time in the form of determine an elevation angle, a side angle and a distance; if the primary sensor device comprises a lidar sensor, a light signal can be sent from a transmitter of the lidar sensor. The light signal is scattered or reflected back on an object located in the vicinity of the vehicle and propagates back to a receiver of the lidar sensor. The period of time in which the light signal is propagated between the time of transmission and reception is also known as the time of flight. The position data of detected objects can also be determined from the flight time; geographical position of vehicle in the environment is determined) and wherein the scanned object is recognized based on map data in a region of the geographical position. (Tang, [0006], discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 10,
The combination of Chen, Tang and Benemann further discloses wherein a texture on one section of the scanned object, with respect to which no scan is present, is determined based on a texture on another section of the scanned object. (Tang, [0006], [0016-0018], [0022], the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image). discloses the frame encoder 1041 may comprise a stack of convolutional layers configured to extract multi-scale context information represented in an IVF feature map. The frame decoder 1042 may comprise a stack of deconvolutional layers to reconstruct an intact vehicle frame from the IVF feature map. Skip connections may be used to connect the convolutional layers of the frame encoder 1041 to the deconvolution layers of the frame decoder 1042 such that fine details and reconstruction locations of edges in the vehicle frame at pixel level can be better preserved; the texture encoder 102, frame extractor 103, IVF reconstructor 104, image reconstructor 105, and anomaly locator 106 are generative convolutional neural networks (generators) which are trained under a generative contextualized adversarial network (GCAN) model with a training dataset containing a plurality of images of undamaged vehicles, IVF images and augmented vehicle frame images of the vehicles. The trained generators are then used to perform the anomaly detection in the training of the generators, the apparatus further includes: a frame encoder discriminator 107 configured for generating a score of likelihood between two IVF feature maps and an image discriminator 108 configured for generating a score of likelihood between an input vehicle image and a corresponding reconstructed vehicle image. In one embodiment, the frame encoder discriminator 107 and image discriminator 108 may be discriminative convolutional neural networks. The frame encoder discriminator 107 and image discriminator 108 can also be trained via the GCAN model; [0022] S203: perform a damage augmentation to the IVF image D201 to generate a damage-augmented vehicle frame image D203 by the image processor 101; wherein in one embodiment, the damage augmentation comprises modifying one or more randomly selected regions in the vehicle frame image D201, and wherein the modification includes one or more of random translation, random rotation, random scaling, random line addition, noise addition, partial occlusion, and region substitution by replacing the selected region with a dissimilar region of another vehicle frame image; various sections in vehicle object image are scanned and analyzed using machine learning and edges of sections of textures are determined to detect anomaly in sections of images that are different and remaining pixels are textured the same other than the anomaly regions if scans are not available of that section and masked similar color pixels and regions of image are substituted by dissimilar regions from other image). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 11,
The combination of Chen, Tang and Benemann further discloses wherein a texture on one section of the scanned object, with respect to which no scan is present, is determined based on a texture on another section of the scanned object. (Tang, [0006], [0016-0018], [0022], discloses the frame encoder 1041 may comprise a stack of convolutional layers configured to extract multi-scale context information represented in an IVF feature map. The frame decoder 1042 may comprise a stack of deconvolutional layers to reconstruct an intact vehicle frame from the IVF feature map. Skip connections may be used to connect the convolutional layers of the frame encoder 1041 to the deconvolution layers of the frame decoder 1042 such that fine details and reconstruction locations of edges in the vehicle frame at pixel level can be better preserved; the texture encoder 102, frame extractor 103, IVF reconstructor 104, image reconstructor 105, and anomaly locator 106 are generative convolutional neural networks (generators) which are trained under a generative contextualized adversarial network (GCAN) model with a training dataset containing a plurality of images of undamaged vehicles, IVF images and augmented vehicle frame images of the vehicles. The trained generators are then used to perform the anomaly detection in the training of the generators, the apparatus further includes: a frame encoder discriminator 107 configured for generating a score of likelihood between two IVF feature maps and an image discriminator 108 configured for generating a score of likelihood between an input vehicle image and a corresponding reconstructed vehicle image. In one embodiment, the frame encoder discriminator 107 and image discriminator 108 may be discriminative convolutional neural networks. The frame encoder discriminator 107 and image discriminator 108 can also be trained via the GCAN model; [0022] S203: perform a damage augmentation to the IVF image D201 to generate a damage-augmented vehicle frame image D203 by the image processor 101; wherein in one embodiment, the damage augmentation comprises modifying one or more randomly selected regions in the vehicle frame image D201, and wherein the modification includes one or more of random translation, random rotation, random scaling, random line addition, noise addition, partial occlusion, and region substitution by replacing the selected region with a dissimilar region of another vehicle frame image; various sections in vehicle object image are scanned and analyzed using machine learning and edges of sections of textures are determined to detect anomaly in sections of images that are different and remaining pixels are textured the same other than the anomaly regions if scans are not available of that section and masked similar color pixels and regions of images are substituted with dissimilar regions from other image). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 12,
The combination of Chen, Tang and Benemann further discloses wherein the texture on the section is determined by machine learning. (Tang, [0004], discloses a problem with the aforesaid neural network-based approaches is that they require a large number of annotated images of damaged vehicles or vehicle components as training datasets for machine learning. As images of vehicle damages are relatively scarce and damages are too varied (e.g., there are no typical damages), the training datasets based on images of damaged vehicles are always insufficient to achieve a detection capability to detect unknown/unseen vehicle damages of stochastic types and extents, such as shape deformation of vehicle frame, from images which are taken in various contexts. Thus, there is a need in the field for a better approach; texture of image section of object is determined by machine learning algorithm). Additionally, the rational and motivation to combine the references Chen and Tang as applied in rejection of claim 1 apply to this claim.
Regarding Claim 13,
The combination of Chen, Tang and Benemann further discloses wherein the texture on the section is determined by inpainting. (Tang, [0004], [0022], discloses a problem with the aforesaid neural network-based approaches is that they require a large number of annotated images of damaged vehicles or vehicle components as training datasets for machine learning. As images of vehicle damages are relatively scarce and damages are too varied (e.g., there are no typical damages), the training datasets based on images of damaged vehicles are always insufficient to achieve a detection capability to detect unknown/unseen vehicle damages of stochastic types and extents, such as shape deformation of vehicle frame, from images which are taken in various contexts. Thus, there is a need in the field for a better approach; texture of image section of object is determined by machine learning algorithm). (Tang, [0006], [0015], Fig. 1, discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; functional units of an apparatus for image-based anomaly detection for vehicle damage assessment in accordance with one embodiment of the present invention. The apparatus may comprise an image processor 101 configured for performing various imaging processing techniques such as image segmentation and augmentation on input images of vehicles; a texture encoder 102 configured for generating texture feature maps from vehicle images processed by the image processor 101; a frame extractor 103 configured for extracting vehicle frame images from vehicle images processed by the image processor 101; an IVF reconstructor 104 configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor 103; an image reconstructor 105 configured for reconstructing intact-vehicle images from the reconstructed IVF images generated by the IVF reconstructor 104 and texture feature maps generated by the texture encoder 102; and an anomaly locator 106 configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions. The IVF reconstructor 104 may include a frame encoder 1041 configured for mapping extracted vehicle frame images to generate IVF feature maps and a frame decoder 1402 configured for remapping IVF feature maps back to IVF images; texture feature maps of vehicle object image are obtained and represented as image and image parts where the damaged parts are skewed (distorted) are compared with trained stored image dataset to determine if they are distorted and segmented as anomaly and inpainted accordingly). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 14,
The combination of Chen, Tang and Benemann further discloses providing map data which comprise the scanned object. (Tang, [0006], [0015], Fig. 1, discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; functional units of an apparatus for image-based anomaly detection for vehicle damage assessment in accordance with one embodiment of the present invention. The apparatus may comprise an image processor 101 configured for performing various imaging processing techniques such as image segmentation and augmentation on input images of vehicles; a texture encoder 102 configured for generating texture feature maps from vehicle images processed by the image processor 101; a frame extractor 103 configured for extracting vehicle frame images from vehicle images processed by the image processor 101; an IVF reconstructor 104 configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor 103; an image reconstructor 105 configured for reconstructing intact-vehicle images from the reconstructed IVF images generated by the IVF reconstructor 104 and texture feature maps generated by the texture encoder 102; and an anomaly locator 106 configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions. The IVF reconstructor 104 may include a frame encoder 1041 configured for mapping extracted vehicle frame images to generate IVF feature maps and a frame decoder 1402 configured for remapping IVF feature maps back to IVF images; texture feature maps of vehicle object image are obtained and represented as image and image parts where the damaged parts are skewed (distorted) are compared with trained stored image dataset to determine if they are distorted and segmented as anomaly and inpainted accordingly). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 15,
The combination of Chen, Tang and Benemann further discloses providing map data which comprise the scanned object. (Tang, [0006], discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Regarding Claim 16,
The combination of Chen, Tang and Benemann further discloses wherein the map data are communicated to a plurality of motor vehicles. (Tang, [0006], discloses the functional units comprise at least an image processor configured for segmenting one or more input vehicle images; a texture encoder configured for generating texture feature maps from the segmented vehicle images; a frame extractor configured for extracting vehicle frame images from the segmented vehicle images; an intact-vehicle frame (IVF) reconstructor configured for reconstructing IVF images from vehicle frame images extracted by the frame extractor; an image reconstructor configured for reconstructing intact-vehicle images from reconstructed IVF images generated by the IVF reconstructor and texture feature maps generated by the texture encoder; an anomaly locator configured for comparing processed vehicle images and their corresponding reconstructed intact-vehicle images to detect anomaly regions; a frame encoder discriminator configured for generating a score of likelihood between two IVF feature maps; and an image discriminator configured for generating a score of likelihood between the input vehicle image and the reconstructed vehicle image; texture feature maps of vehicle object image are obtained and represented as image). Additionally, the rational and motivation to combine the references Chen, Tang and Benemann as applied in rejection of claim 1 apply to this claim.
Claims 17-19 recite system with elements corresponding to the method steps recited in Claims 1-2 and 13 respectively. Therefore, the recited elements of the system claims 17-19 are mapped to the proposed combination in the same manner as the corresponding steps of Claims 1-2 and 13 respectively. Additionally, the rationale and motivation to combine the Chen, Tang and Benemann references presented in rejection of Claim 1, apply to these claims.
Furthermore, the combination of Chen and Tang further discloses A system comprising: a scanning device on board a motor vehicle, wherein the scanning device is configured for scanning an environment of the motor vehicle; and a central station configured to receive a scan (Tang. [0022], discloses S203: perform a damage augmentation to the IVF image D201 to generate a damage-augmented vehicle frame image D203 by the image processor 101; wherein in one embodiment, the damage augmentation comprises modifying one or more randomly selected regions in the vehicle frame image D201, and wherein the modification includes one or more of random translation, random rotation, random scaling, random line addition, noise addition, partial occlusion, and region substitution by replacing the selected region with a dissimilar region of another vehicle frame image).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PINALBEN V PATEL whose telephone number is (571)270-5872. The examiner can normally be reached M-F: 10am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at 571-272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Pinalben Patel/Examiner, Art Unit 2673
Prosecution Insights