DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over ARZOUAN (US 12175651 B1, Date Filed: 2024-03-22), and in view of GOULD (EP 3832528 A1), and further in view of Chen (US 11288789 B1).
Re Claim 1, ARZOUAN discloses a method, comprising
receiving a first plurality of vehicle surface images associated with a vehicle, the first plurality of vehicle surface images used to identify a vehicle object model by applying a neural network to the first plurality of vehicle surface images, the first plurality of vehicle surface images captured from a first plurality of perspectives (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13);
receiving a second plurality of vehicle surface images associated with the vehicle, the second plurality of vehicle surface images captured when the vehicle is illuminated by a plurality of structured lighting pattern panels (see ARZOUAN: e.g., --(33) Image sensors 212 are arranged at different views, optionally with at least some overlap, for capturing images of different parts of the surface of the vehicle. Image sensors 212 may be, for example, standard visible light sensors (e.g., CCD, CMOS sensors, and/or red green blue (RGB) sensor). Computing device 204 receives sequences of time-spaced images captured by multiple image sensors 212 positioned in different views, for example cameras.
(34) Image sensors 212 may transmit captured images to computing device 204, for example, via a direct connected (e.g., local bus and/or cable connection and/or short range wireless connection), and/or via a network 210 and a network interface 222 of computing device 204 (e.g., where sensors are connected via a wireless network, internet of things (IoT) technology and/or are located remotely from the computing device).--, in lines 52-67, col. 10; and, -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13);
ARZOUAN does not explicitly discloses above structure lighting pattern is illuminated by a plurality of striped lighting pattern panels,
GOULD discloses (vehicle surface for damage detection} illuminated by a plurality of structured lighting pattern panels (see GOULD: e.g., -- a structured light source 22 arranged to direct structured light 24 at the vehicle pathway 14 for illuminating the vehicle 12 on the pathway 14 with a structured light image (not shown). In this embodiment the structured light source 22 extends up one side wall 16a from the vehicle pathway 14, across the roof section 16c and down the opposite side wall 16b, back to the vehicle pathway 14 to form an arch of structured lighting. This arrangement enables the structured light image to be projected onto both sides, and the roof, of the vehicle 12 as it passes the structured light source 22. The structured light source 22 is a light array having a set of LED strips arranged in parallel. The LED strips extend along each light array, from the bottom to the top and across the roof section. LEDs can for example comprise ultra-bright cool white LED tape, with a luminosity of 2880 lumens per meter. In one example a set of twenty LED strips can be arranged into 14.25mm wide grooves spaced 15.75mm apart and set 9mm deep with a 10mm backing behind them. Semi opaque diffusers (not shown) can be provided over each strip of LEDs to create a flat light from each strip of tape.
In other embodiment the structured light source 22 can have any suitable configuration arranged to project the structured light image onto one or more surfaces and in some cases all outer surfaces of the vehicle; for example, each light source can include a laser projector configured to project one or more light patterns.
The damage assessment zone 10 includes damage assessment cameras of a first type 26, namely high speed 'dent detecting' cameras arranged to image dents on the vehicle.
Multiple dent cameras 26 can be arranged inside the tunnel 2A, located on the side walls 16a, 16b and roof 16c to form an arch, as shown in Figure 3B, so that the sides and roof of the vehicle 12 can be simultaneously imaged.
Each dent detecting camera 26 is arranged with a field of view 26a comprising a structured light portion of the tunnel volume in which the structured light image will be reflected to be visible to the first camera 26 by a vehicle 12 moving along the vehicle pathway 14. Thus, the dent detecting cameras 26 are located in the tunnel 2A so that the dent camera field of view 26a overlaps with the striped pattern reflecting on the vehicle. The system can be calibrated for an average or expected vehicle profile for example.
Should the vehicle 12 have a dent in the bodywork, the striped reflections will distort around the dent, for example creating a circle like shape in the reflection. The images captured by the dent detection cameras 26 can then be used retrospectively to analyse whether a vehicle 12 has dents at a certain point in time. Thus, the field of view 26a of the dent detecting cameras 26 overlap the reflected striped image area 28 on the vehicle 12.
The structured light source can be in the form of panels mounted on the tunnel side walls and/or roof, either freestanding or mounted separately to the tunnel wall.--, in [0050]-[0056]);
ARZOUAN and GOULD are combinable as they are in the same field of endeavor: using neural network and machine learning in vehicle surface images analysis for damage detection. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify ARZOUAN’s method using GOULD’s teachings by including (vehicle surface for damage detection} illuminated by a plurality of structured lighting pattern panels to ARZOUAN’s structured lighting patterns in order to improve the identification of any damage/dents on the vehicle surface (see GOULD: e.g., in [0050]-[0056]);
ARZOUAN as modified by GOULD further disclose wherein the second plurality of vehicle surface images correspond to a plurality of vehicle object model components, the second plurality of vehicle surface images captured from a second plurality of perspectives, wherein the second plurality of vehicle surface images illuminated by the plurality of striped lighting pattern panels are analyzed to detect vehicle imperfections, wherein the second plurality of vehicle surface images correspond to the plurality of vehicle object model components and detected vehicle imperfections are evaluated to estimate damage to an associated vehicle object model component (see ARZOUAN: e.g.,: -- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13);
generating a map visual representation associated with estimated damage to the associated vehicle object model component (see ARZOUAN: e.g., --iterating the identifying redundancy for identifying a plurality of single physical damage regions within a plurality of physical components of the vehicle, and generating a map of the plurality of physical components of the vehicle marked with respective location of each of the plurality of single physical damage regions.--, in lines 17-23, col. 2);
ARZOUAN (as modified by GOULD) however still do not explicitly discloses above generating a map visual representation is generating a heatmap visual representation;
Chen discloses generating a heatmap visual representation associated with estimated damage to the associated vehicle object model component (see CHEN: e.g., Fig. 17, “1702 Generate Heat map corresponding to vehicle image”; and,
-- The image processing system may next perform a statistical processing routine on each of the identified target object image components, such as a deep learning technique, like one that uses one or more convolutional neural networks, to detect changes, the likelihood of changes, and/or a quantification of an amount or type of change, to each such body component of the target object as depicted in the target object images. Such change detections may then be used for other purposes to, for example, estimate repair costs, estimate the amount of change that has occurred in the object, estimate the amount of time or effort needed to correct or fix the change, etc. Still further, in one case, the change determination may be displayed to a user in the form of a heat map that illustrates areas of the target object that have undergone change, the amount of such change, the type of such change, etc…. a method performed by an image processing system includes obtaining a source image that includes an image of a damaged vehicle, and applying, by a computing device included in the image processing system, an image processor component of the image processing system to the source image to thereby generate a set of values corresponding to damage to the vehicle. Each value included in the set of values respectively corresponds to respective damage at a respective location on a respective segment of the vehicle, and each value may be indicative of a determined degree of severity of damage to the respective location, a level of accuracy of the determined degree of severity of damage at the respective location, a type of damage at the respective location, and/or a likelihood of an occurrence of damage at the respective location. Additionally, the method includes determining, by the computing device from a set of vehicle parts of the vehicle, one or more parts needed to repair the vehicle. The determination of the one or more parts needed to repair the vehicle is based on one or more particular values included in the set of values generated by the image processor component. -, in line 47, col. 2 through line 27, col 3);
Chen and ARZOUAN are combinable as they are in the same field of endeavor: using neural network and machine learning in vehicle surface images analysis for damage detection. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify ARZOUAN’s method using Chen’s teachings by including generating a heatmap visual representation associated with estimated damage to the associated vehicle object model component to ARZOUAN’s generating a map visual representation in order to generate and display to users a heatmap that corresponds to respective damage at a respective location on a respective segment of the vehicle..etc., (see Chen: e.g. Fig. 17, and in line 47, col. 2 through line 27, col 3);
ARZOUAN as modified by GOULD, and Chen further disclose displaying the heatmap visual representation to show vehicle imperfections (see ARZOUAN: e.g., --iterating the identifying redundancy for identifying a plurality of single physical damage regions within a plurality of physical components of the vehicle, and generating a map of the plurality of physical components of the vehicle marked with respective location of each of the plurality of single physical damage regions.--, in lines 17-23, col. 2; also see CHEN: e.g., Fig. 17, “1702 Generate Heat map corresponding to vehicle image”; and,
-- The image processing system may next perform a statistical processing routine on each of the identified target object image components, such as a deep learning technique, like one that uses one or more convolutional neural networks, to detect changes, the likelihood of changes, and/or a quantification of an amount or type of change, to each such body component of the target object as depicted in the target object images. Such change detections may then be used for other purposes to, for example, estimate repair costs, estimate the amount of change that has occurred in the object, estimate the amount of time or effort needed to correct or fix the change, etc. Still further, in one case, the change determination may be displayed to a user in the form of a heat map that illustrates areas of the target object that have undergone change, the amount of such change, the type of such change, etc…. a method performed by an image processing system includes obtaining a source image that includes an image of a damaged vehicle, and applying, by a computing device included in the image processing system, an image processor component of the image processing system to the source image to thereby generate a set of values corresponding to damage to the vehicle. Each value included in the set of values respectively corresponds to respective damage at a respective location on a respective segment of the vehicle, and each value may be indicative of a determined degree of severity of damage to the respective location, a level of accuracy of the determined degree of severity of damage at the respective location, a type of damage at the respective location, and/or a likelihood of an occurrence of damage at the respective location. Additionally, the method includes determining, by the computing device from a set of vehicle parts of the vehicle, one or more parts needed to repair the vehicle. The determination of the one or more parts needed to repair the vehicle is based on one or more particular values included in the set of values generated by the image processor component. -, in line 47, col. 2 through line 27, col 3).
Re Claim 2, ARZOUAN as modified by GOULD, and Chen further disclose wherein a heatmap visual representation is generated upon determining that a likelihood of damage exceeds a threshold (see ARZOUAN: e.g., -- Features external to the first image and/or the second image may be excluded from the computation of the transformation. The transformation may be applied to the first image to generate a transformed first image depicting a transformed first candidate region of damage. A correlation may be computed between the second candidate region of damage and the transformed first candidate region of damage. Redundancy of the first candidate region of damage and the second candidate region of damage when the correlation is above a threshold. The threshold may indicate, for example, an amount of overlap of the second candidate region of damage and the transformed first candidate region of damage, at the common physical location. In another example, the threshold may indicate a selected likelihood (e.g., probability) corresponding to a value of the correlation. For example, a correlation of 0.85 may correspond to a probability of 85% of matching.
(65) Redundancy refers to the first candidate region of damage of the first image corresponding to a same physical location on the vehicle as the second candidate region of damage of the second image. Or in other words, that the same physical location on the vehicle is depicted in both the first image and the second image as the first and second candidate regions of damage.--, in lines 23-51, col. 15; and, -- (67) A predicted candidate region of damage may be computed. The predicted candidate region of damage may include a location of where the first candidate region of damage depicted in the first image is predicted to be located in the second image…. A correlation between the predicted candidate region of damage and the second candidate region of damage may be computed. Redundancy may be identified when a correlation between the first candidate region of damage and the second candidate region is above a threshold. The threshold may indicate, for example, an amount of overlap of the second candidate region of damage and the transformed first candidate region of damage, at the common physical location. In another example, the threshold may indicate a selected likelihood (e.g., probability) corresponding to a value of the correlation. For example, a correlation of 0.85 may correspond to a probability of 85% of matching.--, in line 64, col. 15 through line 26, col. 16; --iterating the identifying redundancy for identifying a plurality of single physical damage regions within a plurality of physical components of the vehicle, and generating a map of the plurality of physical components of the vehicle marked with respective location of each of the plurality of single physical damage regions.--, in lines 17-23, col. 2; also see CHEN: e.g., Fig. 17, “1702 Generate Heat map corresponding to vehicle image”; and,
-- The image processing system may next perform a statistical processing routine on each of the identified target object image components, such as a deep learning technique, like one that uses one or more convolutional neural networks, to detect changes, the likelihood of changes, and/or a quantification of an amount or type of change, to each such body component of the target object as depicted in the target object images. Such change detections may then be used for other purposes to, for example, estimate repair costs, estimate the amount of change that has occurred in the object, estimate the amount of time or effort needed to correct or fix the change, etc. Still further, in one case, the change determination may be displayed to a user in the form of a heat map that illustrates areas of the target object that have undergone change, the amount of such change, the type of such change, etc…. a method performed by an image processing system includes obtaining a source image that includes an image of a damaged vehicle, and applying, by a computing device included in the image processing system, an image processor component of the image processing system to the source image to thereby generate a set of values corresponding to damage to the vehicle. Each value included in the set of values respectively corresponds to respective damage at a respective location on a respective segment of the vehicle, and each value may be indicative of a determined degree of severity of damage to the respective location, a level of accuracy of the determined degree of severity of damage at the respective location, a type of damage at the respective location, and/or a likelihood of an occurrence of damage at the respective location. Additionally, the method includes determining, by the computing device from a set of vehicle parts of the vehicle, one or more parts needed to repair the vehicle. The determination of the one or more parts needed to repair the vehicle is based on one or more particular values included in the set of values generated by the image processor component. -, in line 47, col. 2 through line 27, col 3).
Re Claim 3, ARZOUAN as modified by GOULD, and Chen further disclose wherein extracted features including shape, intensity, and texture are used to calculate the likelihood of damage (see ARZOUAN: e.g.,: -- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13).
Re Claim 4, ARZOUAN as modified by GOULD, and Chen further disclose wherein the heatmap visual representation is provided with an overlay of damage likelihood scores (see ARZOUAN: e.g., -- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13; and, -- Features external to the first image and/or the second image may be excluded from the computation of the transformation. The transformation may be applied to the first image to generate a transformed first image depicting a transformed first candidate region of damage. A correlation may be computed between the second candidate region of damage and the transformed first candidate region of damage. Redundancy of the first candidate region of damage and the second candidate region of damage when the correlation is above a threshold. The threshold may indicate, for example, an amount of overlap of the second candidate region of damage and the transformed first candidate region of damage, at the common physical location. In another example, the threshold may indicate a selected likelihood (e.g., probability) corresponding to a value of the correlation. For example, a correlation of 0.85 may correspond to a probability of 85% of matching.
(65) Redundancy refers to the first candidate region of damage of the first image corresponding to a same physical location on the vehicle as the second candidate region of damage of the second image. Or in other words, that the same physical location on the vehicle is depicted in both the first image and the second image as the first and second candidate regions of damage.--, in lines 23-51, col. 15; and, -- (67) A predicted candidate region of damage may be computed. The predicted candidate region of damage may include a location of where the first candidate region of damage depicted in the first image is predicted to be located in the second image…. A correlation between the predicted candidate region of damage and the second candidate region of damage may be computed. Redundancy may be identified when a correlation between the first candidate region of damage and the second candidate region is above a threshold. The threshold may indicate, for example, an amount of overlap of the second candidate region of damage and the transformed first candidate region of damage, at the common physical location. In another example, the threshold may indicate a selected likelihood (e.g., probability) corresponding to a value of the correlation. For example, a correlation of 0.85 may correspond to a probability of 85% of matching.--, in line 64, col. 15 through line 26, col. 16; --iterating the identifying redundancy for identifying a plurality of single physical damage regions within a plurality of physical components of the vehicle, and generating a map of the plurality of physical components of the vehicle marked with respective location of each of the plurality of single physical damage regions.--, in lines 17-23, col. 2; also see CHEN: e.g., Fig. 17, “1702 Generate Heat map corresponding to vehicle image”; and,
-- The image processing system may next perform a statistical processing routine on each of the identified target object image components, such as a deep learning technique, like one that uses one or more convolutional neural networks, to detect changes, the likelihood of changes, and/or a quantification of an amount or type of change, to each such body component of the target object as depicted in the target object images. Such change detections may then be used for other purposes to, for example, estimate repair costs, estimate the amount of change that has occurred in the object, estimate the amount of time or effort needed to correct or fix the change, etc. Still further, in one case, the change determination may be displayed to a user in the form of a heat map that illustrates areas of the target object that have undergone change, the amount of such change, the type of such change, etc…. a method performed by an image processing system includes obtaining a source image that includes an image of a damaged vehicle, and applying, by a computing device included in the image processing system, an image processor component of the image processing system to the source image to thereby generate a set of values corresponding to damage to the vehicle. Each value included in the set of values respectively corresponds to respective damage at a respective location on a respective segment of the vehicle, and each value may be indicative of a determined degree of severity of damage to the respective location, a level of accuracy of the determined degree of severity of damage at the respective location, a type of damage at the respective location, and/or a likelihood of an occurrence of damage at the respective location. Additionally, the method includes determining, by the computing device from a set of vehicle parts of the vehicle, one or more parts needed to repair the vehicle. The determination of the one or more parts needed to repair the vehicle is based on one or more particular values included in the set of values generated by the image processor component. -, in line 47, col. 2 through line 27, col 3).
Re Claim 5, ARZOUAN as modified by GOULD, and Chen further disclose wherein images captured when illuminated by a plurality of uniform lighting pattern panels are used to generate a Multiview Interactive Digital Media Representation (MVIDMR) (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13; and, --(78) Alternatively or additionally, a map of the physical components of the vehicle marked with respective locations of each of the single physical damage regions may be generated. The map may be generated based on images depicting multiple different regions of the vehicle. The map may be presented within an interactive GUI, where a user may click on an indication of each physical damage region to obtain additional details, for example, recommendation, type of damage, estimated cost to fix, and the like.
(79) Alternatively or additionally, a user interface, optionally an interactive GUI, may be generated and/or presented on a display.--, in lines 56-67, col. 17).
Re Claim 6, ARZOUAN as modified by Chen further disclose wherein a plurality of MVIDMRs are generated for a plurality of different components of the vehicle (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13; and, --(78) Alternatively or additionally, a map of the physical components of the vehicle marked with respective locations of each of the single physical damage regions may be generated. The map may be generated based on images depicting multiple different regions of the vehicle. The map may be presented within an interactive GUI, where a user may click on an indication of each physical damage region to obtain additional details, for example, recommendation, type of damage, estimated cost to fix, and the like.
(79) Alternatively or additionally, a user interface, optionally an interactive GUI, may be generated and/or presented on a display.--, in lines 56-67, col. 17).
Re Claim 7, ARZOUAN as modified by GOULD, and Chen further disclose wherein each of the plurality of MVIDMRs is user navigable along at least two different axes (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13; and, --(78) Alternatively or additionally, a map of the physical components of the vehicle marked with respective locations of each of the single physical damage regions may be generated. The map may be generated based on images depicting multiple different regions of the vehicle. The map may be presented within an interactive GUI, where a user may click on an indication of each physical damage region to obtain additional details, for example, recommendation, type of damage, estimated cost to fix, and the like.
(79) Alternatively or additionally, a user interface, optionally an interactive GUI, may be generated and/or presented on a display.--, in lines 56-67, col. 17).
Re Claim 8, ARZOUAN as modified by GOULD, and Chen further disclose wherein a plurality of MVIDMRs are generated for a plurality of vehicle components including damaged components (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13; and, --(78) Alternatively or additionally, a map of the physical components of the vehicle marked with respective locations of each of the single physical damage regions may be generated. The map may be generated based on images depicting multiple different regions of the vehicle. The map may be presented within an interactive GUI, where a user may click on an indication of each physical damage region to obtain additional details, for example, recommendation, type of damage, estimated cost to fix, and the like.
(79) Alternatively or additionally, a user interface, optionally an interactive GUI, may be generated and/or presented on a display.--, in lines 56-67, col. 17).
Re Claim 9, ARZOUAN as modified by GOULD, and Chen further disclose wherein the damaged components are navigable along at least two different axes (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13; and, --(78) Alternatively or additionally, a map of the physical components of the vehicle marked with respective locations of each of the single physical damage regions may be generated. The map may be generated based on images depicting multiple different regions of the vehicle. The map may be presented within an interactive GUI, where a user may click on an indication of each physical damage region to obtain additional details, for example, recommendation, type of damage, estimated cost to fix, and the like.
(79) Alternatively or additionally, a user interface, optionally an interactive GUI, may be generated and/or presented on a display.--, in lines 56-67, col. 17).
Re Claim 10, ARZOUAN as modified by GOULD, and Chen further disclose wherein images captured by a plurality of uniform lighting pattern panels are used to detect damage to a first component of the vehicle (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13; and, --(78) Alternatively or additionally, a map of the physical components of the vehicle marked with respective locations of each of the single physical damage regions may be generated. The map may be generated based on images depicting multiple different regions of the vehicle. The map may be presented within an interactive GUI, where a user may click on an indication of each physical damage region to obtain additional details, for example, recommendation, type of damage, estimated cost to fix, and the like.
(79) Alternatively or additionally, a user interface, optionally an interactive GUI, may be generated and/or presented on a display.--, in lines 56-67, col. 17).
Re Claim 11, ARZOUAN as modified by GOULD, and Chen further disclose wherein images captured by the plurality of striped lighting pattern panels are used to analyze an extent of damage to the first component of the vehicle (see ARZOUAN: e.g., --(33) Image sensors 212 are arranged at different views, optionally with at least some overlap, for capturing images of different parts of the surface of the vehicle. Image sensors 212 may be, for example, standard visible light sensors (e.g., CCD, CMOS sensors, and/or red green blue (RGB) sensor). Computing device 204 receives sequences of time-spaced images captured by multiple image sensors 212 positioned in different views, for example cameras.
(34) Image sensors 212 may transmit captured images to computing device 204, for example, via a direct connected (e.g., local bus and/or cable connection and/or short range wireless connection), and/or via a network 210 and a network interface 222 of computing device 204 (e.g., where sensors are connected via a wireless network, internet of things (IoT) technology and/or are located remotely from the computing device).--, in lines 52-67, col. 10; and, -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; also see GOULD: e.g., -- a structured light source 22 arranged to direct structured light 24 at the vehicle pathway 14 for illuminating the vehicle 12 on the pathway 14 with a structured light image (not shown). In this embodiment the structured light source 22 extends up one side wall 16a from the vehicle pathway 14, across the roof section 16c and down the opposite side wall 16b, back to the vehicle pathway 14 to form an arch of structured lighting. This arrangement enables the structured light image to be projected onto both sides, and the roof, of the vehicle 12 as it passes the structured light source 22. The structured light source 22 is a light array having a set of LED strips arranged in parallel. The LED strips extend along each light array, from the bottom to the top and across the roof section. LEDs can for example comprise ultra-bright cool white LED tape, with a luminosity of 2880 lumens per meter. In one example a set of twenty LED strips can be arranged into 14.25mm wide grooves spaced 15.75mm apart and set 9mm deep with a 10mm backing behind them. Semi opaque diffusers (not shown) can be provided over each strip of LEDs to create a flat light from each strip of tape.
In other embodiment the structured light source 22 can have any suitable configuration arranged to project the structured light image onto one or more surfaces and in some cases all outer surfaces of the vehicle; for example, each light source can include a laser projector configured to project one or more light patterns.
The damage assessment zone 10 includes damage assessment cameras of a first type 26, namely high speed 'dent detecting' cameras arranged to image dents on the vehicle.
Multiple dent cameras 26 can be arranged inside the tunnel 2A, located on the side walls 16a, 16b and roof 16c to form an arch, as shown in Figure 3B, so that the sides and roof of the vehicle 12 can be simultaneously imaged.
Each dent detecting camera 26 is arranged with a field of view 26a comprising a structured light portion of the tunnel volume in which the structured light image will be reflected to be visible to the first camera 26 by a vehicle 12 moving along the vehicle pathway 14. Thus, the dent detecting cameras 26 are located in the tunnel 2A so that the dent camera field of view 26a overlaps with the striped pattern reflecting on the vehicle. The system can be calibrated for an average or expected vehicle profile for example.
Should the vehicle 12 have a dent in the bodywork, the striped reflections will distort around the dent, for example creating a circle like shape in the reflection. The images captured by the dent detection cameras 26 can then be used retrospectively to analyse whether a vehicle 12 has dents at a certain point in time. Thus, the field of view 26a of the dent detecting cameras 26 overlap the reflected striped image area 28 on the vehicle 12.
The structured light source can be in the form of panels mounted on the tunnel side walls and/or roof, either freestanding or mounted separately to the tunnel wall.--, in [0050]-[0056]).
Re Claim 12, ARZOUAN as modified by GOULD, and Chen further disclose wherein capture of additional images by the plurality of striped lighting pattern panels is triggered if damage is detected by images captured using the plurality of uniform lighting pattern panels (see GOULD: e.g., -- a structured light source 22 arranged to direct structured light 24 at the vehicle pathway 14 for illuminating the vehicle 12 on the pathway 14 with a structured light image (not shown). In this embodiment the structured light source 22 extends up one side wall 16a from the vehicle pathway 14, across the roof section 16c and down the opposite side wall 16b, back to the vehicle pathway 14 to form an arch of structured lighting. This arrangement enables the structured light image to be projected onto both sides, and the roof, of the vehicle 12 as it passes the structured light source 22. The structured light source 22 is a light array having a set of LED strips arranged in parallel. The LED strips extend along each light array, from the bottom to the top and across the roof section. LEDs can for example comprise ultra-bright cool white LED tape, with a luminosity of 2880 lumens per meter. In one example a set of twenty LED strips can be arranged into 14.25mm wide grooves spaced 15.75mm apart and set 9mm deep with a 10mm backing behind them. Semi opaque diffusers (not shown) can be provided over each strip of LEDs to create a flat light from each strip of tape.
In other embodiment the structured light source 22 can have any suitable configuration arranged to project the structured light image onto one or more surfaces and in some cases all outer surfaces of the vehicle; for example, each light source can include a laser projector configured to project one or more light patterns.
The damage assessment zone 10 includes damage assessment cameras of a first type 26, namely high speed 'dent detecting' cameras arranged to image dents on the vehicle.
Multiple dent cameras 26 can be arranged inside the tunnel 2A, located on the side walls 16a, 16b and roof 16c to form an arch, as shown in Figure 3B, so that the sides and roof of the vehicle 12 can be simultaneously imaged.
Each dent detecting camera 26 is arranged with a field of view 26a comprising a structured light portion of the tunnel volume in which the structured light image will be reflected to be visible to the first camera 26 by a vehicle 12 moving along the vehicle pathway 14. Thus, the dent detecting cameras 26 are located in the tunnel 2A so that the dent camera field of view 26a overlaps with the striped pattern reflecting on the vehicle. The system can be calibrated for an average or expected vehicle profile for example.
Should the vehicle 12 have a dent in the bodywork, the striped reflections will distort around the dent, for example creating a circle like shape in the reflection. The images captured by the dent detection cameras 26 can then be used retrospectively to analyse whether a vehicle 12 has dents at a certain point in time. Thus, the field of view 26a of the dent detecting cameras 26 overlap the reflected striped image area 28 on the vehicle 12.
The structured light source can be in the form of panels mounted on the tunnel side walls and/or roof, either freestanding or mounted separately to the tunnel wall.--, in [0050]-[0056]).
Re Claim 13, ARZOUAN as modified by GOULD, and Chen further disclose wherein a plurality of striped lighting pattern panels are striped lighting pattern filters (see GOULD: e.g., -- a structured light source 22 arranged to direct structured light 24 at the vehicle pathway 14 for illuminating the vehicle 12 on the pathway 14 with a structured light image (not shown). In this embodiment the structured light source 22 extends up one side wall 16a from the vehicle pathway 14, across the roof section 16c and down the opposite side wall 16b, back to the vehicle pathway 14 to form an arch of structured lighting. This arrangement enables the structured light image to be projected onto both sides, and the roof, of the vehicle 12 as it passes the structured light source 22. The structured light source 22 is a light array having a set of LED strips arranged in parallel. The LED strips extend along each light array, from the bottom to the top and across the roof section. LEDs can for example comprise ultra-bright cool white LED tape, with a luminosity of 2880 lumens per meter. In one example a set of twenty LED strips can be arranged into 14.25mm wide grooves spaced 15.75mm apart and set 9mm deep with a 10mm backing behind them. Semi opaque diffusers (not shown) can be provided over each strip of LEDs to create a flat light from each strip of tape.
In other embodiment the structured light source 22 can have any suitable configuration arranged to project the structured light image onto one or more surfaces and in some cases all outer surfaces of the vehicle; for example, each light source can include a laser projector configured to project one or more light patterns.
The damage assessment zone 10 includes damage assessment cameras of a first type 26, namely high speed 'dent detecting' cameras arranged to image dents on the vehicle.
Multiple dent cameras 26 can be arranged inside the tunnel 2A, located on the side walls 16a, 16b and roof 16c to form an arch, as shown in Figure 3B, so that the sides and roof of the vehicle 12 can be simultaneously imaged.
Each dent detecting camera 26 is arranged with a field of view 26a comprising a structured light portion of the tunnel volume in which the structured light image will be reflected to be visible to the first camera 26 by a vehicle 12 moving along the vehicle pathway 14. Thus, the dent detecting cameras 26 are located in the tunnel 2A so that the dent camera field of view 26a overlaps with the striped pattern reflecting on the vehicle. The system can be calibrated for an average or expected vehicle profile for example.
Should the vehicle 12 have a dent in the bodywork, the striped reflections will distort around the dent, for example creating a circle like shape in the reflection. The images captured by the dent detection cameras 26 can then be used retrospectively to analyse whether a vehicle 12 has dents at a certain point in time. Thus, the field of view 26a of the dent detecting cameras 26 overlap the reflected striped image area 28 on the vehicle 12.
The structured light source can be in the form of panels mounted on the tunnel side walls and/or roof, either freestanding or mounted separately to the tunnel wall.--, in [0050]-[0056]).
Re Claim 14, ARZOUAN as modified by GOULD, and Chen further disclose wherein a plurality of uniform lighting pattern panels are uniform lighting pattern diffusers (see GOULD: e.g., -- a structured light source 22 arranged to direct structured light 24 at the vehicle pathway 14 for illuminating the vehicle 12 on the pathway 14 with a structured light image (not shown). In this embodiment the structured light source 22 extends up one side wall 16a from the vehicle pathway 14, across the roof section 16c and down the opposite side wall 16b, back to the vehicle pathway 14 to form an arch of structured lighting. This arrangement enables the structured light image to be projected onto both sides, and the roof, of the vehicle 12 as it passes the structured light source 22. The structured light source 22 is a light array having a set of LED strips arranged in parallel. The LED strips extend along each light array, from the bottom to the top and across the roof section. LEDs can for example comprise ultra-bright cool white LED tape, with a luminosity of 2880 lumens per meter. In one example a set of twenty LED strips can be arranged into 14.25mm wide grooves spaced 15.75mm apart and set 9mm deep with a 10mm backing behind them. Semi opaque diffusers (not shown) can be provided over each strip of LEDs to create a flat light from each strip of tape.
In other embodiment the structured light source 22 can have any suitable configuration arranged to project the structured light image onto one or more surfaces and in some cases all outer surfaces of the vehicle; for example, each light source can include a laser projector configured to project one or more light patterns.
The damage assessment zone 10 includes damage assessment cameras of a first type 26, namely high speed 'dent detecting' cameras arranged to image dents on the vehicle.
Multiple dent cameras 26 can be arranged inside the tunnel 2A, located on the side walls 16a, 16b and roof 16c to form an arch, as shown in Figure 3B, so that the sides and roof of the vehicle 12 can be simultaneously imaged.
Each dent detecting camera 26 is arranged with a field of view 26a comprising a structured light portion of the tunnel volume in which the structured light image will be reflected to be visible to the first camera 26 by a vehicle 12 moving along the vehicle pathway 14. Thus, the dent detecting cameras 26 are located in the tunnel 2A so that the dent camera field of view 26a overlaps with the striped pattern reflecting on the vehicle. The system can be calibrated for an average or expected vehicle profile for example.
Should the vehicle 12 have a dent in the bodywork, the striped reflections will distort around the dent, for example creating a circle like shape in the reflection. The images captured by the dent detection cameras 26 can then be used retrospectively to analyse whether a vehicle 12 has dents at a certain point in time. Thus, the field of view 26a of the dent detecting cameras 26 overlap the reflected striped image area 28 on the vehicle 12.
The structured light source can be in the form of panels mounted on the tunnel side walls and/or roof, either freestanding or mounted separately to the tunnel wall.--, in [0050]-[0056]).
Re Claims 15-19, claims 15-19 are the corresponding system claim to claims 1-5, thus are rejected for the similar reasons as for claims 1-5 respectively. Furthermore, ARZOUAN as modified by GOULD, and Chen further disclose a system, comprising:
storage configured to maintain a first plurality of vehicle surface images associated with a vehicle, and an interface configured to receive a second plurality of vehicle surface images associated with the vehicle, and a processor configured to generate a heatmap visual representation associated with estimated damage to the associated vehicle object model component; and a display configured to show the heatmap visual representation to depict vehicle imperfections (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13).
Re Claim 20, claim 20 is the corresponding medium claim to claim 1, thus is rejected for the similar reasons as for claim 1 respectively. Furthermore, ARZOUAN as modified by GOULD, and Chen further disclose a non-transitory computer readable medium comprising: computer code for performing the method (see ARZOUAN: e.g., -- the vehicle is moving relative to the plurality of image sensors, and the spatiotemporal correlation includes correlating between different images of different image sensors captured at different points in time.--, in the last para. of col. 1; and, -- mapping the vehicle to one predefined 3D model of a plurality of predefined 3D models, wherein the plurality of components are defined on the 3D model, mapping the at least one detected region of damage to the plurality of components on the 3D model, and presenting, within the user interface, the 3D model with the at least one detected region depicted thereon.
(32) In a further implementation form of the first, second, and third aspects, further comprising: receiving via the user interface, instructions for rotating, displacement, and/or zoom in/out of the 3D model, and presenting the 3D model with implementation of the instructions.--, in lines 44-56, col. 4; and also see: -- Multiple time-spaced image sequences depicting a region of a vehicle are accessed. The multiple time-spaced image sequences are captured by multiple image sensors (e.g., cameras) positioned at multiple different views. Each image sensor may capture a sequence of images captured at different times, for example, frames captured at a defined frame rate. Multiple candidate regions of damage are identified in the time-spaced image sequences, for example, by feeding the images into a detector machine learning model trained to detect damage. It is undetermined whether the multiple candidate regions of damage represent different physical regions of damage, or correspond to the same physical region of damage.--, in line 65, col. 5 through line 16, col. 6; and see Fig. 2, and see: -- Exemplary architectures of machine learning model(s) may include, for example, one or more of: a detector architecture, a classifier architecture, and/or a pipeline combination of detector(s) and/or classifier(s), for example, statistical classifiers and/or other statistical models,…--, in lines 35-67, col. 10; further see: -- (50) The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.--, in line 65, col. 12 through line 7, col. 13; and,
-- (53) At 106, candidate regions of damage are identified in the time-spaced image sequences. One or more candidate regions of damage may be identified per image of the time-spaced image sequences.
(54) The candidate regions of damage may be identified by feeding each image into a machine learning model (e.g., detector) trained on a training dataset of sample images of region(s) of a body of a sample vehicle labelled with ground truth indicating candidate regions of damage and optionally including images without damage. The machine learning model may generate the candidate region of damage as an outcome, for example, an outline encompassing the damage (e.g., bounding box), a tag indicating presence of damage in the image, markings (e.g., overlay) of the identified damage, and the like. In another example, the candidate regions of damage may be identified by image processing code, for example, by shining structured light on the body of the vehicle, and extracting features from the image to identify disruption of the a pattern of the structured light on the body of the vehicle. The disruption of the patter of the structured light may indicate an aberration on the smooth surface of the body, such as scratch and/or dent, likely being damage.--, in lines 46-67, col. 13).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEI WEN YANG whose telephone number is (571)270-5670. The examiner can normally be reached on 8:00 - 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEI WEN YANG/Primary Examiner, Art Unit 2662
Prosecution Insights