DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 5-6, 9-12, 15-16, and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li et al. (US 20180260793 A1).
Regarding claim 1, Li et al. teach A system ([0057] automatic vehicle damage assessment system) comprising:
an interface configured to receive visual data, the visual data associated with a plurality of images of a vehicle ([0057] capturing images of the damaged vehicle ) the plurality of images captured using an image capture device from a plurality of viewpoints ([0161] capture photos of eight views of the vehicle, for example: front-left perspective, front plan, front-right perspective, left plan, right plan, rear-left perspective, rear plan, rear-right perspective);
a processor configured to generate a user interface based on the visual data, the user interface including a multi-view interactive digital media representation (MVIDMR) of the vehicle, the MVIDMR navigable in multiple dimensions ([0159] the interface in FIG. 19A may be displayed. A 3d model of the user's vehicle is displayed and the user is prompted to tap on the portion of the vehicle that is damaged. For example, the user may tap on the hood of the vehicle, which causes an interface such as the one shown in FIG. 19B to be displayed. If the user selects “Yes” in the prompt in FIG. 19B, the interface in FIG. 19C may be displayed. In FIG. 19C, an outline 1902 is displayed for the hood of the vehicle superimposed on a live camera view from the client device ); and
an output interface configured to send the user interface to a display device configured to display the user interface, wherein selection of a designated portion of the MVIDMR causes the user interface to present a perspective view associated with the designated portion ([0159] A 3d model of the user's vehicle is displayed and the user is prompted to tap on the portion of the vehicle that is damaged. For example, the user may tap on the hood of the vehicle, which causes an interface such as the one shown in FIG. 19B to be displayed. If the user selects “Yes” in the prompt in FIG. 19B, the interface in FIG. 19C may be displayed. In FIG. 19C, an outline 1902 is displayed for the hood of the vehicle superimposed on a live camera view from the client device), wherein the perspective view corresponds to a two-dimensional top-down view of the vehicle (Fig. 28) presented with an overlaid heatmap illustrating damage to the vehicle ([0272] the localized damage on the 2D images can be shown, for example, by drawing a 2D heatmap overlaying the image to localize where on the image the damage is located), the overlaid heatmap being determined at least in part by analyzing the plurality of images ([0196] a vehicle is divided up into twenty-four (24) exterior parts, [0223] After all the images in a claim have been presented to the exterior damage detection engine 2506, the machine learning system has a prediction for damage to each of the exterior parts that we can infer from the collection of images for the claim , [0271] The result of performing step 2706 is a binary decision as to whether each of the 24 parts is damaged in a 2D image).
Regarding claim 2, Li et al teach the system of claim 1, wherein the perspective view is provided with a two- dimensional top-down view of the vehicle (Fig. 28).
Regarding claim 5, Li et al teach the system of claim l, wherein a plurality of designated portions correspond to a plurality of components of the vehicle ([0196]-[0219] a vehicle is divided up into twenty-four (24) exterior parts... Pr1=‘Front Bumper’; Pr2=‘Back Bumper’;..., Pr24=‘Trunk’).
Regarding claim 6, Li et al teach the system of claim 5, wherein a plurality of MVIDMRs are generated for the plurality of components of the vehicle (Fig. 28).
Regarding claim 9, Li et al teach the system of claim 1, wherein the user interface includes a damage detection panel corresponding to a damage component, wherein the damage detection panel includes a subset of the plurality of images of the damage component of the vehicle ([0005] performing computerized image processing based on the one or more images to generate one or more damage detection images, wherein each damage detection image is a two-dimensional (2D) image that includes indications of areas of damage to the vehicle in the damage detection image, [0223] After all the images in a claim have been presented to the exterior damage detection engine 2506, the machine learning system has a prediction for damage to each of the exterior parts that we can infer from the collection of images for the claim)
Regarding claim 10, Li et al teach the system of claim 1, wherein the MVIDMR is presented with an overlaid MVIDMR heatmap ([0272] the localized damage on the 2D images can be shown, for example, by drawing a 2D heatmap overlaying the image to localize where on the image the damage is located).
Regarding claim 11, Li et al teach A method comprising:
receiving visual data, the visual data associated with a plurality of images of a vehicle ([0057] capturing images of the damaged vehicle ) the plurality of images captured using an image capture device from a plurality of viewpoints ([0161] capture photos of eight views of the vehicle, for example: front-left perspective, front plan, front-right perspective, left plan, right plan, rear-left perspective, rear plan, rear-right perspective);
generating a user interface based on the visual data by using a processor, the user interface including a multi-view interactive digital media representation (MVIDMR) of the vehicle, the MVIDMR navigable in multiple dimensions ([0159] the interface in FIG. 19A may be displayed. A 3d model of the user's vehicle is displayed and the user is prompted to tap on the portion of the vehicle that is damaged. For example, the user may tap on the hood of the vehicle, which causes an interface such as the one shown in FIG. 19B to be displayed. If the user selects “Yes” in the prompt in FIG. 19B, the interface in FIG. 19C may be displayed. In FIG. 19C, an outline 1902 is displayed for the hood of the vehicle superimposed on a live camera view from the client device ); and sending the user interface to a display device configured to display the user interface, wherein selection of a designated portion of the MVIDMR causes the user interface to present a perspective view associated with the designated portion([0159] A 3d model of the user's vehicle is displayed and the user is prompted to tap on the portion of the vehicle that is damaged. For example, the user may tap on the hood of the vehicle, which causes an interface such as the one shown in FIG. 19B to be displayed. If the user selects “Yes” in the prompt in FIG. 19B, the interface in FIG. 19C may be displayed. In FIG. 19C, an outline 1902 is displayed for the hood of the vehicle superimposed on a live camera view from the client device), wherein the perspective view corresponds to a two-dimensional top-down view of the vehicle (Fig. 28) presented with an overlaid heatmap illustrating damage to the vehicle ([0272] the localized damage on the 2D images can be shown, for example, by drawing a 2D heatmap overlaying the image to localize where on the image the damage is located), the overlaid heatmap being determined at least in part by analyzing the plurality of images ([0196] a vehicle is divided up into twenty-four (24) exterior parts, [0223] After all the images in a claim have been presented to the exterior damage detection engine 2506, the machine learning system has a prediction for damage to each of the exterior parts that we can infer from the collection of images for the claim , [0271] The result of performing step 2706 is a binary decision as to whether each of the 24 parts is damaged in a 2D image).
Regarding claim 12, Li et al teach the method of claim 11, wherein the perspective view is provided with a two- dimensional top-down view of the vehicle (Fig. 28)..
Regarding claim 15, Li et al teach the method of claim 11, wherein a plurality of designated portions correspond to a plurality of components of the vehicle ([0196]-[0219] a vehicle is divided up into twenty-four (24) exterior parts... Pr1=‘Front Bumper’; Pr2=‘Back Bumper’;..., Pr24=‘Trunk’).
Regarding claim 16, Li et al teach the method of claim 15, wherein a plurality of MVIDMRs are generated for the plurality of components of the vehicle (Fig. 28)..
Regarding claim 19, Li et al teach the method of claim 11, wherein the user interface includes a damage detection panel corresponding to a damage component, wherein the damage detection panel includes a subset of the plurality of images of the damage component of the vehicle ([0005] performing computerized image processing based on the one or more images to generate one or more damage detection images, wherein each damage detection image is a two-dimensional (2D) image that includes indications of areas of damage to the vehicle in the damage detection image, [0223] After all the images in a claim have been presented to the exterior damage detection engine 2506, the machine learning system has a prediction for damage to each of the exterior parts that we can infer from the collection of images for the claim).
Regarding claim 20, Li et al teach a non-transitory computer readable medium ([0074] the software modules are contained in storage devices 208 or a combination of memory 204 and storage devices 208) comprising:
computer code for receiving visual data, the visual data associated with a plurality of images of a vehicle ([0057] capturing images of the damaged vehicle ) the plurality of images captured using an image capture device from a plurality of viewpoints ([0161] capture photos of eight views of the vehicle, for example: front-left perspective, front plan, front-right perspective, left plan, right plan, rear-left perspective, rear plan, rear-right perspective);
computer code for generating a user interface based on the visual data by using a processor, the user interface including a multi-view interactive digital media representation (MVIDMR) of the vehicle, the MVIDMR navigable in multiple dimensions ([0159] the interface in FIG. 19A may be displayed. A 3d model of the user's vehicle is displayed and the user is prompted to tap on the portion of the vehicle that is damaged. For example, the user may tap on the hood of the vehicle, which causes an interface such as the one shown in FIG. 19B to be displayed. If the user selects “Yes” in the prompt in FIG. 19B, the interface in FIG. 19C may be displayed. In FIG. 19C, an outline 1902 is displayed for the hood of the vehicle superimposed on a live camera view from the client device ); and
computer code for sending the user interface to a display device configured to display the user interface, wherein selection of a designated portion of the MVIDMR causes the user interface to present a perspective view associated with the designated portion([0159] A 3d model of the user's vehicle is displayed and the user is prompted to tap on the portion of the vehicle that is damaged. For example, the user may tap on the hood of the vehicle, which causes an interface such as the one shown in FIG. 19B to be displayed. If the user selects “Yes” in the prompt in FIG. 19B, the interface in FIG. 19C may be displayed. In FIG. 19C, an outline 1902 is displayed for the hood of the vehicle superimposed on a live camera view from the client device), wherein the perspective view corresponds to a two-dimensional top-down view of the vehicle (Fig. 28) presented with an overlaid heatmap illustrating damage to the vehicle ([0272] the localized damage on the 2D images can be shown, for example, by drawing a 2D heatmap overlaying the image to localize where on the image the damage is located), the overlaid heatmap being determined at least in part by analyzing the plurality of images ([0196] a vehicle is divided up into twenty-four (24) exterior parts, [0223] After all the images in a claim have been presented to the exterior damage detection engine 2506, the machine learning system has a prediction for damage to each of the exterior parts that we can infer from the collection of images for the claim , [0271] The result of performing step 2706 is a binary decision as to whether each of the 24 parts is damaged in a 2D image).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3-4 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in view of MAGNUSZEWSKI et al. (US 20200322545 A1).
Regarding claim 3, Li et al teach the system of claim 1, but do not teach
wherein the perspective view is provided with a two- dimensional undercarriage view of the vehicle.
In a similar endeavor, MAGNUSZEWSKI et al. teach
wherein the perspective view is provided with a two- dimensional undercarriage view of the vehicle (MAGNUSZEWSKI Fig. 34).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating MAGNUSZEWSKI et al. undercarriage imaging to arrive at the invention
The motivation of doing so would have detected damage for bottom parts of the vehicle.
Regarding claim 4, Li et al teach the system of claim 1, but do not teach
wherein the perspective view is provided with an undercarriage MVIDMR.
In a similar endeavor, MAGNUSZEWSKI et al. teach
wherein the perspective view is provided with an undercarriage MVIDMR (Fig. 34, [0175] user interface screen 980 for uploading a vehicle undercarriage composite image, according to an example embodiment of the present disclosure. For example, the user 108 may select the upload button illustrated in FIG. 34, causing the vehicle inspection application 612 to transfer the vehicle undercarriage composite image to a mobile device, data consumer devices 122, 124, or 126, or a server 112 for storage and further processing or distribution)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating MAGNUSZEWSKI et al. user interface to arrive at the invention
The motivation of doing so would have detected damage for bottom parts of the vehicle.
Regarding claim 13, Li et al teach the method of claim11, but do not teach
wherein the perspective view is provided with a two-dimensional undercarriage view of the vehicle.
In a similar endeavor, MAGNUSZEWSKI et al. teach
wherein the perspective view is provided with a two- dimensional undercarriage view of the vehicle (MAGNUSZEWSKI Fig. 34).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating MAGNUSZEWSKI et al. undercarriage imaging to arrive at the invention
The motivation of doing so would have detected damage for bottom parts of the vehicle.
Regarding claim 14, Li et al teach the method of claim 11, but do not teach
wherein the perspective view is provided with an undercarriage MVIDMR.
In a similar endeavor, MAGNUSZEWSKI et al. teach
wherein the perspective view is provided with an undercarriage MVIDMR (Fig. 34, [0175] user interface screen 980 for uploading a vehicle undercarriage composite image, according to an example embodiment of the present disclosure. For example, the user 108 may select the upload button illustrated in FIG. 34, causing the vehicle inspection application 612 to transfer the vehicle undercarriage composite image to a mobile device, data consumer devices 122, 124, or 126, or a server 112 for storage and further processing or distribution)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating MAGNUSZEWSKI et al. user interface to arrive at the invention
The motivation of doing so would have detected damage for bottom parts of the vehicle.
Claim 7-8 and 17-18 -are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in view of in view of Chen et al. (US 9886771 B1)
Regarding claim 7, Li et al teach the system of claim 1, but do not teach
wherein the overlaid heatmap shows a probability of damage to the designated portion using a color scale.
In a similar endeavor, Chen et al. teach
wherein the overlaid heatmap shows a probability of damage to the designated portion using a color scale (Chen Col 36 lines 4-9 each different color, color gradation, and/or other visual indicator may represent or be indicative of a respective amount of damage, a level of accuracy of an estimated or predicted amount of damage, a type of damage, and/or a probability or likelihood of an occurrence of damage at a respective location, point, or area of or on the vehicle
).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating Chen et al. heatmap to arrive at the invention
The motivation of doing so would have determined damage locations.
Regarding claim 8, Li et al teach the system of claim 1, but do not teach
wherein the overlaid heatmap shows a severity of damage to the designated portion using a color scale.
In a similar endeavor, Chen et al. teach
wherein the overlaid heatmap shows a severity of damage to the designated portion using a color scale Col 29 lines 32-37 using different colors or other visible indicia superimposed onto the image of the target vehicle (or the object model image of the target vehicle), to define or illustrate the presence, severity, type, and/or likelihood of damage at each component, pixel, surface segment, or other part of the damaged component of the target vehicle).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating Chen et al. heatmap to arrive at the invention .
The motivation of doing so would have determined the damage severity.
Regarding claim 17, Li et al teach the method of claim 11, but do not teach
wherein the overlaid heatmap shows a probability of damage to the designated portion using a color scale.
In a similar endeavor, Chen et al. teach
wherein the overlaid heatmap shows a probability of damage to the designated portion using a color scale (Chen Col 36 lines 4-9 each different color, color gradation, and/or other visual indicator may represent or be indicative of a respective amount of damage, a level of accuracy of an estimated or predicted amount of damage, a type of damage, and/or a probability or likelihood of an occurrence of damage at a respective location, point, or area of or on the vehicle).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating Chen et al. heatmap to arrive at the invention
The motivation of doing so would have determined damage locations.
Regarding claim 18, Li et al teach the method of claim l1, but do not teach
wherein the overlaid heatmap shows a severity of damage to the designated portion using a color scale.
In a similar endeavor, Chen et al. teach
wherein the overlaid heatmap shows a severity of damage to the designated portion using a color scale Col 29 lines 32-37 using different colors or other visible indicia superimposed onto the image of the target vehicle (or the object model image of the target vehicle), to define or illustrate the presence, severity, type, and/or likelihood of damage at each component, pixel, surface segment, or other part of the damaged component of the target vehicle).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the examined application to have modified Li et al. by incorporating Chen et al. heatmap to arrive at the invention .
The motivation of doing so would have determined the damage severity.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAID M ELNOUBI whose telephone number is (571)272-9732. The examiner can normally be reached Monday-Friday 9:30AM to 6:00PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kathy Wang-Hurst can be reached at 571-270-5371. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAID M ELNOUBI/Examiner, Art Unit 2644
