DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-21 are pending for examination in the Application No. 18/653,799 filed May 2nd, 2024.
Response to Amendment
Applicant’s preliminary amendments filed May 20th, 2025, to the Drawings have been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on May 2nd, 2024, is in compliance with the provisions of 37 CFR 1.97. Accordingly, the IDS is being considered and attached by the examiner.
Claim Objections
Claims 1, 8, and 15 are objected to because of the following informalities failing to comply with 37 CFR 1.71(a) for "full, clear, concise, and exact terms" (see MPEP § 608.01(m)):
In the last limitations of claims 1, 8, and 15, the examiner respectfully suggests amending the phrase “each area of interest on the object based on the 3D contour of the area of interest” to recite “each area of interest on the object based on the respective 3D contour of the area of interest” or similar in order to clarify that the “3D contour” that is being based on to determine the location and absolute size of an “area of interest” is the “3D contour” respective to that “area of interest” and not just any one of an “area of interest” in the one or more areas of interest.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier, as explained in MPEP § 2181, subsection I (note that the list of generic placeholders below is not exhaustive, and other generic placeholders may invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph):
A. The Claim Limitation Uses the Term "Means" or "Step" or a Generic Placeholder (A Term That Is Simply A Substitute for "Means")
With respect to the first prong of this analysis, a claim element that does not include the term "means" or "step" triggers a rebuttable presumption that 35 U.S.C. 112(f) does not apply. When the claim limitation does not use the term "means," examiners should determine whether the presumption that 35 U.S.C. 112(f) does not apply is overcome. The presumption may be overcome if the claim limitation uses a generic placeholder (a term that is simply a substitute for the term "means"). The following is a list of non-structural generic placeholders that may invoke 35 U.S.C. 112(f): "mechanism for," "module for," "device for," "unit for," "component for," "element for," "member for," "apparatus for," "machine for," or "system for." Welker Bearing Co., v. PHD, Inc., 550 F.3d 1090, 1096, 89 USPQ2d 1289, 1293-94 (Fed. Cir. 2008); Mass. Inst. of Tech. v. Abacus Software, 462 F.3d 1344, 1354, 80 USPQ2d 1225, 1228 (Fed. Cir. 2006); Personalized Media, 161 F.3d at 704, 48 USPQ2d at 1886–87; Mas-Hamilton Group v. LaGard, Inc., 156 F.3d 1206, 1214-1215, 48 USPQ2d 1010, 1017 (Fed. Cir. 1998). Note that there is no fixed list of generic placeholders that always result in 35 U.S.C. 112(f) interpretation, and likewise there is no fixed list of words that always avoid 35 U.S.C. 112(f) interpretation. Every case will turn on its own unique set of facts.
Such claim limitation(s) is/are:
"at least one processing device configured to…" in claims 8-14 implemented on hardware disclosed in paras. [0027] (e.g., "The processing device 202 includes any suitable number(s) and type(s) of processors or other processing devices in any suitable arrangement. …").
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6-11, 13-18, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (Li-877; US 2019/0095877 A1, cited in Applicant’s IDS filed May 2nd, 2024) in view of Hever et al. (Hever; US 2025/0363819 A1).
Regarding claim 1, Li-877 discloses a method comprising:
obtaining, using at least one processing device (para(s). [0021], recite(s)
[0021] “Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.”
), multiple images of a three-dimensional (3D) object (para(s). [0029], recite(s)
[0029] “At 130, images 132.sub.i of a rental vehicle's exterior are received by a rental vehicle management application (also referred to herein as the “management application”). In one embodiment, such images may include frames from a video recording taken by a user walking around the rental vehicle. For example, as shown in FIG. 2, a customer may use a handheld device 210 to record a video of a rental vehicle 200 while walking around the vehicle 200 in a substantially circular path, with such a video providing a 360 degree view of the vehicle's exterior from the front, back, and sides of the vehicle 200. …”
, where the “video recording” is multiple images of a 3D object (e.g., “vehicle”));
generating, using the at least one processing device, a 3D representation of the object with absolute metrics based on the images (para(s). [0031] and [0039], recite(s)
[0031] “At 150, the management application determines sizes of the detected vehicle damage. Returning to the example above in which the machine learning model outputs bounding boxes identifying the locations of vehicle damage, the management application may determine the real-world sizes of those bounding boxes by converting the pixel height and width of the bounding boxes to real-world units (e.g., feet or meters) based on known dimensions of the rental vehicle's make, model, and year, or based on measurement directly from the images. …”
[0039] “In one embodiment, the management application 420 may use triangulation to generate a 3D model representing the rental vehicle and including any detecting vehicle damage. Triangulation works on the principle that a point's location in three-dimensional (3D) space can be recovered from images depicting that point from different angles. In one embodiment, the management application 420 may determine portions of frames of a video captured by the customer that overlap and recover the 3D locations of points in those overlapping portions. In particular, the management application 420 may compute features (e.g., color, shape, thickness, etc.) of each of the points in the video frames and determine matching points across video frames based on matching features of those points. In one embodiment, RANSAC (Random Sample Consensus) features may be computed. Having determined the location of a given point in at least three video frames, the management application 420 may then use triangulation to determine that point's location in 3D space. By repeating this process for multiple points, the management application 420 may generate a 3D point cloud. In one embodiment, the management application 420 may further add texture to the 3D point cloud by extracting the texture and color of each of the points and averaging over neighboring points.”
, where “generat[ing] a 3D model representing the rental vehicle including any detecting vehicle damage” is generating a 3D representation (e.g., “3D model”) of the object (e.g., “vehicle”) with at least absolute metrics (e.g., “real-world units (e.g., feet or meters)”));
detecting, using the at least one processing device, one or more areas of interest associated with the object based on the images (para(s). [0030], recite(s)
[0030] “At 140, the management application inputs some or all of the received images 132.sub.i into the trained machine learning model to detect vehicle damage in the input images. …Given the input images, the trained machine learning model outputs locations (e.g., in the form of bounding boxes) of identified vehicle damage and classifications of the same (e.g., as dents or scratches) in one embodiment.”
, where the “locations (e.g., in the form of bounding boxes) of identified vehicle damage” are one or more areas of interest);
identifying, using the at least one processing device, a 3D(para(s). [0039]—see citation in preceding limitation “generating, using the at least one processing device, a 3D representation of the object,…” above—, where the “features (e.g., color, shape, thickness, etc.)” are at least 3D features of each area of interest (e.g., the feature are points in “3D space”)); and
determining, using the at least one processing device, a location and an absolute size of each area of interest on the object based on the 3D(para(s). [0031]—see citation(s) above—, where the “locations” and “real-world sizes” of “vehicle damage” are locations and absolute sizes of areas of interest; wherein para(s). [0039]—see citation(s) above—further recite(s) the locations and absolute sizes are based on 3D features of the area of interest of the 3D representation of the object (e.g., “features of those points” in “3D space”)).
Where Li-877 does not specifically disclose
identifying, using the at least one processing device, a 3D contour of each area of interest, each 3D contour identifying the area of interest within the 3D representation of the object; and
determining, using the at least one processing device, a location and …size of each area of interest on the object based on the 3D contour of the area of interest and the 3D representation of the object;
Hever teaches in the same field of endeavor of identifying 3D features of an area of interest within a generated 3D representation of an object
identifying, using the at least one processing device, a 3D contour of each area of interest, each 3D contour identifying the area of interest within the 3D representation of the object (para(s). [0151-0152], [0176], and [0187], recite(s)
[0151] “Optionally, each of the time-spaced images of each sequence may be classified into a classification category. The classification category may correspond to a physical component of the vehicle, optionally according to regions which may be replaceable and/or fixable. Examples of classification categories include: front bumper, rear bumper, hood, doors, grill, roof, sunroof, windows, front wind shield, read wind shield, and trunk. Alternatively or additionally, the classification categories may include sub-regions of components. The sub-regions may be selected according to considerations for a recommendation of whether the component should be fixed or replaced. For example, a driver's side door may be divided into 4 quadrants. Damage to 2 or more quadrants may generate a recommendation to replace the door, rather than fixing damage to the 2 or more quadrants. The time-spaced images may be clustered into multiple clusters, where each cluster corresponds to one of the classification categories. One or more features described herein, such as identification of damage, performing spatiotemporal correlation and/or multi-level redundancy validation, identification of redundancy, detection of damage, and/or other features described herein, may be performed per cluster. Analyzing each cluster may improve the recommendation for whether the physical component corresponding to the cluster should be fixed or replaced.”
[0152] “The classification may be performed, for example, by a machine learning model (e.g., detector, classifier) training on a training dataset of image of different physical components labelled with a ground truth of the physical component, and/or by image processing code that analyses features of the image to determine the physical component (e.g., shape outline of the physical component, pattern of structured light indicating curvature of the surface of the physical component, and/or key features such as door handle or designs.”
[0176] “…The predefined marker may be a known physical features of the vehicle, for example, a door, a window, a bumper, a wheel, a door handle, gas tank cover, and the like. The predefined marker may be identified, for example, by feeding the image into a machine learning model trained to detect and/or segment the predefined marker (e.g., neural network, detector), by identifying known features of the marker (e.g., edge detection, intensity pattern, line patterns), and the like. The predefined marker(s) (or features extracted from the predefined marker(s) detected in a first image captured by a first image sensor depicting a first candidate region of damage, may be matched with at least one predefined marker detected in a second image captured by a second image sensor depicting a second candidate region of damage. The matching may be done in two dimensions. The matching may be done, for example, by registering the first image to the second image and registering the first marker to the second marker, by transforming both images to a common plane, and the like. The matching of the markers between the two images enables computing a mapping between the two candidate regions of damage of the two images. A three dimensional mapping may be computed between a first pose of the first sensor and a second pose of the second sensor. The 3D mapping may be computed according to the two dimensional location of the candidate region of damage and intrinsic information of the different image sensors. The 3D mapping may be used to identify redundancy by validating that the first candidate region of damage captured by the first image sensor is the same as the second candidate region of damage captured by the second image second.”
[0187] “An exemplary not necessarily limiting approaches for temporal correlation 108B is now described. The temporal correlation may be computed for a first image and a second image captured by a same image sensor at different times. …”
, where “detect[ing]… [a] region of damage” includes identifying “features of the marker” like through “edge detection” and/or “line patterns” by “3D mapping” is identifying a 3D contour (e.g., “edge”, “line pattern[s]”, and/or “shape outline” in “3D”) of each area of interest (e.g., “damage”) within the 3D representation of the object (e.g., “3D mapping”)); and
determining, using the at least one processing device, a location and …size of each area of interest on the object based on the 3D contour of the area of interest and the 3D representation of the object (para(s). [0151-0152] and [0176]—see citations immediately above—, where para(s). [0115], [0159], and [0214] further recite(s):
[0115] “An indication of the common physical location of the vehicle corresponding to the single physical damage region is provided, for example, presented on a display, optionally within a user interface such as a graphical user interface (GUI). The GUI may be designed to enable the user to interact with the image depicting the physical damage reason, for example, to obtain more information in response to selection of the damage on an image.”
[0159] “Additional exemplary details regarding the overlap used to compute the confidence score are now provided. The confidence score may be computed for a first candidate region of damage depicted in a first image according to overlap with a second candidate region(s) of damage depicted in a second image(s). The higher the amount of overlap between the first and second candidate regions of damage, the higher the likelihood that the first and second candidate regions of damage represent the same region of damage. Overlap may refer, for example, to one or more of: similarity between size of the first candidate region of the first image and the second candidate region of the second image, ratio of the first candidate region and the second candidate region, and a distance between a center of the first candidate region and the second candidate region. …”
[0214] “Optionally, the detected region(s) of damage are mapped to the components of the representation (e.g., 3D model, images). The representation (e.g., 3D model, images) with the detected region(s) depicted thereon may be presented within the UI, for example, visually indicated by boundaries, color coding, and the like.”
, where determining the mapping of the “damage” on a “3D model” of the object (e.g., vehicle) is determining at least a “location” and “size” of the areas of interest (e.g., “region(s) of damage”) on the object based on the 3D contour of the areas of interest and the 3D representation of the object (e.g., the “damage” identified via “3D mapping” its contour features like “edge”, “line pattern[s]”, and/or “shape outline” as recited previously in para(s). [0151-0152] and [0176] above)).
Since Hever teaches that the 3D features of vehicle damage on a 3D generated model of a vehicle include contours such as “edge”, “line pattern[s]”, and/or “shape outline” as detailed above, it would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Li-877 to incorporate identification of 3D features of each area of interest as 3D contours of each area of interest to improve identification of damage regions on the 3D object amongst a plurality of images overlapping in capturing one or more damage regions of the 3D object as taught by Hever above.
Regarding claim 2, Li-877 in view of Hever discloses the method of Claim 1, wherein Hever further teaches identifying the 3D contour of each area of interest comprises:
identifying 2D contours for each area of interest in the images (para(s). [0151-0152], [0176], and [0187]—see citations in claim 1 above—, where identifying the “feature[s]” of “damage” in images (e.g., a “first image” and “second image”) by “edge detection” and/or “line patterns” is identifying 2D (i.e., image) contours for each area of interest (e.g., damage) in the images (e.g., “a first image and a second image captured by a same image sensor at different times”));
converting the 2D contours into intermediate 3D contours (para(s). [0176]—see citation in claim 1 above—, where determining a “three dimensional [3D] mapping” of “candidate” 2D contours (e.g., “regions of damage”) is converting the 2D contours into intermediate 3D contours (i.e., “candidate” damage regions in “3D”)); and
aggregating the intermediate 3D contours for each area of interest to generate the 3D contour for the area of interest (para(s). [0176]—see citation in claim 1 above—, where para(s). [0195-0196] further recite(s):
[0195] “A baseline region represent one candidate region of damage in one of the time-spaced images corresponding to the physical location of the vehicle may be selected. Candidate regions of damage in other time-spaced images that correlate to the same physical location of the vehicle as the baseline region of damage may be ignored. The correlation may be spatiotemporal, for example, overlap in candidate regions of damage of at least a predefined threshold. Candidate regions of damage in other time-spaced images that do not correlate to the same physical location of the vehicle as the base line region of damage and are located in another physical location of the vehicle, may be labelled as actual regions of damage.”
[0196] “At 112, an indication of the common physical location(s) of the vehicle corresponding to the single physical damage region(s) is provided. The common physical location(s) of the vehicle corresponding to the single physical damage region(s) represent the set of actual damage regions (i.e., one or more), after redundancy depicted in the images is accounted for, for example, by being removed and/or ignored.”
, where filtering (e.g., “remov[ing] and/or ignor[ing]”) “redundan[t]” “candidate regions of damage” of one “region of damage” (i.e., aggregating “candidate regions of damage in other time-spaced images” correlating to a “same physical location” of a “region of damage” in a “baseline image” into a same “region of damage”) is aggregating intermediate 3D contours for each area of interest (i.e., each “candidate regions of damage” in all images) to generate the 3D contour for the area of interest (i.e., “3D mapping” of the “region of damage” for the area of interest as recited previously in [0176] above)).
Regarding claim 3, Li-877 in view of Hever discloses the method of Claim 2, wherein Hever further teaches identifying the 3D contour of each area of interest further comprises:
tracking each area of interest across the images to identify 2D contours that are associated with one another (para(s). [0113], recite(s)
[0113] “An aspect of some embodiments of the present invention relates to systems, methods, computing devices, and/or code instructions (e.g., stored on a data storage device and executable by one or more processors) for processing of multiple images for detection of damage on a vehicle, optionally for damage to a body of the vehicle such as the doors, hood, roof, bumper, and head/rear lights, for example, scratches and/or dents. …A spatiotemporal correlation is performed between the time-spaced image sequences. The spatiotemporal correlation includes a time correlation and a spatial correlation, which may be computed using different processing pipelines, optionally in parallel. The time correlation is performed between images (e.g., frames) captured by a same image sensor at different times. …Redundancy is identified for the candidate regions of damage corresponding to a common physical location of the vehicle denoting a single physical damage region.”
, where a “spatiotemporal correlation” is performed “between images (e.g., frames) captured by a same image sensor at different times” is tracking each area of interest (e.g., “damage region[s]”) across the images to perform the intended use/result of identifying 2D contours that are associated (e.g., “redundan[t]”) with one another).
Regarding claim 4, Li-877 in view of Hever discloses the method of Claim 3, wherein Li-877 further discloses:
the images comprise images in a video sequence (para(s). [0029]—see citation in claim 1 limitation “…multiple images of a three-dimensional (3D) object…” above—, where the “video recording” is a video sequence); and
tracking each area of interest across the images(para(s). [0039]—see citation in claim 1 limitation “generating…a 3D representation…” above—, where “determin[ing] portions of frames of a video… that overlap and recover the 3D locations of points in those overlapping portions” is tracking each area of interest (e.g., “portions” and/or “features”) across the images).
Where Li-877 does not specifically disclose
tracking each area of interest across the images comprises using temporal cohesion between the images to estimate a camera path over an image capture period during which the images are captured;
Hever further teaches
tracking each area of interest across the images comprises using temporal cohesion between the images to estimate a camera path over an image capture period during which the images are captured (para(s). [0113], recite(s)
[0113] “An aspect of some embodiments of the present invention relates to systems, methods, computing devices, and/or code instructions (e.g., stored on a data storage device and executable by one or more processors) for processing of multiple images for detection of damage on a vehicle, optionally for damage to a body of the vehicle such as the doors, hood, roof, bumper, and head/rear lights, for example, scratches and/or dents. …A spatiotemporal correlation is performed between the time-spaced image sequences. The spatiotemporal correlation includes a time correlation and a spatial correlation, which may be computed using different processing pipelines, optionally in parallel. The time correlation is performed between images (e.g., frames) captured by a same image sensor at different times. The spatial correlation is performed between images (e.g., frames) captured by different image sensors each set at a different view. Redundancy is identified for the candidate regions of damage corresponding to a common physical location of the vehicle denoting a single physical damage region.”
, where a “spatiotemporal correlation” is performed “between images (e.g., frames) captured by a same image sensor at different times” is tracking each area of interest (e.g., “damage region[s]”) across the images using temporal cohesion between the images to perform the intended use/result of estimating a camera path over an image capture period during which the images are captured).
Regarding claim 6, Li-877 in view of Hever discloses the method of Claim 1, wherein Li-877 further discloses at least one trained machine learning model is used to at least one of: generate the 3D representation of the object, detect the one or more areas of interest, or identify the 3D contour of each area of interest (para(s). [0031]—see citation in claim 1 limitation “generating… a 3D representation…” above—, where para(s). [0005] further recite(s):
[0005] “One embodiment includes a method for detecting vehicle damage. The method generally includes training a machine learning model to identify and classify damage to vehicles. The machine learning model is trained using, at least in part, one or more sets of images that each depicts a respective type of vehicle damage and a set of images that do not depict vehicle damage. The method further includes receiving one or more images which provide a 360 degree view of an exterior of a vehicle. In addition, the method includes determining damage to the vehicle as depicted in the received images using the trained machine learning model.”
, where the “trained machine learning model” is a trained machine learning model used to at least detect the one or more areas of interest (e.g., “locations of vehicle damage”)).
Regarding claim 7, Li-877 in view of Hever discloses the method of Claim 1, wherein Li-877 further discloses the method of Claim 1 further comprising:
generating at least one of a graphical user interface or a report that identifies the location and the absolute size of at least one of the one or more areas of interest (para(s). [0032] and [0041], recite(s)
[0032] “…That is, the customer may simply take a video with his or her handheld device while walking around the rental vehicle, and, in turn, the management application may identify and classify vehicle damage from the video and transmit a report and receipt back to the customer's handheld device indicating the damage and estimated cost of repairs, among other things.”
[0041] “Although discussed herein primarily with respect to the management application's 420 interactions with applications running in the customers' handheld devices 440.sub.i, it should be understood that the management application 420 may also provide a platform that other parties can interact with. For example, the management application 420 may also permit insurance carriers to log in and view vehicle damage reports and cost estimates, which may be similar to the reports transmitted to the customers' handheld devices 440.sub.i. As another example, the management application 420 may also permit insurance adjusters or rental car company employees, as opposed to customers themselves, to capture videos and/or images of vehicles that are transmitted and processed by management application 420. In such a case, the management application 420 may further provide a user interface (e.g., a web-based interface) that the insurance adjusters or rental car company employees can use to enter notes and/or other information that the management application 420 may incorporate into vehicle damage and cost estimate reports. As yet another example, the management application 420 may also permit contractors such as vehicle service centers to view information on vehicle damage that the contractors are asked to repair.”
).
Regarding claim 8, the claim recites similar limitations to claim 1 but in the form of an apparatus. Therefore, claim 8 recites similar limitations to claim 1 and is rejected for similar rationale and reasoning (see the analysis for claim 1 above).
Regarding claim 9, the claim recites similar limitations to claim 2 and is rejected for similar rationale and reasoning (see the analysis for claim 2 above).
Regarding claim 10, the claim recites similar limitations to claim 3 and is rejected for similar rationale and reasoning (see the analysis for claim 3 above).
Regarding claim 11, the claim recites similar limitations to claim 4 and is rejected for similar rationale and reasoning (see the analysis for claim 4 above).
Regarding claim 13, the claim recites similar limitations to claim 6 and is rejected for similar rationale and reasoning (see the analysis for claim 6 above).
Regarding claim 14, the claim recites similar limitations to claim 7 and is rejected for similar rationale and reasoning (see the analysis for claim 7 above).
Regarding claim 15, the claim recites similar limitations to claim 1 but in the form of a non-transitory machine readable medium. Therefore, claim 15 recites similar limitations to claim 1 and is rejected for similar rationale and reasoning (see the analysis for claim 1 above).
Regarding claim 16, the claim recites similar limitations to claim 2 and is rejected for similar rationale and reasoning (see the analysis for claim 2 above).
Regarding claim 17, the claim recites similar limitations to claim 3 and is rejected for similar rationale and reasoning (see the analysis for claim 3 above).
Regarding claim 18, the claim recites similar limitations to claim 4 and is rejected for similar rationale and reasoning (see the analysis for claim 4 above).
Regarding claim 20, the claim recites similar limitations to claim 6 and is rejected for similar rationale and reasoning (see the analysis for claim 6 above).
Regarding claim 21, the claim recites similar limitations to claim 7 and is rejected for similar rationale and reasoning (see the analysis for claim 7 above).
Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Hever as applied to claims 1, 8, and 15 above, and further in view of Li et al. (Li-793; US 2018/0260793 A1).
Regarding claim 5, Li-877 in view of Hever discloses the method of Claim 1, wherein Li-793 teaches in the same field of endeavor of generating a 3D representation of an object the method of Claim 1 further comprising:
determining if one or more camera parameters associated with the images are available (para(s). [0164] and [0290], recite(s)
[0164] “In some embodiments, in order to assist the adjustors to make decisions quickly and easily using the output of the disclosed automated system, damaged area in each input image are marked in a contrasting color. Also, a label can be put onto the damaged part. Some embodiments then project the images onto the 3D model of the vehicle using the camera angles determined during the alignment process. The 3D model then shows the damage to the vehicle in an integrated manner. The adjustor can rotate and zoom in on the 3D model as desired. When the adjustor clicks on a damaged part, the interface may show all the original images that contain that part on the side, so that the adjustor can easily examine in the original images where the damage was identified.”
[0290] “In the rendering/sampling step, a discrete set of renderings of a 3D car model is generated from various viewing points by adjusting four parameters: the distance to camera center, elevation angle, azimuth angle, and yaw angle. These parameters are shown in FIG. 41 for an example 3D car model. …”
, where adjusting camera parameters (e.g., “distance to camera center, elevation, angle, azimuth angle, and yaw angel”) to render 2D images into a “3D car model” is determining if one or more camera parameters associated with images are available); and
one of:
using the one or more camera parameters that are available to generate the 3D representation of the object (para(s). [0164] and [0290]—see citations in the current claim above—, where rendering a “3D car model” based on the camera “parameters” is using the one or more camera parameters that are available to generate the 3D representation of the object (e.g., “3D car model”)); or
estimating the one or more camera parameters based on the images and using the one or more estimated camera parameters to generate the 3D representation of the object (para(s). [0307] and [0310], recite(s)
[0307] “In the camera matrix step of pose refinement, embodiments of the disclosure calculate the camera matrix from the registered image boundaries. Each boundary point m.sub.i is associated with a 3D point M.sub.i on the 3D model, as {m.sub.i} is the 2D projection of {M.sub.i}. From last step, we know each m.sub.i has a corresponding v.sub.j on the boundary of the real vehicle. Using {m.sub.i} as a bridge, embodiments of the disclosure establish the 3D-to-2D correspondence between 3D points {M.sub.i} and the 2D points {v.sub.j}. The above correspondence can be utilized to estimate the camera matrix P as mentioned previously.”
[0310] “The linear solution of P obtained by DLT algorithm minimizes an algebraic error that is not geometrically meaningful. Therefore, embodiments of the disclosure can further approximate the optimal solution by minimizing the geometric error. Geometric error is also known as re-projection error, which is defined as the average distance between the re-projected points and the image points. In one implementation, embodiments of the disclosure are solving for a camera matrix P that minimizes the following nonlinear error function:”
, where “estimat[ing] the camera matrix” is estimating one or more camera parameters based on the images to use in generating the 3D representation of the object (e.g., “reproject[ed] points and the image points”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Li-877 in view of Hever to incorporate determining if one or more camera parameters are available, estimating the one or more camera parameters based on the images, and using the one or more estimated camera parameters in order to generate the 3D representation of the object using 2D images as taught by Li-793 above.
Regarding claim 12, the claim recites similar limitations to claim 5 and is rejected for similar rationale and reasoning (see the analysis for claim 5 above).
Regarding claim 19, the claim recites similar limitations to claim 5 and is rejected for similar rationale and reasoning (see the analysis for claim 5 above).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA Z YAO whose telephone number is (571)272-2870. The examiner can normally be reached Monday - Friday (8:30AM - 5PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.Z.Y./Examiner, Art Unit 2666
/MING Y HON/Primary Examiner, Art Unit 2666
Prosecution Insights