DETAILED ACTION
Acknowledgement
This non-final office action is in response to the request for continued examination (RCE) filed on 07/23/2026.
Status of Claims
Claims 2-4, 6, 16-18, and 23 have been cancelled.
Claims 1, 7-15, and 19-22 have been amended.
Claims 24-29 have been added.
Claims 1, 7-15, 19-22, and 24-29 are now pending.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/23/2026 has been entered.
Response to Arguments
Applicant's arguments filed on 07/23/2026 regarding the 35 U.S.C. 101 and 103 rejections of the claims have been fully considered. The Applicant argues the following:
(1) As per the 101 rejection, the Applicant argues, in summary, that (i) amended claim 1 does not recite commercial interaction; (ii) claim 1 integrates the evaluation (i.e. abstract idea) into a practical application. The claimed verification governs whether externally supplied data is admitted to generate the profile. The information obtained from video data controls whether user-supplied information is incorporated into project definition data; and (iii) claim 24 provides an additional and independent basis for eligibility because it recites a particular improvement in how a computer searches information derived from video data (i.e. parameters). This arrangement improves the computer-search process because computationally intensive video analysis is not part of the query-time comparison recited in claim 24. This is analogous to Enfish. The specification describes software analysis of video data to determine structured project parameters, generating and storing project profiles containing those parameters, receiving search parameters, and performing matching or searching based on parameters contained in the profile.
The Examiner respectfully disagrees. Although the Examiner agree that amended claim 1 no longer reflects a commercial interaction, the claims still recite mental processes. These claim limitations describe a process of receiving and verifying project data, determining project parameters, and generating a project profile, which can practically be performed mentally via observation and evaluation with pen and paper. Per MPEP 2106.04(a), a claim recites a judicial exception when the judicial exception is “set forth” or “described” in the claim.
The Examiner maintains the position that the additional elements recited in all the claims and listed in Steps 2A(2) and 2B do not integrate the abstract idea into a practical application nor provide significantly more because the claims do not improve the functioning of a computer nor improve upon another technology. The additional elements are viewed as mere instructions to apply or implement the abstract idea on a computer. The Applicant argues that the improvement in technology is in the “governing and controlling” of data and in the “search” process. The governing, controlling, and searching process is based on project parameters (i.e. data), and not directly tied to a specific implementation and/or improvement in technology. This is unlike Enfish where the invention improved the way the computer stores and retrieves data in memory in combination with the specific data structure recited in the claims that demonstrated eligibility. For these reasons, the 35 U.S.C. 101 rejection is maintained.
(2) As per the 103 rejections, the Applicant argues that (i) Nvs (2024/0428204 A1) does not appear to be citable since it has a filing date of June 11, 2024, and a publication date of December 26, 2024. Each of these dates is after the filing date of the present application; (ii) Roth does not teach verification-controlled profile generation. Nvs does not cure Roth's deficiencies; and (iii) Roth nor Nvs teach the limitations of the amended or newly added claims.
The Examiner somewhat disagrees with the Applicant’s arguments. The Examiner submits that based on the Applicant’s amended claim limitations, the Roth reference no longer teaches the limitations recited in independent claims 1, 19, and 20. Therefore, the previous 35 U.S.C. 103 rejections have been withdrawn. However, upon further search and consideration, a new ground of 103 rejection is made in light of the teachings of the prior arts of Nvs, Singh, Tsai, and Jakka. See details below.
The Examiner submits that the Nvs reference is a citable reference because it has an effective filing date of 10/26/2020. It is a continuation of parent application 17/080,563 which has the exact same specification. Nvs effective filing date of 10/26/2020 is before the Applicant’s filing date of 8/18/2023. Therefore, the citation and use of the Nvs reference for the 103 rejection is proper.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 7-15, 19-22, and 24-29 are rejected under 35 U.S.C. 101 because the claimed invention, “Video-Based Configuration of a Profile for a Project to be Performed at a Physical Location”, is directed to an abstract idea, specifically Mental Processes, without significantly more. The claims as a whole do not include additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the abstract idea because the additional elements individually or in combination provide mere instructions to implement the abstract idea on a computer.
Step 1: Claims 1, 7-15, 19-22, and 24-29 are directed to a statutory category, namely a process (claims 1, 7-15, 21-22, and 24-29), a machine (claim 19), and a manufacture (claim 20).
Step 2A (1): Independent claims 1, 19, and 20 are directed to an abstract idea of Certain Methods of Organizing Human Activity, based on the following claim limitations: “receiving/receive,…, video data defining a video purporting to include at least one scene of a work environment for a project to be performed at a physical location; receiving/receive, …, user input defining a first project parameter; obtaining/obtain, based on the video data, verification information corresponding to the first project parameter; determining/determine whether the first project parameter corresponds to the verification information; responsive to determining that the first project parameter corresponds to the verification information: including/include the first project parameter in project definition data; generating/generate a project profile based on the project definition data; and responsive to determining that the first project parameter does not correspond to the verification information, refraining/refrain from generating the project profile (claims 1, 19, and 20)”. These claim limitations describe a process of receiving and verifying project data, determining project parameters, and generating a project profile, which can practically be performed mentally via observation and evaluation with pen and paper. These are actions that can be performed by a human without a computer. Dependent claims 7-15, 21-22, and 24-29 further describe the determination of project parameters, verification information, and generation of a project profile. Therefore, these limitations, under the broadest reasonable interpretation, fall within the abstract grouping of Mental Processes which include concepts performed in the human mind such as observations, evaluations, judgments, and opinions. Mental Processes include claims directed to collecting information, analyzing it, and displaying certain results of the collection and analysis even if they are claimed as being performed on a computer. The courts have found claims requiring a generic computer or nominally reciting a generic computer may still recite a mental process even though the claim limitations are not performed entirely in the human mind. Therefore, claims 1, 7-15, 19-22, and 24-29 as a whole are directed to an abstract idea and are not patent eligible.
Step 2A (2): The claims as a whole do not integrate this abstract idea into a practical application. In particular, claims 1, 8-10, 12, 13, 19, 20-22, and 24-29 recite additional elements of “a computer-implemented method; video data; performed at a server; at the server from the remote computing device; (claim 1); video data (claim 8); software analysis of the video data; a speech-to-text module that performs speech recognition to obtain text data (claim 9); a machine-learning module trained (claim 10); software analysis of the video data comprises performing automated image analysis of one or more frames of the video (claims 12 and 21); wherein performing the automated image analysis of one or more frames of the video includes passing at least a portion of the video data to a machine-learning module trained (claim 13); A server comprising: a communications system; a memory; and a processor coupled to the communications system and the memory, the memory having stored thereon processor-executable instructions which, when executed, configure the processor to cause the server to; video data (claim 19); A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed, configure a server to; a remote computing device (claim 20); software analysis of the video data (claim 22); video data (claim 24); video data; performing automated image analysis of a plurality of frames of the video data (claims 25 and 28); causing the remote computing device to output (claims 26 and 27); passing an audio portion of the video data to a speech-to-text module that performs speech recognition to generate text data; a data-cleaning module; a machine-learning module (claim 29). These additional elements do not integrate the abstract idea into a practical application because the claims do not recite (a) an improvement to another technology or technical field and (b) an improvement to the functioning of the computer itself and (c) implementing the abstract idea with or by use of a particular machine, (d) effecting a particular transformation or reduction of an article, or (e) applying the judicial exception in some other meaningful way beyond generally linking the use of an abstract idea to a particular technological environment. These additional elements evaluated individually and in combination are viewed as computing and display devices that are used to perform the abstract process identified in Step 2A(1). Limitations that recite mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform an abstract idea are not indicative of integration into a practical application (see MPEP 2106.05(f)). Therefore, claims 1, 7-15, 19-22, and 24-29 as a whole do not include individual or a combination of additional elements that integrate the abstract idea into a practical application and thus are not patent eligible.
Step 2B: The claims as a whole do not include additional elements that are sufficient to amount to significantly more than the abstract idea. Claims 1, 8-10, 12, 13, 19, 20-22, and 24-29 recite additional elements of “a computer-implemented method; video data; performed at a server; at the server from the remote computing device; (claim 1); video data (claim 8); software analysis of the video data; a speech-to-text module that performs speech recognition to obtain text data (claim 9); a machine-learning module trained (claim 10); software analysis of the video data comprises performing automated image analysis of one or more frames of the video (claims 12 and 21); wherein performing the automated image analysis of one or more frames of the video includes passing at least a portion of the video data to a machine-learning module trained (claim 13); A server comprising: a communications system; a memory; and a processor coupled to the communications system and the memory, the memory having stored thereon processor-executable instructions which, when executed, configure the processor to cause the server to; video data (claim 19); A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed, configure a server to; a remote computing device (claim 20); software analysis of the video data (claim 22); video data (claim 24); video data; performing automated image analysis of a plurality of frames of the video data (claims 25 and 28); causing the remote computing device to output (claims 26 and 27); passing an audio portion of the video data to a speech-to-text module that performs speech recognition to generate text data; a data-cleaning module; a machine-learning module (claim 29)”. These additional elements evaluated individually and in combination are viewed as mere instructions to apply or implement the abstract idea on a computer. Applying an abstract idea on a computer does not integrate a judicial exception into a practical application or provide an inventive concept (see MPEP 2106.05(f)). Therefore, claims 1, 7-15, 19-22, and 24-29 as a whole do not include individual or a combination of additional elements that are sufficient to amount to significantly more than the abstract idea and thus are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 7, 19, 20, and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Nvs et al. (US 2024/0428204 A1) in view of Singh et al. (US 2023/0245686 A1).
As per claims 1, 19, and 20 (Currently Amended), Nvs teaches a computer-implemented method for controlling inclusion of user-supplied data in a project profile using video data, the method being performed at a server, the method comprising (Nvs e.g. A method and system are provided for assessing damage to a structure (Abstract).): a server comprising: a communications system; a memory; and a processor coupled to the communications system and the memory, the memory having stored thereon processor-executable instructions which, when executed, configure the processor to cause the server to: a non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed, configure a server to: (Nvs e.g. A system for assessing damage to a structure, the system comprising: a processor; and a memory in communication with the processor and comprising instructions which, when executed by the processor, program the processor to (claim 13).)
Nvs teaches receiving/receive, at the server and from a remote computing device, video data defining a video purporting to include at least one scene of a work environment for a project to be performed at a physical location; (Nvs e.g. Upon determining that the vehicle is repairable, the damage thereto must be assessed, so in the optional step 106 the adjuster sends one or more inspectors to obtain images of the vehicle, sometimes at the scene of the accident or elsewhere [0016]. FIG. 2B illustrates the timeline of another exemplary artificial intelligence (AI)/machine learning (ML) based process 250 for assessing damage to a vehicle and for processing vehicle insurance claims. In step 252, a customer captures a video of a vehicle involved in an accident and registers an FNOL, submitting therewith the captured video. A suitable device, such as a smartphone, tablet, digital camera, etc., may be used to capture the video, and it can be sent to the insurance company using an app supplied by the insurance company or using a third-party app [0021]. FIG. 3 is a flow chart of the overall process 300 of assessing damage to a structure (e.g., a vehicle involved in an accident), according to some embodiments [0024]. In step 302, the user uploads the recorded video (which may but need not include the associated audio), e.g., using a client mobile app, to a server that may be hosted in the cloud, where the content is analyzed using trained AI/ML models [0024])
Nvs teaches receiving/receive, at the server and from the remote computing device, user input defining a first project parameter; (Nvs e.g. When a vehicle accident occurs, the persons involved typically call the insurance carrier and file a First Notice of Loss (FNOL), which helps determine the extent of damage to the vehicle, and, subsequently, the liability and/or proper coverage for the required repairs. Using the information about the accident included in the FNOL, the adjuster may decide in step 104 whether the vehicle should be declared a total loss or may be repaired [0015]. In step 252, a customer captures a video of a vehicle involved in an accident and registers an FNOL, submitting therewith the captured video. A suitable device, such as a smartphone, tablet, digital camera, etc., may be used to capture the video, and it can be sent to the insurance company using an app supplied by the insurance company or using a third-party app [0021].)
Nvs teaches generating/generate a project profile based on the project definition data; and (Nvs e.g. FIG. 1 illustrates the timeline of a typical, conventional FNOL handling process 100 for processing insurance claims for damaged vehicles. In the process 100, an agent of the insurance carrier receives the FNOL and forwards it to an adjuster, at step 102. Using the information about the accident included in the FNOL, the adjuster may decide in step 104 whether the vehicle should be declared a total loss or may be repaired [0015]. FIG. 3 is a flow chart of the overall process 300 of assessing damage to a structure (e.g., a vehicle involved in an accident), according to some embodiments [0024]. At step 304, encoded information about the user/owner of the vehicle is obtained or retrieved from the uploaded information, and the user information is decoded so that an insurance claim can be processed. A user's personal information may include meta information about the vehicle such as make, model, year, body type (sedan, truck, hatchback, etc.), odometer reading, license plate no., etc., of the vehicle [0025].)
Nvs does not explicitly teach a verification process, however, Singh teaches the following:
Singh teaches obtaining/obtain, based on the video data, verification information corresponding to the first project parameter; (Sing e.g. According to various embodiments, the server system 202 includes a management web application 204, an input video upload and verification service 206, a remote computing device directory 208, a video rendering engine 210, an analytics service 212, and a communication interface 214 [0035].
Singh teaches determining/determine whether the first project parameter corresponds to the verification information; (Singh e.g. In some implementations, the input video upload and verification service 206 may be configured to receive input videos from remote computing devices, store the videos for later retrieval, and make those videos available for inclusion in an aggregate video. In addition, the input video upload and verification service 206 may be further operable to perform one or more verification tasks for uploaded videos. Such tasks may include analyzing a video to determine whether it is valid according to one or more criteria for input videos. Such criteria may be related to characteristics such as video length, resolution, quality, relevance, or other such features, and are discussed in additional detail with respect to the method 500 shown in FIG. 5 [0037].)
Singh teaches responsive to determining that the first project parameter corresponds to the verification information: including/include the first project parameter in project definition data; (Singh e.g. The recorded input video is uploaded at 410. In some implementations, the video may be uploaded to the video upload and verification service 206 shown in FIG. 2. In particular embodiments, the video upload and verification service 206 may provide a response message indicating that the uploaded video fails to meet one or more criteria associated with valid responses to the input video request. In this case, operations 408 and 410 may be repeated until a valid video is created and uploaded [0073].)
Singh teaches responsive to determining that the first project parameter does not correspond to the verification information, refraining/refrain from generating the project profile (Singh e.g. A response message is transmitted at 412. In some implementations, the response message may include information such as whether the request received at 402 has been accepted or rejected, whether a valid video has been uploaded at 410, and/or any other supporting information. Supporting information may include, for example, the identity of one or more individuals featured within the video, a time and/or location at which the video was captured, and/or whether to include a video or audio component of the video within the aggregate video [0074].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Nvs’s AI/ML Video Analysis process with Singh verification process in order to determine whether the video content is valid according to one or more criteria (Singh e.g. [0037]).
As per claim 7 (Currently Amended), Nvs in view of Singh teach the computer-implemented method of claim 1, Nvs does not explicitly teach, however, Singh teaches wherein obtaining the verification information includes obtaining the verification information from metadata included in the video data (Singh e.g. In particular embodiments, rendering the aggregate video may involve generate metadata information such as time codes that uniquely identify portions of the video. Such portions may correspond with the input video clips [0091].).
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Nvs’s AI/ML Video Analysis process with Singh verification process in order to determine whether the video content is valid according to one or more criteria (Singh e.g. [0037]).
As per claim 26 (New), Nvs in view of Singh teach the computer-implemented method of claim 1, further comprising, Nvs does not explicitly teach, however, Singh teaches responsive to determining that the first project parameter does not correspond to the verification information, causing the remote computing device to output an error message indicating that the project will not be accepted (Singh e.g. A response message is transmitted at 412. In some implementations, the response message may include information such as whether the request received at 402 has been accepted or rejected, whether a valid video has been uploaded at 410, and/or any other supporting information. Supporting information may include, for example, the identity of one or more individuals featured within the video, a time and/or location at which the video was captured, and/or whether to include a video or audio component of the video within the aggregate video [0074]. A determination is made at 312 as to whether the received input video is valid. In some implementations, the determination may be made by evaluating the input video against one or more quality criteria [0058]. If it is determined that the received input video is not valid, then it is rejected at 314. In some implementations, rejecting an input video may involve sending a rejection message to the remote computing device that generated the input video. The rejection message may identify one or more reasons that the input video was rejected. In such a situation, the recipient may elect to create and submit a replacement input video [0059].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Nvs’s AI/ML Video Analysis process with Singh verification process in order to determine whether the video content is valid according to one or more criteria (Singh e.g. [0037]).
Claims 8-15, 22, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Nvs et al. (US 2024/0428204 A1) in view of Singh et al. (US 2023/0245686 A1) and in further view of Tsai et al. (US 2012/0078626 A1).
As per claim 8 (Currently Amended) Nvs in view of Singh teach the computer-implemented method of claim 7, wherein
Nvs teaches the first project parameter comprises a user-supplied location parameter representing the physical location at which the project is to be performed; (Nvs e.g. When a vehicle accident occurs, the persons involved typically call the insurance carrier and file a First Notice of Loss (FNOL), which helps determine the extent of damage to the vehicle, and, subsequently, the liability and/or proper coverage for the required repairs. Using the information about the accident included in the FNOL, the adjuster may decide in step 104 whether the vehicle should be declared a total loss or may be repaired [0015]. Upon determining that the vehicle is repairable, the damage thereto must be assessed, so in the optional step 106 the adjuster sends one or more inspectors to obtain images of the vehicle, sometimes at the scene of the accident or elsewhere [0016]. Some embodiments can thus automate the process of estimating vehicle damage and repair costs from a video of the vehicle, such as those taken at the site of the accident [0017].)
Although Nvs in view of Singh teaches a verification process and information, Singh does not explicitly teach, however, Tsai teaches:
Tsai teaches the verification information comprises geospatial information included in the metadata and representing a location at which the video data was obtained; and (Tsai e.g. The present disclosure relates to methods and systems for receipt, processing, and delivery of multimedia content, as well as enrichment of multimedia content for enhanced search and delivery [0041]. In the context of the present disclosure, multimedia content can include any type of content containing, for example, one or more of images, video, audio, or a combination thereof [0042]. The video conversion module 610 converts the received multimedia content from a format in which it is received from a content provider into a format useable with the metadata generated by the audio processing module 606 and the video processing module 608 [0098]. The metadata from the audio processing module 606 and video processing module 608 is passed to a database 626, which collects metadata and other information derived from the multimedia content. In the embodiment shown, the database receives click through events 628, a full text search database 630, video 632, video metadata 634, and position metadata 636 based on processing of content [0100]. Video metadata 634 includes any of the data describing the video that can be tracked. For example, the video metadata 632 can include information about objects of interest defined as associated with multimedia content [0103]. The position metadata 636 defines the position or location of one or more objects in the video content. The position metadata 636 can take any of a number of forms. In certain embodiments, the position metadata 636 corresponds to GPS metadata associated with one or more pieces of content. In other embodiments, the position metadata 636 can be captured from some other type of position sensor, such as a location sensor or radio frequency identification (RFID) tag [0104].)
Tsai teaches determining whether the first project parameter corresponds to the verification information comprises determining whether the physical location represented by the user- supplied location parameter and the location represented by the geospatial information are within a defined proximity of one another. (Tsai e.g. FIG. 15 is a flowchart of methods and systems 1500 for detecting positions of objects within a scene in video content, according to a possible embodiment of the present disclosure [0196]. The positions captured by the capture position operation 1508 can be, in various embodiments, absolute positions (e.g. determined by GPS sensor positioning) or positions relative to a camera capturing video of those objects. By concurrently capturing both multimedia content and position information, the content and positions of objects within the content can be synchronized [0197]. An object identification operation 1512 then associates the objects detected in the multimedia content with the positional information received in the capture position operation 1508. An optional user confirmation operation 1514 receives an indication from the user confirming that the objects of interest are in fact those associated with the positional information [0198]. A location extrapolation operation 1516 extrapolates locations of the objects of interest within the multimedia content based on the relative positions of the objects as known from the positional information [0199].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Nvs in view of Singh’s AI/ML Video Analysis and Verification Process with Tsai’s system for capturing and processing video metadata in order to provide an easy way to identify objects appearing in the content such that a user can individually search for and identify those objects (Tsai e.g. [0005]).
As per claim 9 (Currently Amended), Nvs in view of Singh and Tsai teach the computer-implemented method of claim 24, Nvs teaches wherein determining the respective second project parameter of the project profile through software analysis of the video data comprises: passing an audio portion of the video data to a speech-to-text module that performs speech recognition to generate text data; and obtaining the respective second project parameter of the project profile from the text data. (Nvs e.g. This disclosure generally relates to artificial intelligence (AI) based/machine learning (ML) techniques and, in particular, to training and use of AI/ML systems to: determine damage to physical structures and estimate the effort required to repair the damage by analyzing videos of the physical structures [0001]. Some embodiments of a technique described herein feature artificial intelligence/machine learning enabled computer vision and analysis, along with optional speech analysis, so that different parts and/or components of a physical structure are recognized from a video of the structure [0017]. FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 602, the noise suppressed audio content 604 is received from the pre-processing stage (e.g., the audio content 412 provided by the pre-processing module 408 shown in FIG. 4). In step 602, the audio content 604 is converted to text using a speech-to-text engine [0052].)
As per claim 10 (Currently Amended), Nvs in view of Singh and Tsai teach the computer-implemented method of claim 9, Nvs teaches wherein obtaining the respective second project parameter of the project profile from the text data comprises passing at least a portion of the text data to a machine-learning module trained to identify the respective second project parameter of the project profile. (Nvs e.g. Some embodiments of a technique described herein feature artificial intelligence/machine learning enabled computer vision and analysis, along with optional speech analysis, so that different parts and/or components of a physical structure are recognized from a video of the structure [0017]. FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 606, the generated text is contextually parsed using computation linguistic techniques and stored for every specified time window (e.g., 5, 10, 12 seconds, etc.) of the corresponding extracted video content (e.g., the frames 410 shown in FIG. 4). Based on the contextual content parsing, various keywords and/or phrases in the speech that may be relevant to the detection of parts and/or damage are identified in the step 606 [0052]. In step 608, the identified keywords and/or phrases are matched with a parts dictionary, to detect vehicle parts (structure parts, in general) such as door, headlight, etc., that are described in the speech. In step 610, the identified keywords and/or phrases are matched with a damage-types dictionary, to detect the different types of damages, such as dents, scratches, missing parts, etc., that are described in the speech [0053].)
As per claim 11 (Currently Amended), Nvs in view of Singh and Tsai teach the computer-implemented method of claim 24, Nvs teaches wherein the respective second project parameter of the project profile comprises at least one of: a category parameter defining a category of the project; a time parameter defining one or more timing preferences for the project; a material parameter defining a material to be used in performing the project; a tool parameter defining a tool to be used in performing the project; a size parameter for the project; and a location parameter defining a location at which the project is to be performed (Nvs e.g. FIG. 2A illustrates the time line of an exemplary artificial intelligence (AI)/machine learning (ML) based process 200 for assessing damage to a vehicle and for processing vehicle insurance claims [0018]. In step 208, a video of the vehicle (structure, in general) taken by the inspector or the user/owner of the vehicle is received. In step 210, an AI/ML system analyzes the video received from the inspector/customer and determines whether the vehicle is a total loss or is repairable [0018]. In step 212, the AI/ML system analyzes the information about the parts detected to be damaged and the types of damages, and may estimate damage to any internal parts. Using this information, the AI/ML system then generates a list of repairs and replacements that are likely needed and predicts the required time and/or cost for the repairs and/or replacements. The damage analysis and the repair/replacement estimates generated by the AI/ML system are presented in an estimation platform [0019].)
As per claim 12 (Currently Amended), Nvs in view of Singh and Tsai teach the computer-implemented method of claim 24, wherein determining the respective second project parameter of the project profile through software analysis of the video data comprises performing automated image analysis of one or more frames of the video. (Nvs e.g. Some embodiments of a technique described herein feature artificial intelligence/machine learning enabled computer vision and analysis, along with optional speech analysis, so that different parts and/or components of a physical structure are recognized from a video of the structure. The video includes a visual component and an optional audio component. Any damaged parts and/or components of the structure may be identified, and the severity of the damage can be assessed automatically [0017]. FIG. 4 is a block diagram of a systems 400 for assessing damage to a structure using video of the structure, according to some embodiments. In the system 400, the damage assessment includes two parallel analyses, one of visual content provided in video content that includes several image frames, and may include one or more images, of the structure and the other of the associated audio content [0027]. The video 402 uploaded by the user (as described above with reference to FIG. 3 ) includes image frames 404 and audio content 406 [0029]. After the pre-processing, an AI/ML model is used by the visual assessment module 414. The visual assessment module 414 may include one or more processors that are configured using the AI/ML model (e.g., a deep learning model), where the model is trained to detect damaged structure parts from the selected video frames 410, and to assess the scope of damage to such part(s) [0032].)
As per claim 13 (Currently Amended), Nvs in view of Singh and Tsai teach the computer-implemented method of claim 12, Nvs teaches wherein performing the automated image analysis of one or more frames of the video includes passing at least a portion of the video data to a machine-learning module trained to identify the respective second project parameter of the project profile (Nvs e.g. Some embodiments of a technique described herein feature artificial intelligence/machine learning enabled computer vision and analysis, along with optional speech analysis, so that different parts and/or components of a physical structure are recognized from a video of the structure. The video includes a visual component and an optional audio component. Any damaged parts and/or components of the structure may be identified, and the severity of the damage can be assessed automatically [0017]. The models may be trained to perform deep learning based computer vision and speech analytics, to detect automatically any damaged part(s) and the damage to such parts, and to assess the severity of the damage [0024].).
As per claim 14 (Currently Amended), Nvs in view of Singh and Tsai teach the computer-implemented method of claim 13, Nvs teaches wherein the respective second project parameter of the project profile comprises one or more size parameters determined based on the one or more frames of the video (Nvs e.g. Any damaged parts and/or components of the structure may be identified, and the severity of the damage can be assessed automatically [0017]. FIG. 2A illustrates the timeline of an exemplary artificial intelligence (AI)/machine learning (ML) based process 200 for assessing damage to a vehicle and for processing vehicle insurance claims [0018]. In step 208, a video of the vehicle (structure, in general) taken by the inspector or the user/owner of the vehicle is received. In step 210, an AI/ML system analyzes the video received from the inspector/customer and determines whether the vehicle is a total loss or is repairable [0018]. In step 212, the AI/ML system analyzes the information about the parts detected to be damaged and the types of damages, and may estimate damage to any internal parts. Using this information, the AI/ML system then generates a list of repairs and replacements that are likely needed and predicts the required time and/or cost for the repairs and/or replacements (i.e. size of project). The damage analysis and the repair/replacement estimates generated by the AI/ML system are presented in an estimation platform [0019]. The models may be trained to perform deep learning based computer vision and speech analytics, to detect automatically any damaged part(s) and the damage to such parts, and to assess the severity of the damage [0024].).
As per claim 15 (Currently Amended), Nvs in view of Singh and Tsai teach The computer-implemented method of claim 14, Nvs teaches further comprising: determining, based on the one or more size parameters, a quantity of a material to be used in performing the project; and including the quantity of the material in the project profile. (Nvs e.g. FIG. 2A illustrates the timeline of an exemplary artificial intelligence (AI)/machine learning (ML) based process 200 for assessing damage to a vehicle and for processing vehicle insurance claims [0018]. In step 208, a video of the vehicle (structure, in general) taken by the inspector or the user/owner of the vehicle is received. In step 210, an AI/ML system analyzes the video received from the inspector/customer and determines whether the vehicle is a total loss or is repairable [0018]. In step 212, the AI/ML system analyzes the information about the parts detected to be damaged and the types of damages, and may estimate damage to any internal parts. Using this information, the AI/ML system then generates a list of repairs and replacements that are likely needed and predicts the required time and/or cost for the repairs and/or replacements. The damage analysis and the repair/replacement estimates generated by the AI/ML system are presented in an estimation platform [0019].)
As per claim 22 (Currently Amended), Nvs in view of Singh and Tsai teach the method of claim 24, wherein determining the respective second project parameter of the project profile through software analysis of the video data comprises determining a category parameter defining a category of the project. (Nvs e.g. FIG. 5 is a flow chart of a process 500 for assessing damage to a structure using visual content extracted from a video of the structure, according to various embodiments [0035]. The part identification information provided by the step 510 and the damage classification provided by the step 512 are provided together as damage information 512. Thus, the damage information 512 indicates the part(s) that are damaged and the nature/severity of the damage to each damaged part. For a vehicle, for example, the parts may include fenders, bumpers, hood, grill, rear gate, doors, side mirrors, wheels, etc. [0048]. One benefit of the technique described herein is that the part detection and identification (also referred to as part classification) and/or damage detection are performed using not just one image but using several relevant frames 506. Some of these frames can yield a more accurate determination than other frames. As such, in some embodiments, a confidence score is provided for each type of estimation, i.e., part detection and damage detection. A final inference of a damaged part and the type/severity of the damage may include a weighted average of the respective inferences derived from several frames [0049]. FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 608, the identified keywords and/or phrases are matched with a parts dictionary, to detect vehicle parts (structure parts, in general) such as door, headlight, etc., that are described in the speech. In step 610, the identified keywords and/or phrases are matched with a damage-types dictionary, to detect the different types of damages, such as dents, scratches, missing parts, etc., that are described in the speech [0053].)
As per claim 24 (New), Nvs in view of Singh teach the computer-implemented method of claim 1, Nvs teaches wherein each of a plurality of project profiles, including the project profile, comprises a respective second project parameter determined through software analysis of respective video data, and wherein the method further comprises (Nvs e.g. Some embodiments of a technique described herein feature artificial intelligence/machine learning enabled computer vision and analysis, along with optional speech analysis, so that different parts and/or components of a physical structure are recognized from a video of the structure [0017].) :
Nvs teaches storing the respective second project parameters in association with the plurality of project profiles; (Nvs e.g. The visual and audio assessments may be consolidated, and the costs for repair/replacement of the damaged parts and the total cost of vehicle repair are estimated in step 310. A human review may approve the AI/ML generated assessment in step 312 and, in step 314 the claim may be approved for processing and the case may be closed [0026]. The final output from the visual assessment process 500 can provide an overall damage assessment for the vehicle (a structure, in general), with a part level granularity [0050]. In some embodiments, feedback on the damage and/or cost estimates is sent to the user, along with portions of the video used for the assessment [0064].)
Nvs nor Singh explicitly teach, however Tsai teaches the following:
Tsai teaches after storing the respective second project parameters, receiving a search parameter corresponding to the second project parameters; and (Tsai e.g. The present disclosure relates to methods and systems for receipt, processing, and delivery of multimedia content, as well as enrichment of multimedia content for enhanced search and delivery [0041]. The system 400 illustrates components for access by a content provider to upload content to a multimedia processing system, and for handling search and content request messages received from content consumers [0057]. The request handler 414 can also receive search queries relating to the metadata stored in the grid 408, for example from content consumers seeking a particular piece of multimedia content, or seeking a list of pieces of multimedia content in which the search criteria is found [0070]. Data exchanged can include, both top level metadata describing the content and other types of metadata describing enhancements to the content. Top level metadata associated with the content can include, for example, an identifier of the content or its provider; a description of the content as a whole; the runtime of the content; the size of the content file; and other assorted data. Metadata associated with content enhancements can include for example, the content provider's identifier, the resource used, the name of the multimedia processing project, an identifier for the multimedia content, the size or type of the content, the size or number of objects of interest included in the content; the size or number of speech to text elements associated with the content, as well as the speech to text elements themselves; any advertisements or promotions to be included alongside the content; and keywords associated with the content [0071]. The metadata from the audio processing module 606 and video processing module 608 is passed to a database 626, which collects metadata and other information derived from the multimedia content. In the embodiment shown, the database receives click through events 628, a full text search database 630, video 632, video metadata 634, and position metadata 636 based on processing of content [0100].)
Tsai teaches searching the plurality of project profiles by comparing the search parameter with the stored respective second project parameters to identify at least one of the plurality of project profiles. (Tsai e.g. The requests from the content consumer 106 typically take one of a number of forms, such as a search request, a request for playback of content, or other requests related to the multimedia content, such as relating to the particular subjects or objects of interest displayed in the content [0048]. In the embodiment shown, a request handler 414 can be interfaced to the grid 408, and is configured to receive requests for data stored in the grid. The request handler 414 can also receive search queries relating to the metadata stored in the grid 408, for example from content consumers seeking a particular piece of multimedia content, or seeking a list of pieces of multimedia content in which the search criteria is found [0070]. The multimedia processing system (i.e., the overall distributed computing architecture) can respond directly to search requests or playback requests from content consumers seeking to search for information related to the multimedia content [0125]. A search field 1408 and search results field 1410 allow a user (e.g., a content producer or consumer) to search video metadata stored at a multimedia processing system while the user is viewing the video. Search data results can be selected to move to a new section of the current video or to a specific section of a different video. The search results can be represented in the search results field 1410 as graphical scenes, portions of a transcript associated with the content, or other recognizable elements (Fig. 14 and [0187]). The search performed within the content can in certain embodiments, be performed based on natural language processing of an existing transcript (closed captioning or subtitles file provided by the content provider) or from a new transcript created using speech to text technology and edited by the content provider [0188]. The enhanced search systems allow a user to perform intelligent searches within content processed by the multimedia processing system, using the metadata and other information stored (e.g. in database 726) as each video is processed. Each time a video is processed, additional information can be added to the stored information that is available to be searched, either within the video or as made available for indexing by larger search engines external to the multimedia processing system [0188].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Nvs in view of Singh’s AI/ML Video Analysis and Verification Process with Tsai’s system for capturing and processing video metadata in order to provide an easy way to identify objects appearing in the content such that a user can individually search for and identify those objects (Tsai e.g. [0005]).
Claims 21 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Nvs et al. (US 2024/0428204 A1) in view of Singh et al. (US 2023/0245686 A1), in further view of Tsai et al. (US 2012/0078626 A1), and in further view of Jakka et al. (US 2021/0279852 A1).
As per claim 21 (Currently Amended), Nvs in view of Singh and Tsai teach the method of claim 24, wherein determining the respective second project parameter of the project profile through software analysis of the video data comprises:
Nvs teaches performing automated image analysis on a plurality of frames of the video data; (Nvs e.g. FIG. 4 is a block diagram of a systems 400 for assessing damage to a structure using video of the structure, according to some embodiments. In the system 400, the damage assessment includes two parallel analyses, one of visual content provided in video content that includes several image frames, and may include one or more images, of the structure and the other of the associated audio content [0027]. The video 402 uploaded by the user (as described above with reference to FIG. 3 ) includes image frames 404 and audio content 406 [0029]. After the pre-processing, an AI/ML model is used by the visual assessment module 414. The visual assessment module 414 may include one or more processors that are configured using the AI/ML model (e.g., a deep learning model), where the model is trained to detect damaged structure parts from the selected video frames 410, and to assess the scope of damage to such part(s) [0032].)
Nvs teaches identifying, within the frames, a region the work environment or an object within the work environment based on a category of the project; and (Nvs e.g. A method and system are provided for assessing damage to a structure (Abstract). The method includes detecting one or more external parts of the structure from a video of the structure using a first machine learning (ML) module trained to identify in one or more frames of a video of a structure an external part of the structure (Abstract). The method also includes using a second ML module, trained to detect and classify damaged regions of a structure from one or more frames of the video: (i) identifying one or more damaged regions of the structure, and (ii) classifying the one or more damaged regions based on damage types (Abstract). FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 608, the identified keywords and/or phrases are matched with a parts dictionary, to detect vehicle parts (structure parts, in general) such as door, headlight, etc., that are described in the speech. In step 610, the identified keywords and/or phrases are matched with a damage-types dictionary, to detect the different types of damages, such as dents, scratches, missing parts, etc., that are described in the speech [0053].)
Nvs, Singh, nor Tsai explicitly teach, however, Jakka teaches determining, from depth data or LiDAR data associated with the video data, one or more physical size parameters of the identified region, the size parameters including at least one of a surface area, a linear dimension, or a radial dimension. (Jakka e.g. Disclosed is a method for using a virtual representation of an indoor environment to identify contents that have been damaged (e.g., by flooding).The virtual representation may include 2-dimensional representations of the physical scene (e.g., images or video) or a 3-dimensional representation of the physical scene (e.g., 3D digital model) (Abstract). FIG. 1 illustrates a system configured for facilitating AI-based cost estimates for services, in accordance with one or more implementations [0067]. The receiving scanned data component 108 may be configured to receive, at one or more hardware processors, data from a scan of a location, the scan performed by one or more of a camera, a computer vision device, an inertial measurement unit, a depth sensor, or other scanners. In some implementations, scanning includes data generated by video or image acquisition devices, voice recording devices, a user interface, or any combination thereof [0069]. In some implementations, a workflow may include a user launching an app or another messaging channel (SMS, MMS, Facebook Messenger, web browser, etc.) and scanning a location (e.g., a home or another location) where camera(s) data or sensor(s) data may be collected. The app may use the camera or IMU, and optionally a depth sensor, to collect and fuse data to detect surfaces to be painted, objects to be moved, etc. and estimate their surface area data, or move related data, in addition to answers to specific questions. An AI algorithm (or neural network etc.) specifically trained to identify key elements may be used (e.g., walls, ceiling, floor, furniture, wall hangings, appliances, or other objects). Other relevant characteristics may be detected including identification of light switches/electrical outlets that would need to be covered or replaced, furniture that would need to be moved, carpet/flooring that would need to be covered, or other relevant characteristics [0077]. FIG. 5 illustrates an exemplary system 500 wherein detected objects may create an inventory, size, or weight information for objects that are detected, as well as create a list of questions that the AI algorithm may need to provide a more accurate data to a service provider or user (for example, questions on the pictures sent by the user, or follow up questions based on past responses) [0097].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Nvs in view of Singh and Tsai’s AI/ML Video Analysis Estimation and Verification Process to include determining from depth data associated with the video physical size parameters of the identified region as taught by Jakka in order to provide upfront, accurate cost/price estimates (Jakka e.g. [0061])
As per claim 25 (New), Nvs in view of Singh teach the computer-implemented method of claim 1, wherein:
Nvs teaches the first project parameter comprises a user-supplied location parameter representing the physical location at which the project is to be performed; (Nvs e.g. When a vehicle accident occurs, the persons involved typically call the insurance carrier and file a First Notice of Loss (FNOL), which helps determine the extent of damage to the vehicle, and, subsequently, the liability and/or proper coverage for the required repairs. Using the information about the accident included in the FNOL, the adjuster may decide in step 104 whether the vehicle should be declared a total loss or may be repaired [0015]. Upon determining that the vehicle is repairable, the damage thereto must be assessed, so in the optional step 106 the adjuster sends one or more inspectors to obtain images of the vehicle, sometimes at the scene of the accident or elsewhere [0016]. Some embodiments can thus automate the process of estimating vehicle damage and repair costs from a video of the vehicle, such as those taken at the site of the accident [0017].)
Nvs teaches performing automated image analysis of a plurality of frames of the video data (Nvs e.g. FIG. 4 is a block diagram of a systems 400 for assessing damage to a structure using video of the structure, according to some embodiments. In the system 400, the damage assessment includes two parallel analyses, one of visual content provided in video content that includes several image frames, and may include one or more images, of the structure and the other of the associated audio content [0027]. The video 402 uploaded by the user (as described above with reference to FIG. 3 ) includes image frames 404 and audio content 406 [0029]. After the pre-processing, an AI/ML model is used by the visual assessment module 414. The visual assessment module 414 may include one or more processors that are configured using the AI/ML model (e.g., a deep learning model), where the model is trained to detect damaged structure parts from the selected video frames 410, and to assess the scope of damage to such part(s) [0032].) to identify, based on a category of the project, a region corresponding to the work environment or an object within the work environment; (Nvs e.g. A method and system are provided for assessing damage to a structure (Abstract). The method includes detecting one or more external parts of the structure from a video of the structure using a first machine learning (ML) module trained to identify in one or more frames of a video of a structure an external part of the structure (Abstract). The method also includes using a second ML module, trained to detect and classify damaged regions of a structure from one or more frames of the video: (i) identifying one or more damaged regions of the structure, and (ii) classifying the one or more damaged regions based on damage types (Abstract). FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 608, the identified keywords and/or phrases are matched with a parts dictionary, to detect vehicle parts (structure parts, in general) such as door, headlight, etc., that are described in the speech. In step 610, the identified keywords and/or phrases are matched with a damage-types dictionary, to detect the different types of damages, such as dents, scratches, missing parts, etc., that are described in the speech [0053].)
Nvs teaches determining, based on the physical size parameter, a quantity of a material to be used in performing the project, wherein generating the project profile comprises including the physical size parameter and the quantity of the material in the project profile (Nvs e.g. FIG. 2A illustrates the timeline of an exemplary artificial intelligence (AI)/machine learning (ML) based process 200 for assessing damage to a vehicle and for processing vehicle insurance claims [0018]. In step 208, a video of the vehicle (structure, in general) taken by the inspector or the user/owner of the vehicle is received. In step 210, an AI/ML system analyzes the video received from the inspector/customer and determines whether the vehicle is a total loss or is repairable [0018]. In step 212, the AI/ML system analyzes the information about the parts detected to be damaged and the types of damages, and may estimate damage to any internal parts. Using this information, the AI/ML system then generates a list of repairs and replacements that are likely needed and predicts the required time and/or cost for the repairs and/or replacements. The damage analysis and the repair/replacement estimates generated by the AI/ML system are presented in an estimation platform [0019].)
Nvs nor Singh explicitly teach, however, Tsai teaches the following:
Tsai teaches the verification information comprises geospatial information included in metadata of the video data and representing a location at which the video data was obtained; (Tsai e.g. The present disclosure relates to methods and systems for receipt, processing, and delivery of multimedia content, as well as enrichment of multimedia content for enhanced search and delivery [0041]. In the context of the present disclosure, multimedia content can include any type of content containing, for example, one or more of images, video, audio, or a combination thereof [0042]. The video conversion module 610 converts the received multimedia content from a format in which it is received from a content provider into a format useable with the metadata generated by the audio processing module 606 and the video processing module 608 [0098]. The metadata from the audio processing module 606 and video processing module 608 is passed to a database 626, which collects metadata and other information derived from the multimedia content. In the embodiment shown, the database receives click through events 628, a full text search database 630, video 632, video metadata 634, and position metadata 636 based on processing of content [0100]. Video metadata 634 includes any of the data describing the video that can be tracked. For example, the video metadata 632 can include information about objects of interest defined as associated with multimedia content [0103]. The position metadata 636 defines the position or location of one or more objects in the video content. The position metadata 636 can take any of a number of forms. In certain embodiments, the position metadata 636 corresponds to GPS metadata associated with one or more pieces of content. In other embodiments, the position metadata 636 can be captured from some other type of position sensor, such as a location sensor or radio frequency identification (RFID) tag [0104].)
Tsai teaches determining whether the first project parameter corresponds to the verification information comprises determining whether the physical location represented by the user- supplied location parameter and the location represented by the geospatial information are within a defined proximity of one another; and responsive to determining that the represented locations are within the defined proximity and before generating the project profile (Tsai e.g. FIG. 15 is a flowchart of methods and systems 1500 for detecting positions of objects within a scene in video content, according to a possible embodiment of the present disclosure [0196]. The positions captured by the capture position operation 1508 can be, in various embodiments, absolute positions (e.g. determined by GPS sensor positioning) or positions relative to a camera capturing video of those objects. By concurrently capturing both multimedia content and position information, the content and positions of objects within the content can be synchronized [0197]. An object identification operation 1512 then associates the objects detected in the multimedia content with the positional information received in the capture position operation 1508. An optional user confirmation operation 1514 receives an indication from the user confirming that the objects of interest are in fact those associated with the positional information [0198]. A location extrapolation operation 1516 extrapolates locations of the objects of interest within the multimedia content based on the relative positions of the objects as known from the positional information [0199].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Nvs in view of Singh’s AI/ML Video Analysis and Verification Process with Tsai’s system for capturing and processing video metadata in order to provide an easy way to identify objects appearing in the content such that a user can individually search for and identify those objects (Tsai e.g. [0005]).
Nvs, Singh, nor Tsai explicitly teach, however Jakka teaches the following:
Jakka teaches determining, from depth data or LiDAR data associated with the video data, a physical size parameter of the identified region; (Jakka e.g. Disclosed is a method for using a virtual representation of an indoor environment to identify contents that have been damaged (e.g., by flooding).The virtual representation may include 2-dimensional representations of the physical scene (e.g., images or video) or a 3-dimensional representation of the physical scene (e.g., 3D digital model) (Abstract). FIG. 1 illustrates a system configured for facilitating AI-based cost estimates for services, in accordance with one or more implementations [0067]. The receiving scanned data component 108 may be configured to receive, at one or more hardware processors, data from a scan of a location, the scan performed by one or more of a camera, a computer vision device, an inertial measurement unit, a depth sensor, or other scanners. In some implementations, scanning includes data generated by video or image acquisition devices, voice recording devices, a user interface, or any combination thereof [0069]. In some implementations, a workflow may include a user launching an app or another messaging channel (SMS, MMS, Facebook Messenger, web browser, etc.) and scanning a location (e.g., a home or another location) where camera(s) data or sensor(s) data may be collected. The app may use the camera or IMU, and optionally a depth sensor, to collect and fuse data to detect surfaces to be painted, objects to be moved, etc. and estimate their surface area data, or move related data, in addition to answers to specific questions. An AI algorithm (or neural network etc.) specifically trained to identify key elements may be used (e.g., walls, ceiling, floor, furniture, wall hangings, appliances, or other objects). Other relevant characteristics may be detected including identification of light switches/electrical outlets that would need to be covered or replaced, furniture that would need to be moved, carpet/flooring that would need to be covered, or other relevant characteristics [0077]. FIG. 5 illustrates an exemplary system 500 wherein detected objects may create an inventory, size, or weight information for objects that are detected, as well as create a list of questions that the AI algorithm may need to provide a more accurate data to a service provider or user (for example, questions on the pictures sent by the user, or follow up questions based on past responses) [0097].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Nvs in view of Singh and Tsai’s AI/ML Video Analysis and Verification Process to include determining from depth data associated with the video physical size parameters of the identified region as taught by Jakka in order to provide upfront, accurate cost/price estimates (Jakka e.g. [0061])
Claims 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Nvs et al. (US 2024/0428204 A1) in view of Singh et al. (US 2023/0245686 A1) and in further view of Jakka et al. (US 2021/0279852 A1).
As per claim 27 (New), Nvs in view of Singh teach the computer-implemented method of claim 1, Nvs teaches wherein the first project parameter comprises a numerical project parameter, the method further comprising, responsive to determining that the first project parameter corresponds to the verification information (Nvs e.g. Any damaged parts and/or components of the structure may be identified, and the severity of the damage can be assessed automatically. An audio recording associated with the video may be used to supplement and/or validate the assessment. Then, based on the nature of the damage and using external data sources ( e.g., vehicle history, replacement part data, etc.,) repair cost can be estimated via Machine Leaming models [0017].):
Nvs or Singh explicitly teach, however, Jakka teaches the following:
Jakka teaches comparing the first project parameter with a numerical representation of corresponding parameters of one or more other projects having a same project category; determining whether the first project parameter differs from the numerical representation by at least a threshold amount; and (Jakka e.g. An attribute identification module (illustrated in block 3504) is generally a machine learning algorithm such as decision trees, neural net, etc., which is used to identify the attributes of an item such as material, condition, etc. These attributes are later used in value estimation modules to compare with those of a similar item in new condition or match with a similar item whose value estimate is predetermined and is stored in a database [0293]. In the fair market value estimation module (block 3505) the item identifications and their attributes, such as dimensions, material type, condition, etc., are used to determine relevant information including, but not limited to, value estimates of same items from underwriting, and items that have similar attributes whose value estimates are predetermined [0295]. For example, a part of the wall is damaged and needs to be replaced. The machine learning model compares the attributes of the wall such as its materials, dimensions, etc., to those of a similar wall in new condition, and thereby estimates the replacement value of the damaged wall, regressing from the value of the new wall [0296].)
Jakka teaches responsive to determining that the first project parameter differs from the numerical representation by at least the threshold amount, causing the remote computing device to output a notification indicating that a numerical value of the first project parameter should be increased or decreased (Jakka e.g. In some embodiments, the damage management subsystem 3720 may also facilitate determination of the cost of damage (e.g., repair and replacement costs) in terms of real currency based on information about the type, scope, and extent of damages determined as described above [0331]. The computed costs and the content damage information may be provided to an insurance claim adjustment process, which may determine or update the damage information or costs of damage, and execute a process to approve or decline the claim submitted by the user 3724. In some embodiments, the process may adjust the damage information (e.g., determine a damaged content as not damaged or vice versa, increase or decrease the determined costs) prior to approving the claim. In some embodiments, a human user, such as a reviewer 3726, may review the damage information or the costs in addition to or instead of the insurance claim adjustment process. The reviewer 3726 may also approve or decline the claim with or without adjusting the damage information or the costs [0332].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Nvs in view of Singh’s AI/ML Video Analysis and Verification Process to include comparing and adjusting project parameters based on numerical analysis as taught by Jakka in order to provide upfront, accurate cost/price estimates (Jakka e.g. [0061]).
As per claim 28 (New), Nvs in view of Singh teach the computer-implemented method of claim 1, further comprising, responsive to determining that the first project parameter corresponds to the verification information, and before generating the project profile:
Nvs teaches determining, based on the physical size parameter, a quantity of a material to be used in performing the project, wherein generating the project profile comprises including the physical size parameter and the quantity of the material in the project profile (Nvs e.g. FIG. 2A illustrates the timeline of an exemplary artificial intelligence (AI)/machine learning (ML) based process 200 for assessing damage to a vehicle and for processing vehicle insurance claims [0018]. In step 208, a video of the vehicle (structure, in general) taken by the inspector or the user/owner of the vehicle is received. In step 210, an AI/ML system analyzes the video received from the inspector/customer and determines whether the vehicle is a total loss or is repairable [0018]. In step 212, the AI/ML system analyzes the information about the parts detected to be damaged and the types of damages, and may estimate damage to any internal parts. Using this information, the AI/ML system then generates a list of repairs and replacements that are likely needed and predicts the required time and/or cost for the repairs and/or replacements. The damage analysis and the repair/replacement estimates generated by the AI/ML system are presented in an estimation platform [0019].)
Nvs teaches performing automated image analysis of one or more frames of the video data (Nvs e.g. FIG. 4 is a block diagram of a systems 400 for assessing damage to a structure using video of the structure, according to some embodiments. In the system 400, the damage assessment includes two parallel analyses, one of visual content provided in video content that includes several image frames, and may include one or more images, of the structure and the other of the associated audio content [0027]. The video 402 uploaded by the user (as described above with reference to FIG. 3 ) includes image frames 404 and audio content 406 [0029]. After the pre-processing, an AI/ML model is used by the visual assessment module 414. The visual assessment module 414 may include one or more processors that are configured using the AI/ML model (e.g., a deep learning model), where the model is trained to detect damaged structure parts from the selected video frames 410, and to assess the scope of damage to such part(s) [0032].) to identify, based on a category of the project, a region corresponding to the work environment or an object within the work environment; (Nvs e.g. A method and system are provided for assessing damage to a structure (Abstract). The method includes detecting one or more external parts of the structure from a video of the structure using a first machine learning (ML) module trained to identify in one or more frames of a video of a structure an external part of the structure (Abstract). The method also includes using a second ML module, trained to detect and classify damaged regions of a structure from one or more frames of the video: (i) identifying one or more damaged regions of the structure, and (ii) classifying the one or more damaged regions based on damage types (Abstract). FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 608, the identified keywords and/or phrases are matched with a parts dictionary, to detect vehicle parts (structure parts, in general) such as door, headlight, etc., that are described in the speech. In step 610, the identified keywords and/or phrases are matched with a damage-types dictionary, to detect the different types of damages, such as dents, scratches, missing parts, etc., that are described in the speech [0053].)
Nvs nor Singh explicitly teach, however, Jakka teaches determining, from depth data or LiDAR data associated with the video data, a physical size parameter of the identified region; (Jakka e.g. Disclosed is a method for using a virtual representation of an indoor environment to identify contents that have been damaged (e.g., by flooding).The virtual representation may include 2-dimensional representations of the physical scene (e.g., images or video) or a 3-dimensional representation of the physical scene (e.g., 3D digital model) (Abstract). FIG. 1 illustrates a system configured for facilitating AI-based cost estimates for services, in accordance with one or more implementations [0067]. The receiving scanned data component 108 may be configured to receive, at one or more hardware processors, data from a scan of a location, the scan performed by one or more of a camera, a computer vision device, an inertial measurement unit, a depth sensor, or other scanners. In some implementations, scanning includes data generated by video or image acquisition devices, voice recording devices, a user interface, or any combination thereof [0069]. In some implementations, a workflow may include a user launching an app or another messaging channel (SMS, MMS, Facebook Messenger, web browser, etc.) and scanning a location (e.g., a home or another location) where camera(s) data or sensor(s) data may be collected. The app may use the camera or IMU, and optionally a depth sensor, to collect and fuse data to detect surfaces to be painted, objects to be moved, etc. and estimate their surface area data, or move related data, in addition to answers to specific questions. An AI algorithm (or neural network etc.) specifically trained to identify key elements may be used (e.g., walls, ceiling, floor, furniture, wall hangings, appliances, or other objects). Other relevant characteristics may be detected including identification of light switches/electrical outlets that would need to be covered or replaced, furniture that would need to be moved, carpet/flooring that would need to be covered, or other relevant characteristics [0077]. FIG. 5 illustrates an exemplary system 500 wherein detected objects may create an inventory, size, or weight information for objects that are detected, as well as create a list of questions that the AI algorithm may need to provide a more accurate data to a service provider or user (for example, questions on the pictures sent by the user, or follow up questions based on past responses) [0097].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Nvs in view of Singh’s AI/ML Video Analysis and Verification Process to include determining from depth data associated with the video physical size parameters of the identified region as taught by Jakka in order to provide upfront, accurate cost/price estimates (Jakka e.g. [0061])
As per claim 29 (New), Nvs in view of Singh teach the computer-implemented method of claim 1, further comprising obtaining a description parameter by:
Nvs teaches passing an audio portion of the video data to a speech-to-text module that performs speech recognition to generate text data; and (Nvs e.g. This disclosure generally relates to artificial intelligence (AI) based/machine learning (ML) techniques and, in particular, to training and use of AI/ML systems to: determine damage to physical structures and estimate the effort required to repair the damage by analyzing videos of the physical structures [0001]. Some embodiments of a technique described herein feature artificial intelligence/machine learning enabled computer vision and analysis, along with optional speech analysis, so that different parts and/or components of a physical structure are recognized from a video of the structure [0017]. FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 602, the noise suppressed audio content 604 is received from the pre-processing stage (e.g., the audio content 412 provided by the pre-processing module 408 shown in FIG. 4). In step 602, the audio content 604 is converted to text using a speech-to-text engine [0052].)
Nvs teaches passing at least a portion of the cleaned text data to a machine-learning module trained to generate a concise description of the project, (Nvs e.g. Some embodiments of a technique described herein feature artificial intelligence/machine learning enabled computer vision and analysis, along with optional speech analysis, so that different parts and/or components of a physical structure are recognized from a video of the structure [0017]. FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. FIG. 6 is a flow chart of a process 600 for assessing damage to a structure using audio associated with a visual content in a video of the structure, according to some embodiments [0051]. In step 606, the generated text is contextually parsed using computation linguistic techniques and stored for every specified time window (e.g., 5, 10, 12 seconds, etc.) of the corresponding extracted video content (e.g., the frames 410 shown in FIG. 4). Based on the contextual content parsing, various keywords and/or phrases in the speech that may be relevant to the detection of parts and/or damage are identified in the step 606 [0052]. In step 608, the identified keywords and/or phrases are matched with a parts dictionary, to detect vehicle parts (structure parts, in general) such as door, headlight, etc., that are described in the speech. In step 610, the identified keywords and/or phrases are matched with a damage-types dictionary, to detect the different types of damages, such as dents, scratches, missing parts, etc., that are described in the speech [0053].)
Nvs teaches wherein generating the project profile comprises including the concise description as the description parameter (Nvs e.g. When a vehicle is involved, unfortunately, in an accident and has visibly distorted or damaged parts, the user (e.g., the driver, owner, a passenger, etc.) records a video of the vehicle, e.g., using a smart device (a smart phone, a tablet, etc.) or a video camera. The video may include visual content only or may also include audio content, e.g., a verbal commentary providing a description of the damaged parts and/or the nature of the damage [0024].).
Nvs nor Singh explicitly teach, however, Jaka teaches:
Jakka teaches processing at least a portion of the text data using a data-cleaning module to generate cleaned text data by removing repeated words, removing one or more words identified as adding no value, and adding punctuation; (Jakka e.g. The modules may include a spurious and duplicate detection for an entire dataset module. An artificially intelligent ( e.g., machine learning) algorithm may be used to remove spurious objects across multiple blocks of data [0250]. Block 2505 illustrates a spurious and duplicate detection module. This module removes any duplicates in the inventory item list using object correlations across the input description data. This is used as a post processing step before presenting the final output to the user [0269]. After the contents are identified in the virtual representation, in some embodiments, the content identification subsystem 3710 may then post-process the output of the machine learning model to suppress any duplicate recognition of the content. The content identification subsystem 3710 may be configured to identify and remove duplicates using any of a number of duplicate suppression methods, e.g., non-maximum suppression, confluence, class-based suppression, heuristic suppression (such as for duplication that may occur due to properties of contents in the physical scene such as reflectance in windows, mirrors, or televisions), similarity matching, object re-identification, or other methods [0314]. In some embodiments, the contents identified by the machine learning model may be further processed to identify and remove any duplicates. In another example, a GUI may be presented to the user 3724 who may input the content information, or update the content information generated by the machine learning model [0338].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Nvs in view of Singh’s AI/ML Video Analysis and Verification Process to include cleaning text data as taught by Jakka in order to provide upfront, accurate cost/price estimates (Jakka e.g. [0061]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure include NPL: Anjum et al., 2016 “Video Stream Analysis in Clouds: An Object Detection and Classification Framework for High Performance Video Analytics” and FOR: Cui, Peng-fei (CN-109359202-A) “Electronic Operation and Maintenance Work Order Automatic Generation Method and System Based on Audio Frequency and Video Record”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ayanna Minor whose telephone number is (571)272-3605. The examiner can normally be reached M-F 9am-5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O'Connor can be reached at 571-272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.M./Examiner, Art Unit 3624
/Jerry O'Connor/Supervisory Patent Examiner,Group Art Unit 3624