DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This is a nonfinal rejection in response to claim/amendments filed on 01/28/2026. Claims 1, 4-6, 14, 17-19, and 27 are amended herein, and Claims 28-31 are newly added. Claims 5 and 18 are canceled herein, and claims 3 and 16 were previously cancelled. Therefore, claims 1, 2, 4, 6-15, 17, and 19-31 are pending and are examined herein.
Priority
The earliest filing date is the filing date of the present application which is 12/06/2023.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/28/2026 has been entered.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 2, 4, 6-15, 17, and 19-31 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Is the claim to a Process, Machine, Manufacture, or Composition of Matter?
Claims 1, 2, 4, 6-13, and 28-31: A computer system comprising at least one processor in communication with at least one memory device, wherein the at least one processor programmed to:
Claims 14, 15, 17, 19-26: A computer-implemented method performed by a computer device including at least one processor in communication with at least one memory device, the method comprising: Claim 27: At least one non-transitory computer-readable media having computer-executable instructions embodied thereon, wherein when executed by a computing device including at least one processor in communication with at least one memory device, the computer-executable instructions cause the at least one processor to:
All of the claims fall under at least potentially eligible subject matter category, at least “process, machine, or manufacture,” therefore the claims are to be further analyzed under step 2.
Step 2a Prong 1: Is the claim reciting a Judicial Exception(A Law of Nature, a Natural Phenomenon (Product of Nature), or An Abstract Idea?)
The claims under the broadest reasonable interpretation in light of the specification are analyzed herein. Representative claims 1, 14 and 27 are marked up, isolating the abstract idea from additional elements, wherein the abstract idea is in bold and the additional elements have been italicized as follows:
Claim 1 Preamble: A computer system comprising at least one processor in communication with at least one memory device, wherein the at least one processor programmed to:
Claim 14 Preamble: A computer-implemented method performed by a computer device including at least one processor in communication with at least one memory device, the method comprising: Claim 27 Preamble: At least one non-transitory computer-readable media having computer-executable instructions embodied thereon, wherein when executed by a computing device including at least one processor in communication with at least one memory device, the computer-executable instructions cause the at least one processor to:
Claim 1, 14, 27 Body:
store(ing) one or more models for analyzing images to identify issues pictured in the images;
receive, from a user computer device associated with a location, a real-time video stream of a location;
control a camera of the user computer device to capture a plurality of initial images of the location from the real-time video stream;
store a plurality of initial images of a location;
receive(ing), from the user computer device, a plurality of current images of the location;
execute(ing) a trained image comparison model of the one or more models to compare the plurality of initial images to the plurality of current images, wherein an output from the trained image comparison model includes an identification of one or more differences between the plurality of initial images and plurality of current images;
based on the output, determine whether the one or more differences exceed a predetermined threshold;
for each difference that exceeds the predetermined threshold, define the difference as an issue by executing a trained classification model of the one or more models to identify a reason for the difference; and
schedule at least one appointment at the location to remediate at least one issue.
When evaluating the bolded limitations of the claims under the broadest reasonable interpretation in light of the specification, it is clear that representative claims 1, 14, and 27 recite an abstract idea under the category “certain methods of organizing human activity.” More specifically, the present invention falls under the sub-groupings “commercial or legal interactions” which include agreements in the form of contracts, legal obligations, advertising, marketing or sales activities or behaviors, and business relations” and “managing personal behavior, or interactions, or relationships between people.” The representative claims recite a commercial or legal interaction known as performing inspections, which is made clear in at least specification paragraph [0004], and [0005] is a practice performed in order to manage rental properties. Therefore, the claims, which perform data recordation, processing, and outputting in order to detect issues within a rental property are merely steps to comply with legal obligations and facilitate business relations. Furthermore, the steps merely transmit the notifications in order to manage the personal behavior, relationships and interactions between individuals which is another subcategory of “certain methods of organizing human activity” outlined in MPEP 2106.04(a)(2)(II)(C) which include social activities, teaching and following rules or instructions.
Even when considering “receiving a real-time video stream of a location,” and “capture a plurality of initial images of the location from the real-time video stream”, are recited with such breadth that the scope of the claims includes instructions to perform the intended functions of receiving video data and capturing images from the data. The broadest reasonable interpretation of the claims includes any manner of receiving real-time video data, and capturing images from the real-time video data. This includes instructions to manage the behavior of a person in order to capture the intended images, which would fall under the scope of “certain methods of organizing human activity.” Furthermore, these steps are also mere data collection steps associated with the abstract idea.
Even when considering the amended limitations that the executing of a “trained image comparison model,” which outputs “differences between the initial images and current images,” followed by “determine whether the one or more differences exceed a predetermined threshold,” “define the difference by executing a trained classification model to identify a reason for the difference.” These are still recitations of an abstract idea because it merely claims the use of mathematical models to perform the “certain method of organizing human activity” at a high-level of generality that it is no more than claiming the models as “a black box” to perform the intended outcome. Therefore, the claim limitations in their broadest reasonable interpretation include mere instructions to an individual to perform the abstract tasks at hand using any trained image comparison model, or any trained classification model. Furthermore, “scheduling an appointment to remediate the issue” is still an example of “commercial or legal interactions,” or “managing personal behavior, interactions, or relationships between people,” because it is merely a reciting a social task.
To further show that the amended limitations are broad enough to be mere instructions to a user to manage their personal behavior, the following arguments describe how the amended limitations are merely mental processes that can be performed by a human using a pen and paper.” The particular steps, starting from “execute a trained image comparison model...” and ending with “for each difference that exceeds the predetermined threshold, define the difference by executing a trained classification model to identify a reason for the difference.” Nothing in these limitations, other than merely instructing them to be performed on a computer (which will be addressed later), merely limits the claims from being performed in the human mind. See MPEP 2106.04(a)(2)(III), “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.” For example, a human mind can compare the initial images to the current images, and identify one or more differences between the plurality of initial images and plurality of current images. The human mind, using a pen and paper can determine whether the differences exceed a predetermined threshold (via scoring, or quantifying the amount of differences, or any other viable manner that can be performed in the human mind). Furthermore, a human can define the difference as an issue and identify the reason for the difference, all through a series of observations, evaluations, judgments, and opinions. Therefore, the claims recite “collecting information, analyzing it, and displaying certain results of the collection analysis,” where the data analysis steps are recited at such a high level of generality such that they could practically be performed in the human mind. Requiring that a “trained image comparison model,” and a “trained classification model,” are used to perform the tasks is still broad enough to recite a mental process cause it can include any use of a model (mathematical calculation), such that the use of a physical aid (pencil and paper or a slide rule) would enable a human to perform the steps manually. Therefore, when considering these particular amendments, they fall under “managing personal behavior, or relationships, or interactions between people,” because they encapsulate mere instructions to an individual to manage their personal behavior, supported by the fact that the functions can be performed mentally by a human with a pen and paper.
Therefore, the claims recite an abstract idea of “detecting issues within images of a rental property and transmitting notifications based on the issues” wherein the steps are recited at a high level of generality that they are no more than “certain methods of organizing human activity.” The examiner notes that even though the interactions may be between an individual and a computer, the activity itself is what is considered. MPEP 2106.04(a)(2)(II) states, “Finally, the sub-groupings encompass both activity of a single person (for example, a person following a set of instructions or a person signing a contract online) and activity that involves multiple people (such as a commercial interaction), and thus, certain activity between a person and a computer (for example a method of anonymous loan shopping that a person conducts using a mobile phone) may fall within the "certain methods of organizing human activity" grouping.”
In addition, the claims and are to be further analyzed under Step 2A Prong 2 and step 2b.
Step 2A Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
Claims 1, 14, and 27 recite the following additional elements:
-computer system in claim 1
-processor in claims 1, 14, 27
-memory device in claims 1, 14, 27
-computer-implemented method in claims 14
-computer device in claims 14, 27
-non-transitory computer-readable medium in claim 27
- a user computer device in claims 1, 14, 27
- control a camera of the user computer device; in claims 1, 14, 27
The additional elements listed above, when considered individually and in combination with the claim as a whole, are no more than a recitation of the words “apply it” (or an equivalent) or mere instructions to implement an abstract idea or other exception on generic computing components as outlined in MPEP 2106.05(f). In this case, the abstract idea of “detecting issues within images of a rental property and transmitting notifications based on the issues” are performed on generic computing devices such as computer devices, processors, memory devices, user computer device and non-transitory computer-readable medium. The claims do provide steps of creating models and analyzing images, but these are merely abstract idea steps that can be performed by a generic computing device. Particularly, the ”trained image comparison model,” and “trained classification model” are invoked to perform the abstract idea but are used in a black box manner that does not specify the details of how a solution to a problem is accomplished. There are no specific improvements to image comparison or classification models, and the claims recite that they are already “trained,” without citing any specific training process. Therefore, the claims are merely invoking computers or other machinery to perform an existing process, and recite only the idea of an outcome or solution, therefore, the additional elements are merely equivalent to the words “apply it,” because they are mere instructions to implement an abstract idea or other exception on a computer. Even when considering the amended limitation of capturing “real-time video stream of a location” this is still an example of utilizing the computer to perform economic tasks (such as data capture), in order to perform the abstract idea.
Furthermore, the steps of “control a camera of the user computer device... to capture a plurality of initial images of the location from the real-time video stream;” also falls under “apply it” in MPEP 2106.05(f), because it is using a device (such as a camera) in its ordinary capacity to perform the capturing of images without providing an improvement to camera technology or any technical field (See MPEP 2106.05(a)). The control of a camera of the user computer device, as supported in [0033], includes,
“In additional embodiments, the IA computer device controls the camera of the user's mobile device when taking images of the property. For example, the IA computer device instructs the user where to stand and at what angle to point to take the image.” Therefore, the control a camera of the user computer device step merely includes a set of instructions to perform the abstract idea on a generic computing device or an ordinary device in its ordinary capacity (instructions to capture a photo using a camera).
Even when considering the additional elements individually or as an ordered combination, nothing in claims integrates the abstract idea into a practical application. Therefore, the claims are directed to an abstract idea without integration into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Claims 1, 14, and 27 recite the following additional elements:
-computer system in claim 1
-processor in claims 1, 14, 27
-memory device in claims 1, 14, 27
-computer-implemented method in claims 14
-computer device in claims 14, 27
-non-transitory computer-readable medium in claim 27
- a user computer device in claims 1, 14, 27
- control a camera of the user computer device; in claims 1, 14, 27
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered separately and as an ordered combination, they do not add significantly more (also known as an “inventive concept”) to the exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using computers devices, processors, memory devices, user computer device, camera and non-transitory computer-readable medium to perform the “storing one or more models to identify issues pictured in images, receiving a real-time video stream, capturing initial images, receiving current images, executing a model to compare the images and identify the differences, determine if the differences exceed a threshold and identify the issue using a classification model, and schedule an appointment to remediate the issue” amounts to no more than mere instructions to apply the exception using generic computer components. Even when viewing the claims as a whole, nothing meaningfully limits the claims such that they are significantly more than the abstract idea. There is no improvement to how any models are trained, and there is no improvement to computer functionality, technology, or any technical field. Therefore, the claims are directed to an abstract idea without integration to a practical application or significantly more.
Dependent claims 2, 4, 6-13, 15, 17, and 19-31 are also given the full two part analysis both individually and in combination with the claims they depend on herein:
Claims 2, 15 further define the abstract idea by limiting the data to a particular source, such as historical images of a property, or images received from a computer device associated with the location. This is more of the same abstract idea because they are merely indicating a data source or generally stating a data processing state, whilst still performing the same abstract idea of “detecting issues within images of a rental property and transmitting notifications based on the issues.” The additional elements of processor, and user computing device are still merely linking the abstract idea to generic computing components. Since there are no further additional elements to analyze, the claims are directed to an abstract idea without integration into a practical application or significantly more.
Claims 4, 6, 7, 9, 17, 19, 20, 22 further define the abstract idea by providing instructions to the user in order to capture the data. This is merely a further embellishment of the abstract idea, since it is an “instruction” in order to facilitate human behavior, which is also a “certain method of organizing human activity” in the subcategory of “managing personal behavior, interactions, or relationship between individuals” in MPEP 2106.05(a)(2)(II)(C). The only new additional elements are the “display screen” in claims 7, 20. However, this is merely a generic computing device performing the abstract idea, operating in its ordinary capacity to perform the display of an output, as outlined in MPEP 2106.05(f). Therefore, the claims are directed to an abstract idea without integration into a practical application or significantly more.
Claims 8, 21 further define the abstract idea by “controlling the user computer device to capture the plurality of initial images.” This is merely a data collection step in the processing of performing the abstract idea of “detecting issues within images of a rental property and transmitting notifications based on the issues.” There are no further additional elements to analyze, therefore, the claims are directed to an abstract idea without integration into a practical application or significantly more.
-Claim 10, 23 further define the abstract idea by scheduling an appointment with a service provider to resolve the one or more issues. This is merely a further embellishment of the abstract idea, since transmitting to the scheduling to a service provider is still an instruction to an individual and it is no more than managing an individual’s workflow, which is also a “certain method of organizing human activity” in the subcategory of “managing personal behavior, interactions, or relationship between individuals” in MPEP 2106.05(a)(2)(II)(C). There are no further additional elements to analyze, therefore, the claims are directed to an abstract idea without integration into a practical application or significantly more.
-Claims 11, 12, 13, 24, 25, 26 receives images overtime and determines a trend or calculates a score. These are merely embellishments of the abstract idea, since they recite additional data processing steps that still perform the abstract idea of “detecting issues within images of a rental property and transmitting notifications based on the issues.” There are no further additional elements to analyze, therefore, the claims are directed to an abstract idea without integration into a practical application or significantly more.
-Claims 28 and 29 recite more of the same abstract idea because it merely adds the steps of defining a predefined period of time since images were captures and transmit instructions to the user device to capture the images. The abstract idea is still recited because the instructions are merely to capture the initial images (28) and store the plurality of periodic images in a timeline (29) which are mere data collection and storage steps associated with the abstract idea. The additional element of sending the instructions to the user computer device periodically is still equivalent to apply because performing the abstract idea on a computer at multiple periods is still “mere instructions to carry out the abstract idea.” Even when considering the additional elements individually or in an ordered combination, they still fail to integrate the abstract idea into a practical application because all of the claimed functions are invoking the use of generic computing components to perform an existing abstract idea process. Even when viewed as a whole, nothing meaningfully limits the abstract idea such that it provides significantly more.
Claim 30 further limits the abstract idea by prioritizing issues based on urgency and scheduling an appointment to address the first priority issue. This is more of the same “certain methods of organizing human activity,” because it is merely managing the workflow and performing business relationships. There are no additional elements to consider that have not already been identified as “apply it” level elements, therefore the claims are directed to an abstract idea without integration into a practical application without significantly more.
Claim 31 further limits the abstract idea by adding the step of receiving audio information, and analyzing the audio information to detect the issues at the location. However, this is more of the same abstract idea because it is recited with such generality that it does exclude instructions to an individual to perform the task. Furthermore, it does not specify how the issues are detecting from the audio, therefore, it only recites the idea of the outcome or solution. There are no additional elements to consider that have not already been identified as “apply it” level elements, therefore the claims are directed to an abstract idea without integration into a practical application without significantly more.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4, 6-9, 11-15, 17, 19-22, 24-28, 29, and 31 are rejected under 35 U.S.C. 103 as being obvious over Nelson et al. (US 20200410278 A1) hereinafter Nelson, in view of Chavez et al. (US 11600063 B1) hereinafter Chavez.
Regarding Claims 1, 14, 27:
Nelson discloses a method of determining damage to a property by comparing first images to second images and using a trained model that determines the damage. Nelson teaches
Claim 1 Preamble: A computer system comprising at least one processor in communication with at least one memory device, wherein the at least one processor programmed to: (Nelson [0004] An illustrative non-transitory computer readable medium has instructions stored thereon that, upon execution by a computing device, cause the computing device to perform operations.)
Claim 14 Preamble: A computer-implemented method performed by a computer device including at least one processor in communication with at least one memory device, the method comprising: (Nelson [0005] An illustrative method for capturing an image of an object associated with a property using a mobile computing device includes displaying, by a processor of the mobile computing device, a first image representing a field of view of a camera of the mobile computing device on a display of the mobile computing device. The method further includes overlaying, by the processor, a representation of an object associated with a property on the display that is also displaying the first image.)
Claim 27 Preamble: At least one non-transitory computer-readable media having computer-executable instructions embodied thereon, wherein when executed by a computing device including at least one processor in communication with at least one memory device, the computer-executable instructions cause the at least one processor to: (Nelson [0004] An illustrative non-transitory computer readable medium has instructions stored thereon that, upon execution by a computing device, cause the computing device to perform operations.)
Claims 1, 14, 27 Body:
-stor(ing) one or more models for analyzing images to identify issues pictured in the images; (Nelson [0027] In addition, the various electronic devices may be used to train and implement machine learning models for identifying and assessing damage to objects and property as described herein. For example, the servers 112 may be used to train a model that is stored on one or more of the servers 112 for identifying and assessing damage.)
-storing a plurality of initial images of a location; (Nelson [0074] FIG. 28 is a flow chart illustrating an example method 2800 of determining whether damage to an object exceeds expected wear and tear depreciation damages, in embodiments. As described herein, a user may capture images of various objects or aspects of a property during a property inspection. [0075] In an operation 2802 of the method 2800, a first image of an object is received, the first image being captured at a first time. A first image of an object may be received by virtue of a user using an electronic device to capture an image (e.g., as shown in FIG. 19-26 or 30-37), or a first image may be received by a server or other electronic device from a device that was actually used to capture the image. [0054] FIG. 8 shows a user interface 800 for a move-in inspection that has been completed. The user interface 800 may be navigated to, for example, if a completed inspection is selected at the interface 500 of FIG. 5. A back arrow 802 may be used to navigate back to a list of inspections such as that of FIG. 5. Information 804 may indicate details of the selected inspection, such as a status (e.g., inspection completed), the date completed, and who completed the inspection. [0087] In addition to images relating to a specific inspection, aggregated images and metadata from other inspections may be sanitized and selected at an operation 2910 for use in training a damage identification model. At an operation 2912, those images may be categorized for contents (e.g., what type of object is in the photographs) and presence of damages (e.g., is there damage shown in the photographs).) The first image is mapped to the initial images at a location taught by Nelson since these images are taking at move in. These images are a plurality of image as seen in Figs 6-8, there are multiple initial(first) images being taken.
-receiv(ing), from the user computer device, a plurality of current images of the location; (Nelson [0076] In an operation 2804, a second image of the object is received, the second image being captured at a second time different from the first time. Similar to the first image, the second image of an object may be received by virtue of a user using an electronic device to capture an image (e.g., as shown in FIG. 19-26 or 30-37), or the second image may be received by a server or other electronic device from a device that was actually used to capture the image. The first and second images in this embodiment are captured at different times. For example, the first image may be captured at or near a time when a tenant/renter moved into a property (or unit of the property), and the second image may be captured at or near a time when a tenant/renter moved out of the property. In this way, the images may represent what the object was like when the tenant/renter moved in and what the object was like when they moved out.)
-execut(ing) a trained image comparison model the one or more models to compare the plurality of initial images to the plurality of current images; (Nelson [0077] In other embodiments, the system may perform an image analysis to determine that the objects in an image are the same object. For, example, an image detection system may look for differences that indicate the images do not capture a same object to prevent errors or fraud. In some embodiments, a property manager or landlord may place a visual sticker or other indicator with a code (e.g., QR code, bar code, or other code) on an object so that a visual detection system may identify the code to ensure that the same object is captured. [0078] In various embodiments, a machine learning model trained as described herein may compare images and the objects therein without any alignment, resizing, cropping, etc. [0082] A machine learning model may be trained to consider multiple images of an object together to identify and assess damage as described herein. [0082] For example, additional images taken at different times from the first and second images may be compared to determine damage and/or wear and tear to an object over time. In other examples, the system may be configured to receive multiple contemporaneously taken images of an object to compare to one or more images of the object taken at a different time. For example, multiple images of an object may be taken from different angles, and those images may be used to compare to one or more images of the objects captured at a different time. A machine learning model may be trained to consider multiple images of an object together to identify and assess damage as described herein. [0084] FIG. 29 is a flow chart illustrating an example method 2900 of identifying and assessing damages and training a model for identifying and assessing damages to an object, in embodiments. As described herein, determining that at least one difference between two or more images is associated with damage to the object (e.g., wear and tear depreciation damage or damage that exceeds wear and tear) may include processing the images with a trained machine learning model. In addition to using training a model to identify and assess damages and using a trained model to identify and assess damages as shown in the method 2900, images captured during inspections may be added to data sets that are used to further refine and train models for identifying and assessing damage in images captured of objects over time.)
-wherein an output from the trained image comparison model includes an identification of one or more differences between the plurality of initial images and plurality of current images;(Nelson [0080] In an operation 2810, at least one difference associated with the object based on a comparison between the first image and the second image is determined. [0084] FIG. 29 is a flow chart illustrating an example method 2900 of identifying and assessing damages and training a model for identifying and assessing damages to an object, in embodiments. As described herein, determining that at least one difference between two or more images is associated with damage to the object (e.g., wear and tear depreciation damage or damage that exceeds wear and tear) may include processing the images with a trained machine learning model.)
-based on the output, determine whether the one or more differences exceed a predetermined threshold;(Nelson [0088] The model and the manual review may be configured to identify only damage that exceeds wear and tear damage, or may be configured to identify any type of damage (e.g., material damage and wear and tear damage). In other words, the model and manual review may only be looking for damage that exceeds a particular threshold indicating material damage... In yet another example, a third model (beyond the damage identification and damage assessment models) may be specifically trained to identify whether damage is wear and tear or material damage. [0093] Some models may also be trained using data that indicates the extent of and/or presence of damage shown in images or pairs of images... In another example, information indicating whether damage is wear and tear or material damage may be indicated and input to train a model to identify when damage rises to a material damage threshold. [0102] In various embodiments, the bounding boxes 3206 and 3302 of FIGS. 32 and 33 may demonstrate different thresholds for identifying damage and/or determining that damage exceeds a wear and tear threshold... In contrast, if a threshold for exceeding wear and tear damage is set lower to include smaller holes such as the holes 3304 and 3306, a larger bounding box may be applied as shown in FIG. 33,[0122] In some embodiments, a standard size bounding box may be used to determine that damage exceeds a particular threshold (e.g., exceeds wear and tear damage).)
-for each difference that exceeds the predetermined threshold, define the difference as an issue by executing a trained classification model of the one or more models to identify a reason for the difference.(Nelson [0088] In other words, the model and manual review may only be looking for damage that exceeds a particular threshold indicating material damage. In other embodiments, where the manual review and model are configured to recognize any damage, a model for assessing value of the damage may be used to categorize whether the damage is wear and tear or material damage based on, for example, an amount of damage. [0089] If a manual review at the operation 2914 identifies damage in the images, the images may be tagged as incorrect (or may be re-tagged such that the metadata associated with the image indicates that the image includes damage), so that they can be resubmitted to refine or re-train the model that missed the damage in the images initially at an operation 2920. [0093] Some models may also be trained using data that indicates the extent of and/or presence of damage shown in images or pairs of images. For example, for the model for identifying damage as applied at the operation 2906 of FIG. 29, images and a tag indicating whether or not damage is present in the image, alone or in combination with other information, may be used to train such a model. In another example, information indicating whether damage is wear and tear or material damage may be indicated and input to train a model to identify when damage rises to a material damage threshold. [0090] If damage is identified in images at the operation 2906, the images may be further analyzed at an operation 2916 using a damage pricing model... For example, information may be identified at an operation 2928 to be used for training a damage pricing model. Such information may include, for example, images categorized/tagged by types of damage, types of objects, cost of repair or replacement, geological data, geographic data, demographic data, or any other type of data that may factor into the value of damage to various objects associated with a property... [0111] damage assessment model may use datasets that do not require pre-processing such as image classification or other types of pre-processing...FIG. 38 may be used to build/train models. [0102] In various embodiments, the bounding boxes 3206 and 3302 of FIGS. 32 and 33 may demonstrate different thresholds for identifying damage and/or determining that damage exceeds a wear and tear threshold. For example, the image 3202 shows a portion of a wall. Small holes, such as nail holes for hanging pictures, may not be considered material damage for which a tenant should pay in certain embodiments. In other words, variable-sized bounding boxes may be used so that damage is properly identified and assessed. In some embodiments, a standard size bounding box may be used to determine that damage exceeds a particular threshold (e.g., exceeds wear and tear damage).) The trained models in Nelson satisfy “trained classification model,” because they perform categorization, tagging, and image classification. In Nelson, when damage exceeds a threshold it is determined as an issue, “material damage vs wear and tear” for which “a tenant should pay in certain embodiments. Various embodiments in Nelson above satisfy “identifying a reason for the difference,” such as “types of damage, types of objects,” and [104] where a hole is exceeds a particular threshold, the reason is that the hole is larger than the bounding box.
-schedule at least one appointment at the location to remediate at least one issue(Nelson [0117] scheduling maintenance and damage repairs, [0113] In another example, if a tenant submits a maintenance request because their furnace is broken, the landlord may input that the furnace was repaired or replaced so that the value and/or depreciation of the furnace may be accurately calculated according to the various systems and methods described herein.) This limitation is satisfied by Nelson because the broadest reasonable interpretation of the limitation covers any scheduling of an appointment, even if performed manually by a landlord, to remediate (any) issue at the location. The language of “at least one issue,” does not constrain the remediation of the issue to be the particular issue identified in the previous step, therefore Nelson satisfies the limitation as it is stated.
However, Nelson fails to teach:
-receive, from a user computer device associated with a location, a real-time video stream of a location;
- control a camera of the user computer device to capture a plurality of initial images of the location from the real-time video stream;
Alternatively, Chavez discloses a guided inspection software using augmented reality to perform an inspection in a physical space, utilizing machine learning to automatically detect and classify damage to the space, and prompt a user with instructions to move closer to the detected damages. Chavez teaches:
-receive, from a user computer device associated with a location, a real-time video stream of a location; (Chavez [Col. 4 Lines 33-35] Remote device 204 may comprise a device that can be brought to the location where an inspection is to occur. [Col. 5 Lines 2-5] Additionally, in some embodiments, remote device 204 may include a GPS receiver for receiving GPS information that can be used to determine the location of the remote device. [Col. 7 Lines 30-42] As the user is guided through the living space, remote device 204 may capture images (photos or video) of a living space (i.e., a physical space) during step 402. In some cases, remote device 204 may prompt a user to aim the camera and/or take images of one or more physical structures at the location... In some other embodiments, remote device 204 may automatically take pictures or video without prompting a user. Optionally, remote device 204 could prompt a user to aim the camera at a certain area or feature in the room but may take images or videos automatically without further user action. [Col. 10 Line 23-35] In FIG. 7, the user has moved closer to the door resulting in remote device 204 obtaining better quality images of the region just above door 620. At this point the newly obtained image information is sent to centralized computer system 202 for processing and assessment... To clarify what structure is possibly damaged, remote device 204 may display a highlighted boundary 720 around the damage in the live video feed of the area.)
- control a camera of the user computer device to capture a plurality of initial images of the location from the real-time video stream; (Chavez [Col. 7 Lines 42-47] In step 404, remote device 204 sends image information to server 203 of centralized computer system 202 over a network (for example, network 206). The term “image information”, as used herein, refers to any information corresponding to photos or videos [Col. 10 Lines 42-58] It may be appreciated that during this process centralized computer system 202 could provide other kinds of instructions. As one example, if an image processed at centralized computer system 202 is out of focus or not centered sufficiently on a given physical structure, centralized computer system 202 may prepare and send instructions to have images retaken, either manually by a user or automatically by remote device 204. (56) A guided inspection system may also include provisions for detecting when a structure to be inspected is obscured. For example, during an inspection the blinds on a window may be down. This allows the system to determine if the blinds are damaged, but obscures the window itself from view. In that situation a guided inspection system could be configured to automatically detect the obscured window and prompt the user to raise the blinds so the window can be inspected.) Since Chavez teaches that the remote device (user computer device) sends image information (including videos) to the server and receives instructions to perform the additional capturing (including instructions to the user or to the device itself to automatically control the camera) then the limitation has been taught.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present disclosure to modify Nelson by adding the teachings of Chavez including transmitting a live video stream and controlling a camera to perform automatic capturing from the live video feed. One of ordinary skill in the art would have been motivated to perform this combination by the benefit of simplifying the inspection process enabling a user to quickly and accurately assess possible damage. (Chavez [Col. 3 Lines 4-12] By automatically capturing and analyzing image information about structures in the physical space to determine if there is damage, the system and method improve the efficiency of the inspection process. By using an augmented reality system to prompt a user, the system and method simplify the inspection process and allow users with little to no experience in inspecting properties to quickly and accurately assess possible damage in a physical space.)
Regarding Claims 2, 15:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 14 further comprising:
Furthermore, Nelson teaches:
-receiv(ing) a plurality of historical images of a plurality of properties; (Nelson [0087] In addition to images relating to a specific inspection, aggregated images and metadata from other inspections may be sanitized and selected at an operation 2910 for use in training a damage identification model. At an operation 2912, those images may be categorized for contents (e.g., what type of object is in the photographs) and presence of damages (e.g., is there damage shown in the photographs). In other words, images of objects may be manually categorized for their contents and whether damage is shown and used to train the model for identifying damage. [0084] In addition to using training a model to identify and assess damages and using a trained model to identify and assess damages as shown in the method 2900, images captured during inspections may be added to data sets that are used to further refine and train models for identifying and assessing damage in images captured of objects over time.)
-and generating the one or more models based upon the plurality of historical images. (Nelson [0087] In various embodiments, the images used to train such a model may be pairs of images with elapsed time between them (e.g., two images showing a same object with and without damage, respectively), or may be images unrelated to other images used to train the model. In other words, in an example, a model may be trained by inputting into an untrained machine learning model a plurality of pairs of images associated with a plurality of objects, wherein each pair of the plurality of pairs of images comprises an earlier captured image of an example object of the plurality of objects and a later captured image of the example object. In such examples, a machine learning algorithm may be used that can incorporate the relationship between two photographs of the same object taken at different times into its learning. Models for assessing an amount of damages (e.g., a damage pricing model as discussed below with respect to operations 2916, 2926, and 2928 below) may be trained in similar or different ways.)
Regarding Claims 4, 17:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 14 further comprising:
Furthermore, Nelson teaches:
-Transmit(ting) instructions to the user computer device for capturing one or more of the plurality of initial images. (Nelson [0034] In an operation 206, the step-by-step instructions for inspecting the property are displayed on an electronic device (e.g., a mobile device like a smartphone or tablet) so that the user can complete the inspection. Examples of user interfaces that may be used to display step-by-step instructions may include the user interfaces shown in FIGS. 16-27. A user may complete the step-by-step instructions by, for example, answering prompts displayed on a user interface, entering notes related to aspects of a property being inspected, taking photographs of objects on the property, identifying damaged portions of objects in the photographs, etc. as described herein. [0049] Inspection indicator 506 shows an example of an inspection that has yet to be started, and may be started by selecting the inspection indicator 506 (as indicated by the “Start Inspection” text that occurs in the inspection indicator 506. The inspection indicator 506 also indicates the type of inspection being performed, such as an annual or periodic inspection. Other inspection types may also be used, such as move-in, move-out, before-remodel, post-remodel, post-repair, etc., so that the property and its various objects may be properly tracked for proper recordkeeping and tracking over time.) These instructions are shown both during move in and move out, therefore the limitation has been taught.
Regarding Claims 6, 19:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 14 further comprising:
Furthermore, Nelson teaches:
- transmit(ting) instructions to a user computer device associated with a tenant of the location, (Nelson [0034] In an operation 206, the step-by-step instructions for inspecting the property are displayed on an electronic device (e.g., a mobile device like a smartphone or tablet) so that the user can complete the inspection. Examples of user interfaces that may be used to display step-by-step instructions may include the user interfaces shown in FIGS. 16-27. A user may complete the step-by-step instructions by, for example, answering prompts displayed on a user interface, entering notes related to aspects of a property being inspected, taking photographs of objects on the property, identifying damaged portions of objects in the photographs, etc. as described herein. [0049] Inspection indicator 506 shows an example of an inspection that has yet to be started, and may be started by selecting the inspection indicator 506 (as indicated by the “Start Inspection” text that occurs in the inspection indicator 506. The inspection indicator 506 also indicates the type of inspection being performed, such as an annual or periodic inspection. Other inspection types may also be used, such as move-in, move-out, before-remodel, post-remodel, post-repair, etc., so that the property and its various objects may be properly tracked for proper recordkeeping and tracking over time.) These instructions are shown both during move in and move out, therefore the limitation has been taught.
-wherein the instructions identify one or more items to capture in the plurality of current images of at the location. (Nelson [0042] In an operation 306, instructions for capturing a second image may be displayed on the display along with the representation and the first image. Examples of such instructions are shown in FIGS. 19, 21, 23, and 30-37. In FIG. 19, for example, instructions state “Front Door Exterior—While standing outside the front door, take a picture of the entire front door.”)
Regarding Claims 7, 20:
The combination of Nelson and Chavez teach The computer system of Claim 6, /The computer-implemented method of Claim 19:
Furthermore, Nelson teaches:
-wherein the instructions include an overlay to display on a display screen of the user computer device. (Nelson [0096] FIGS. 30 and 31 are example user interfaces 3000 and 3100, respectively, for overlaying a representation of an object onto a user interface of a mobile computing device, in embodiments. For example, such overlays may be used as described in the operation 304 of FIG. 3. The user interface 3000 shows a first image 3002 that represents a field of view of a camera of an electronic device such as a smartphone. Overlaid on the first image 3002 is a representation 3004 of a refrigerator, and in particular an image of a refrigerator. The overlaid representation 3004 may be used to align the field of view with an actual refrigerator that may appear in the first image 3002 if the actual refrigerator is within the field of view of the camera (though not shown in FIG. 30). In some embodiments, the overlaid representation may be a prior image captured of the actual refrigerator associated with a property. For example, if a property was inspected a year prior to a current inspection, prior photos of objects taken during the prior inspection may be modified to be partially transparent and used as overlays for images captured for the current inspection. As described herein, the overlaid representations may also cause users to capture more consistent and/or better-quality images that may more easily be used to identify and/or assess damage in the captured images.)
Regarding Claims 8, 21:
The combination of Nelson and Chavez teach The computer system of Claim 6, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 19 further comprising:
Furthermore, Nelson teaches:
-Control(ling) the user computer device to capture the plurality of current images. (Nelson [0067] For example, in FIG. 19, instructions 1904 overlaid onto the first image 1902 explain to the user what to take a picture of. While the user interface 1900 does not show a representation of the front door for aligning a photograph, such a representation may optionally be overlaid as described herein. A user may press a button 1906 to capture a second image that coincides with the first image currently being displayed at the time the button 1906 is pressed. In the alternative, the user may select a skip button 1908 to skip capturing an image at this stage.) The broadest reasonable interpretation (BRI) of this limitation in the plain language that the claim is written includes any control of the user device to capture the initial images, such as pressing a button which then indicates the computing device to capture the images. In view of the specification paragraph [0056], the limitation does not necessarily require an augmented reality overlay, or automatic capture of the images, since the claim language is so broad.
Regarding Claims 9, 22:
The combination of Nelson and Chavez teach The computer system of Claim 6, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 19 further comprising:
Furthermore, Nelson teaches:
- transmit(ting) the instructions to the user computer device of the tenant on a periodic basis. (Nelson [0049] The inspection indicator 506 also indicates the type of inspection being performed, such as an annual or periodic inspection. [0051] FIG. 6 shows a user interface 600 for a periodic inspection, for example if an inspection indicator in FIG. 5 is selected. The user interface 600 includes a back arrow 602 that may navigate the user back to the user interface 500 of FIG. 5.)
Regarding Claims 11, 24:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 14 further comprising:
Furthermore, Nelson teaches:
- receiv(ing) a plurality of sets of current images of a first location over a plurality of periods of time; and (Nelson [0023] As described herein, if multiple inspections of a property are performed, the images or other documentation related to those inspections may be analyzed over time to automatically identify changes to the property or objects on the property to determine a material wear and tear depreciation or specific damage value that exceeds wear and tear associated with a property, building, or object on the property. [0074] In addition, other images may be captured of those objects or aspects of a property during subsequent inspections. As described in the method 2800 those subsequent images of a same object may be compared to determine changes to the object, which may be wear and tear due to normal use and passage of time, or may be material damage for which a tenant should be liable for repairs or replacement of the object.)
-compar(ing) the plurality of sets of current images of the first location to determine at least one trend at the first location. (Nelson [0094] As described herein, a machine learning model may also be trained to determine an estimated value associated with the damage to the object. That value may be associated with wear and tear (e.g., a depreciation of the object) or may be associated with material damage for which a tenant should be liable. The material damage value may represent a cost to repair or replace the object. In various embodiments, a landlord or property manager may input the value of various objects when they are installed in a property. In this way, the systems and methods herein may be used to track the value of those objects as they are inspected over time. For example, a washing machine may be purchased for $500, and the systems and methods herein may indicate during an annual inspection that such a washing machine depreciates approximately $50 a year. In other examples, various objects may depreciate at varying rates over time. Thus, a value remaining of an object may be used to determine a replacement cost of the washing machine if a tenant damages it beyond repair, or the new cost of the object may be used as the replacement cost. For example, if the washing machine described above is damaged beyond repair after three years of use, the system may recommend a replacement cost for which a tenant is liable of $350.) Estimating the value overtime falls within the scope of the limitation since it is a determination of a “trend.”
Regarding Claims 12, 25:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 14 further comprising:
Furthermore, Nelson teaches:
-receiv(ing) a plurality of sets of current images of a first location over a plurality of periods of time; and (Nelson [0074] In addition, other images may be captured of those objects or aspects of a property during subsequent inspections. As described in the method 2800 those subsequent images of a same object may be compared to determine changes to the object, which may be wear and tear due to normal use and passage of time, or may be material damage for which a tenant should be liable for repairs or replacement of the object.)
-calculat(ing) a score for a tenant at the first location based upon the plurality of sets of current images of the first location. (Nelson [0112] Other features related to the inspection systems and methods described herein may also be implemented in various embodiments. For example, the systems and methods herein may be used to determine a score for any of landlords, property managers, and/or renters. For example, parties may rate other parties they interact with based on their experiences. For example, a renter may rate their landlord and vice versa. In addition, the parties’ scores may be adjusted based on data collected and/or entered related to inspections. For example, if after an inspection, an object is determined to be damaged beyond wear and tear, a renter’s score may be adjusted downward based on damage to the object during the time in which the renter occupied the property. In some embodiments, an extent of damage to a property may be used to adjust a renter score up or down, such that a renter score may improve or decrease over time depending on how well the renter took care of properties they rented. Thus, a renter score may indicate how likely it is that the renter will cause more than wear and tear damage to a property.)
Regarding Claims 13, 26:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 14 further comprising:
Furthermore, Nelson teaches:
-receiv(ing) a plurality of sets of current images from one or more locations associated with a landlord; and (Nelson [0074] In addition, other images may be captured of those objects or aspects of a property during subsequent inspections. As described in the method 2800 those subsequent images of a same object may be compared to determine changes to the object, which may be wear and tear due to normal use and passage of time, or may be material damage for which a tenant should be liable for repairs or replacement of the object.)
-calculat(ing) a score for the landlord based upon the plurality of sets of current images of the one or more locations.( Nelson [0112] Other features related to the inspection systems and methods described herein may also be implemented in various embodiments. For example, the systems and methods herein may be used to determine a score for any of landlords, property managers, and/or renters. [0117] Landlord ratings may also be adjusted based on condition of property and objects on the property as determined by automatic image analysis as described herein.)
Regarding Claim 28:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to:
Furthermore, Nelson teaches:
-determine a predefined period of time has elapsed since a plurality of previous images of the location were captured; and(Nelson [0083] In various embodiments, the automated system for identifying and assessing damage in an image may consider an amount of elapsed time between when a first image was captured of an object and when a second image of the object was captured. In addition, renters and landlords may choose to each perform an inspection around the same time (e.g., around move-in, move-out, etc.). Inspections performed within a predetermined threshold of time (e.g., within a day, within a week, within a month) of one another may be aligned inspections.)
-transmit a prompt to the user computer device to initiate the capture of the plurality of current images. (Nelson [0049] Inspection indicator 506 shows an example of an inspection that has yet to be started, and may be started by selecting the inspection indicator 506 (as indicated by the “Start Inspection” text that occurs in the inspection indicator 506. The inspection indicator 506 also indicates the type of inspection being performed, such as an annual or periodic inspection. Other inspection types may also be used, such as move-in, move-out, before-remodel, post-remodel, post-repair, etc., so that the property and its various objects may be properly tracked for proper recordkeeping and tracking over time. As also shown in the user interface 500, other inspection indicators may indicate inspections at various states of completion (e.g., “Continue Inspection”). Other state indicators could be “In-Progress,” “Completed,” “Not Started,” etc. [0018] A user of a mobile application may be presented with a number of prompts based on the type of property being inspected. At each prompt, the user may be asked to provide a response to said prompt, which may be an answer to a question or a picture taken using the camera of the mobile device.)
Regarding Claim 29:
The combination of Nelson and Chavez teach The computer system of Claim 28, wherein the at least one processor is further programmed to:
Furthermore, Nelson teaches:
- transmit further prompts to the user computer device according to a predefined periodicity, each further prompt requesting the initiation of capture of a respective plurality of periodic current images;(Nelson [0119] For example, inspections may be designed to focus more on items that are determined to be more likely to be damaged by a renter. In another example, more frequent and brief inspections may focus only on problem areas identified from aggregated data, while less frequent and exhaustive inspections may inspect and document all aspects of a property. Thus, the most problematic areas of a property that are likely to be damaged may be inspected more often. [0049] (as indicated by the “Start Inspection” text that occurs in the inspection indicator 506. The inspection indicator 506 also indicates the type of inspection being performed, such as an annual or periodic inspection. Other inspection types may also be used, such as move-in, move-out, before-remodel, post-remodel, post-repair, etc., so that the property and its various objects may be properly tracked for proper recordkeeping and tracking over time.)
-receive, in response to each further prompt, the respective plurality of periodic current images; and(Nelson [0082] Although the method 2800 describes comparing two images, additional images may be considered and/or compared by the system. For example, additional images taken at different times from the first and second images may be compared to determine damage and/or wear and tear to an object over time. In other examples, the system may be configured to receive multiple contemporaneously taken images of an object to compare to one or more images of the object taken at a different time. For example, multiple images of an object may be taken from different angles, and those images may be used to compare to one or more images of the objects captured at a different time.)
-store each plurality of periodic images linked to a timeline of when the respective plurality of periodic images was captured, relative to one another and to the plurality of current images.(Nelson [0092] In such embodiments, times at which each of the images in a pair of images were taken or an elapsed time indicating the time passed between the earlier captured image and the later captured image of the image pairs may also be input. This, together with other inputs such as damage value, type of damage, object type, etc., may train the model to more accurately identify and assess damage in images based on an amount of time that has passed between the capture of images... Accordingly, identifying objects in images and tracking them over time may be helpful in assessing the extent of and value of damage. Thus, inputting images with data indicating their relationship to one another (whether two images show the same object and the time passed between the capture of the images) may be helpful in adequately identifying and assessing damages. Thus, when inspections are performed as described herein, it may be valuable to determine, either through manual tagging, automatic tagging based on image recognition, or any other method, when images taken at different times are of the same object. In this way, a trained model may be able to better track the extent and value of damage sustained to various objects. Thus, the methods and systems herein may accommodate inputs from a user to indicate when objects are replaced so that the system may begin the timeline for tracking damage to an object again. [0036] In an operation 210, the images captured are timestamped and stored. The images may be timestamped at or near the moment the images are captured. The images may be stored in memory of a device which captured the images. In various embodiments, the images along with their timestamps may be stored in other devices such as servers. [0049] Other inspection types may also be used, such as move-in, move-out, before-remodel, post-remodel, post-repair, etc., so that the property and its various objects may be properly tracked for proper recordkeeping and tracking over time)
Regarding Claim 31:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to:
However, Nelson fails to teach:
-receive audio information from the real-time video stream; and
-analyze the audio information to detect one or more further issues at the location.
Alternatively, Chavez teaches:
-receive audio information from the real-time video stream; and(Chavez [Col. 4 Lines 44-58] Still further, remote device 204 can include speakers and a microphone for receiving and generating audible sounds. In the exemplary embodiment of FIG. 2, remote device 204 comprises a tablet computing device. In other embodiments, however, a remote device could comprise a smartphone, a laptop, or similar kind of device. (22) Remote device 204 may include hardware components for capturing sensory information, as well as storing and/or transmitting captured information. As used herein the term “sensory information” can include visual information, audible information, tactile information and/or information related to the motion of the remote device (for example, acceleration information). In an exemplary embodiment, remote device 204 includes a camera for capturing images in the form of photos or video.)
-analyze the audio information to detect one or more further issues at the location. (Chavez [Col. 12 Lines 14-27] To detect and classify structures and/or damage, the embodiments may utilize a machine learning system. As used herein, the term “machine learning system” refers to any collection of one or more machine learning algorithms. Some machine learning systems may incorporate various different kinds of algorithms, as different tasks may require different types of machine learning algorithms. Generally, a machine learning system will take input data and output one or more kinds of predicted values. The input data could take any form including image data, text data, audio data or various other kinds of data. The output predicted values could be numbers taking on discrete or continuous values. The predicted values could also be discrete classes (for example, a “damaged” class and an “undamaged” class).)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present disclosure to modify Nelson by adding the teachings of Chavez including using the audio from the video stream as another source of input data to determine one or more issues. One of ordinary skill in the art would have been motivated to perform this combination by the benefit of using more sources of information to make a more accurate detection of damage (Chavez [Col. 3 Lines 4-12] By automatically capturing and analyzing image information about structures in the physical space to determine if there is damage, the system and method improve the efficiency of the inspection process. By using an augmented reality system to prompt a user, the system and method simplify the inspection process and allow users with little to no experience in inspecting properties to quickly and accurately assess possible damage in a physical space.)
Claims 10, 23, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Nelson (US 20200410278 A1), in view of Chavez (US 11600063 B1), further in view of Jorey E. Ramer (US 20230141593 A1) hereinafter Ramer.
Regarding Claims 10, 23:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to/The computer-implemented method of Claim 14 further comprising:
However, neither Nelson nor Chavez teach or suggest:
-schedule the at least one appointment with a service provider to resolve the one or more issues. (Both Nelson and Chavez merely teach the inspection of the location but not the process of resolving any detected problems)
Alternatively, Ramer discloses a system of sending service calls to maintenance and repair services based on sensor data. Ramer teaches:
-schedule the at least one appointment with a service provider to resolve the one or more issues (Ramer [0070] In example embodiments, the data associated with the entities at each property of each subscriber may be sensor data sent from one or more entities at the properties. In example embodiments, the prediction engine may analyze the sensor data to determine whether the entities may be in need of repair or servicing...[0239] When the action type is “proactive execution,” the sensor management system 250 may configure the platform 10, in example embodiments, to automatically schedule a service visit. [0334] The claim adjudication system 260 may receive the request for repair service, and may automatically adjudicate the request. For this purpose, in one example, the system 260 may associate a job with the service request and designate an appropriate servicer to perform the job. [0335] Additionally and/or alternatively, the claim adjudication system 260 may also pre-authorize or automatically enable the servicer to perform the repair service at the home, without the servicer requiring manual approval to perform the repair.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the present disclosure to modify Nelson by adding Ramer’s features of automatically determining and predicting what services are needed from sensor data and automatically scheduling a service visit in response to the detection of a problem. Nelson’s system already determines a type of damage and predicted cost of damage, therefore a combination would yield the predictable outcome of contacting the appropriate service provider based on the damage. One of ordinary skill in the art would have been motivated to combine as it would provide the benefit of automatically finding high quality repair services at fair prices and with predictable costs (Ramer [0009] As a result, a need exists for methods and systems that improve the consumer/homeowner experience of scheduling repair services, such as by assuring the provision of available, high quality repair services at fair prices and with predicable costs.)
Regarding Claim 30:
The combination of Nelson and Chavez teach The computer system of Claim 1, wherein the at least one processor is further programmed to:
However, neither Nelson nor Chavez teach or suggest:
-prioritize each issue based on a respective urgency of the issue; and
-schedule the at least one appointment including a first appointment to address a first priority issue.
However, Ramer teaches:
-prioritize each issue based on a respective urgency of the issue; and(Ramer [0237] Upon finding a matching alert message in the action table based upon the sensor data 321, the prediction engine 252 may include the contents of the alert in a message, may send the message to various stakeholders, and/or execute a set of actions in response. The actions to take may incorporate varying levels of urgency and an action type (e.g., recommendation or proactive execution). Example actions may include recommending scheduling of a service job on a next visit, recommending scheduling a visit within a time period (e.g., within the next two weeks), recommending an immediate service call, recommending an action pending a service call (e.g., stopping usage of a potentially damaged item to prevent further harm, replacing an element (e.g., a filter or battery, or the like), and/or recommending complete replacement of an item, among others. [0254This may include adjusting the weights of a set of parameters in a scheduling rule, such as to rank order a set or servicers for a job.)
-schedule the at least one appointment including a first appointment to address a first priority issue.(Ramer [0237] Example actions may include recommending scheduling of a service job on a next visit, recommending scheduling a visit within a time period (e.g., within the next two weeks), recommending an immediate service call,) These examples of recommending an immediate service call, or scheduling a job on the next visit satisfy “scheduling...to address a first priority issue.”
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present disclosure to further modify Nelson and Chavez by adding Ramer’s features of prioritizing issues based on levels of urgency, and scheduling an immediate appointment for urgent items. By adding Ramer’s system at the end of Nelson’s system, one would arrive at the predictable outcome of performing the limitations of claim 30 because by the end of Nelson the problem is already identified and Ramer merely performs the prioritization and scheduling steps. One of ordinary skill in the art would have been motivated to perform the combination by the benefit of preventing further harm, and improving the efficiency of scheduling an appointment with a servicer. (Ramer [0129] Where the host system 100 is permitted to schedule the first appointment of the window, it can determine if an actual appointment time has an effect on customer satisfaction. If the host system 100 schedules most or all of the appointments within a window, it can determine the most efficient routing, and it can use time estimation to reduce the appointment window (or provide an actual appointment time) for a consumer.)
Response to Arguments
Applicant's arguments filed 01/28/2026 have been fully considered.
Regarding applicant’s arguments over 35 U.S.C. 112(a), the applicant has provided the support for the limitation “control a camera of the user computer device to capture a plurality of initial images of the location from the real-time video stream” in [0033]. Since it would have been conventional or well known to one of ordinary skill in the art at the time of the effective filing date, that a camera of a user computer device can be controlled to capture images from a real-time video stream, it is not required for such limitation to be disclosed in detail. MPEP 2163(3), "The ‘written description’ requirement must be applied in the context of the particular invention and the state of the knowledge…. As each field evolves, the balance also evolves between what is known and what is added by each inventive contribution." Therefore, in view of the applicant’s citation to the appropriate specification paragraph, the rejection under 35 U.S.C. 112(a) has been withdrawn.
In regards to applicant’s arguments over 35 U.S.C. 101 rejections, the applicant alleges that the reasons that made the “ARP” decision eligible are applicable to the present claims. However, the examiner respectfully disagrees. Firstly, the applicant’s point regarding the ARP decision identifying “improvements in training the machine learning model itself,” such as AI systems to “use less of their storage capacity,” and “enables reduced system complexity” have been considered. The examiner notes that the ARP decision at hand (Ex Parte Desjardins) results in the following examples that may show an improvement in computer functionality, “xiii. An improved way of training a machine learning model that protected the model’s knowledge about previous tasks while allowing it to effectively learn new tasks; Ex Parte Desjardins, Appeal No. 2024-000567 (PTAB September 26, 2025, Appeals Review Panel Decision) (precedential); and xiv. Improvements to computer component or system performance based upon adjustments to parameters of a machine learning model associated with tasks or workstreams; Ex Parte Desjardins, Appeal No. 2024-000567 (PTAB September 26, 2025, Appeals Review Panel Decision) (precedential).” However, the claims merely recite “executing a trained image comparison model of the one or models to compare the plurality of initial images to the plurality of current images” and “define the difference as an issue by executing a trained classification model of the one or more models to identify a reason for the difference.” In the current scope of the claims, there is nothing about how these models are trained, nor that they improve computer components based on the adjustments of parameters of machine learning models. Therefore, there is no improvement to artificial intelligence in the claims because there are no artificial intelligence systems being claimed, nor are they being used to “use less of their storage capacity,” or “enable reduced system complexity.” Furthermore, the examiner has considered the applicant’s comments regarding “Examiners and panels should not evaluate claims at such a high level of generality.” The examiner notes that the claims have been evaluated at their broadest reasonable interpretation, and whether exact language of the claims reflects the alleged improvement.
On page 12 of the applicant’s remarks the applicant cites to paragraphs [0002] to [0004] to point to the technical problems in “image labeling and processing is resource intensive,” and that “review of property damage requires manual review and subjective evaluation.” The examiner acknowledges these citations, noting that improvements to the resource consumption of “image labeling and processing” may be a consideration, but an improvement to “review of property damage,” is an improvement to the abstract idea. MPEP 2106.05(a) states, “Notably, the court did not distinguish between the types of technology when determining the invention improved technology. However, it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.” Furthermore, the applicant cites to [0019] and [0020] of the specification to identify technical solutions, however, there are no apparent improvements to “deep artificial intelligence (AI) image comparison models” purported in the specification, even in [0038] and [0039], because there are no teachings in the specification regarding an improvement to the technological field of artificial intelligence itself (whether that is through an improved algorithm, or computational process, or an improvement to computer functionality lent by the adjustment of machine learning parameters.) The specification merely assert the improvement without any accompanying details necessary to be considered an improvement at the time of the effective filing date. MPEP 2106.05(a) states, “Conversely, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology. An indication that the claimed invention provides an improvement can include a discussion in the specification that identifies a technical problem and explains the details of an unconventional technical solution expressed in the claim, or identifies technical improvements realized by the claim over the prior art.” Therefore, the applicant’s arguments that the claims reflect improvements to image analysis and damage assessment are not persuasive because they do not reflect the level of detail necessary to make it apparent to one of ordinary skill in the art of improvements to artificial intelligence, at least not at the same level of the eligible claims in the ARP decision or Ex Parte Desjardins. Furthermore, “receiving images of the location at difference times and processed using an evaluation model to identify differences that are only further examined when the differences exceed a threshold” does not satisfy the level of particularity required to determine that the claim improves technology. MPEP 2106.05(a) states, “An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome. McRO, 837 F.3d at 1314-15, 120 USPQ2d at 1102-03; DDR Holdings, 773 F.3d at 1259, 113 USPQ2d at 1107. In this respect, the improvement consideration overlaps with other considerations, specifically the particular machine consideration (see MPEP § 2106.05(b)), and the mere instructions to apply an exception consideration (see MPEP § 2106.05(f)). Thus, evaluation of those other considerations may assist examiners in making a determination of whether a claim satisfies the improvement consideration.” Furthermore, MPEP 2106.05(f) states, “Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".”
The claims only recite the idea or outcome of “identifying differences that are only further examined when the differences exceed a threshold,” and fail to recite a particular details or description of the mechanism that determines how the model or algorithm determines whether that threshold has been satisfied. Even assuming that the claims defined the threshold and how it is accomplished, it still would not reflect an improvement to Artificial Intelligence, because generic use of artificial intelligence to carry out the abstract idea does not qualify as a technical improvement. The concept of determining a threshold to further examine differences may provide a reduction in computing resources, however, this is merely reducing the amount of information being processed by using an arbitrary cut-off point. At the level of detail recited in the claims, it is not clear that this is an improvement lent by the artificial intelligence algorithm or any other technology because the claims do not recite any particular improved technique in the field of artificial intelligence. Therefore, the arguments that the claims improve the accuracy and objectivity of property assessment are not persuasive for the reasons above. Furthermore, the applicant, under the section in the remarks titled A. The Pending Claims Are Not Directed to An Abstract Idea, repeats more of the same points provided in the remarks dated 10/10/2025, therefore, only the new arguments are revisited as follows. The applicant’s argument in page 15 regarding “performing inspections” is not recited anywhere in the present claims, and “a claim that may facilitate business relations has no bearing on what the claim actually recites, and substantially any activity could be considered to facilitate business relationships in some context” has been considered. Upon revisiting the claims as amended, the updated rejection has been updated to reflect the abstract idea category now includes “managing personal behavior or interactions or relationships between people,” because the claims are recited at a high level of generality such that they are no more than mere instructions to an individual to manage their personal behavior. Furthermore, the steps are directed to data collection, data analysis, and data output, wherein the data analysis steps are recited a level of generality that they could be performed by a human with a pen and paper. Furthermore, by the final step of “schedule at least one appointment at the location to remediate at least one issue.” It is clear that the claims are not just “construed” to involve or facilitate “business relationships” but are directly reciting a business relationship in and of itself. Therefore, the applicant’s argument regarding “this allegation regarding what may be performed after a claimed method is not relevant to the actual limitations of the claim,” is not persuasive because within the claim itself exists at least a business relation, and more recitations which fall under at least one abstract idea category.
The applicant’s argument in page 16, “When taken together, these features amount to a specific practical and technical solution to image analysis and damage assessment...that in no way preempts the entire alleged judicial exception.” However, this argument is not persuasive for the same reasons provided above regarding the arguments over Desjardins. Furthermore, MPEP 2106.04(I) states, “questions of preemption are inherent in and resolved by the two-part framework from Alice Corp. and Mayo (the Alice/Mayo test referred to by the Office as Steps 2A and 2B)” therefore, the applicant’s argument regarding the combination of features not preempting the entire judicial exception is not persuasive because the claim has been viewed as a whole under the full two-part framework to consider that they are directed to an abstract idea without significantly more.
The applicant’s further notes regarding the August 4, 2025 memorandum, particularly that the additional elements, “should be evaluated in a vacuum completely separated from the judicial exception.” The examiner has considered all of these concerns when performing the evaluation and affirms that even after viewing the claims as a whole, including the additional elements in combination, the claims still fail to provide an integration into a practical application or significantly more, established by a preponderance of the evidence against the applicant’s bare assertions of an improvement to a technical field. Therefore, none of the arguments over step 2a Prong 2 are persuasive.
Regarding applicant’s arguments under B. Applicant’s Claims Are Directed to “Significantly More” than the abstract idea, the applicant repeat some arguments from the 10/10/2025 remarks that have already been addressed. Regarding the applicant’s arguments that the claims recite more than “well-understood, routine, conventional activity,” these arguments remain unpersuasive even when considering the amended claims because the rejection does not rely on an assertion in Step 2B that requires the examiner to show factual support for “well-understood, routine, or conventional activity.” Since the examiner has not labeled any of the activity as “insignificant extra-solution activity” that is well-known, and the claims are rejected with the considerations under MPEP 2106.05(f), and MPEP 2106.05(a), there is no requirement by the examiner to further to provide a factual determination. Therefore, the applicant’s arguments that the claims recite a combination of limitations that operate in a non-conventional and non-generic way to compare images of a location over time and assess a limited subset of those images to identify sources of damage for remediation is not persuasive because the claims do not reflect an improvement to any particular technological environment, nor do they recite the details to arrive at the proposed solution at the necessary level of specificity. Furthermore, the claims do not overcome prior art, therefore, the applicant’s arguments regarding the pending claims overcoming the prior art is neither relevant nor persuasive because it is not a consideration in the 101 rejection.
Regarding the applicant’s arguments over 35 U.S.C. 103, the applicant asserts, “neither references describes a computer device configured to designate differences in images as “issues” only when they exceed a threshold, execute a trained classification model only for those issues, and automatically schedule appointments to remediate those identified issues.” However, this argument is not persuasive because Nelson teaches or suggests :
-“wherein an output from the trained image comparison model includes an identification of one or more differences between the plurality of initial images and plurality of current images;(Nelson [0080] [0084])”,
-“based on the output, determine whether the one or more differences exceed a predetermined threshold;(Nelson [0088] particular threshold indicating material damage. [0093] [0102] [0122])
-“for each difference that exceeds the predetermined threshold, define the difference as an issue by executing a trained classification model of the one or more models to identify a reason for the difference.(Nelson [0088] [0089] [0093] [0090] types of damage, types of objects, [0111] [0102])
Furthermore, the examiner notes that Nelson does not explicitly disclose, “automatically schedule appointments to remediate those identified issues.” In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., automatically schedule appointments to remediate those identified issues) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The claim merely requires, “schedule at least one appointment at the location to remediate at least one issue.” The broadest reasonable interpretation of this limitation does not require the scheduling to happen subsequent to the issue being identified, nor does it require the scheduling to be performed “automatically,” and it does not require the appointment to remediate “those identified issues,” because it only requires them to remediate “at least one issue.” Even assuming arguendo, that the claims did require, “automatically schedule appointments to remediate those identified issues,” such an amendment would likely be rendered obvious by the combination of Nelson, Chavez, and Ramer. Furthermore, Ramer is not relied on to teach, “designate differences in images as “issues” only when they exceed a threshold, execute a trained classification model only for those issues, and automatically schedule appointments to remediate those identified issues.”
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Therefore, since the prior art of record in combination teaches each and every limitation of claims 1, 2, 4-15, 17, and 19-31, the claims stand rejected under 35 U.S.C. 103.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
-Carone et al. (US 11055797 B1) discloses obtaining previously determined sensor data for an object, collecting a present status associated with particular objects based on determining the difference between a first set and second set of image data.
- Buentello et al. (US 12394031 B1) discloses a detection of property condition overtime using LIDAR sensors and a set of instructions to laymen to carry out data collection of particular areas of the property.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICO LAUREN PADUA whose telephone number is (703)756-1978. The examiner can normally be reached Mon to Fri: 8:30 to 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jessica Lemieux can be reached at (571) 270-3445. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICO L PADUA/ Junior Patent Examiner, Art Unit 3626
/ASFAND M SHEIKH/ Primary Examiner, Art Unit 3626
Prosecution Insights