DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendment
Applicant’s Amendment filed on 4/20/2026 has been entered and made of record.
Currently Pending claims: 1-20
Independent claims: 1, 11, and 20
Amended claims: 1, 5, 9, 11, 15, 19, and 20
Response to Arguments
This office action is responsive to Applicant’s Arguments/Remarks Made in an Amendment received on 4/20/2026.
Applicant’s arguments regarding objections to the drawings, specification, and claim 15, see page 12 of the Remarks, received on 4/20/2026, have been fully considered and are persuasive. The objections to the drawings, specification, and claim 15 have been withdrawn.
Applicant’s arguments regarding rejections under 35 U.S.C. 101, see pages 13-16 of the Remarks, received on 4/20/2026, have been fully considered and are persuasive. The rejections under 35 U.S.C. 101 have been withdrawn.
Applicant’s arguments regarding rejections under 35 U.S.C. 102 with respect to claims 1-3, 5, 6, 11-13, 15, 16, and 20, see pages 16-17 of the Remarks, received on 4/20/2026, have been fully considered and are persuasive. The rejections have been withdrawn. However, upon further consideration, a new ground of rejection, as necessitated by amendment, is made in regards to claims 1-3, 5, 6, 11-13, 15, 16, and 20 in view of US 12,288,399 to Sandoval et al., US 2019/0251352 to Hovden et al., and US 2024/0144343 to Kim.
Applicant’s arguments regarding rejections under 35 U.S.C. 103 with respect to claims 4, 7-10, 14, and 17-19, see page 18 of the Remarks, received on 4/20/2026, have been fully considered and are considered moot as they rely upon the claims’ dependency on claims 1 and 11, which have been rejected under a new ground of rejections as noted above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5, 6, 8, 9, 11-13, 15, 16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sandoval et al. (US 12,288,399) (hereafter, “Sandoval”) in view of Hovden et al. (US 2019/0251352) (hereafter, “Hovden”) and further in view of Kim (US 2024/0144343).
Regarding claim 1, Sandoval discloses a computer system for hierarchical model analysis (Col. 8, lines 22-26, In addition, the model management engine 210 can
use the output of models to determine which sensor data classification models 220 are to be
activated. For example, if one model detects a license plate, a face detection model might be
activated. Examiner interprets the activation of one model depending on the outcome of a first model as “hierarchical model analysis”), the computer system comprising at least one processor in communication with at least one memory device (Col. 4, lines 24-35 processors 52 and data storage 56), wherein the at least one processor programmed to receive a plurality of images (Col. 4, lines 50-56, the camera 110, …, can collect images of the environment in the vicinity of the rig 105. The camera 110 can capture images including, in this example, a distressed child 105A, a suspicious party 105B near the distressed child 105A, and the license plate on a vehicle 107; Col. 6, lines 18-20, Returning to the example 100 illustrated in FIG. 1, image recognition models 120 in the edge platform 101 can process the image data produced by the camera 110. Examiner interprets the images being processed to indicate that they were “received”); and for each image of the plurality of images, retrieve an image of the plurality of images (Col.6 and Col. 7, provided data, received from edge platform 101, Col. 8 lines 59-61 Sensor data classification models 220 can also provide additional information about the sensor data and include that data in the classification output data 222; processing of individual images from a plurality of images is an inherent feature of the system described in Sandoval); execute a hierarchy of models with the retrieved image as input (Col. 7, line 67, examples of sensors 205 can include cameras; Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated; Col. 8, lines 35-37, The model management engine 210 can accept sensor data 207 from sensors 205 and direct the sensor data 207 to the appropriate, active sensor data classification models 220; Examiner interprets data from cameras as “image” and the activation of one model depending on the outcome of a first model as “a hierarchy of models”), the hierarchy of models comprising [an inside/outside analysis model] and a plurality of additional models (Col. 8, lines 35-37, The model management engine 210 can accept sensor data 207 from sensors 205 and direct the sensor data 207 to the appropriate, active sensor data classification models 220. Examiner interprets the active sensor data classification models as “additional models”), wherein executing the hierarchy of models comprises: routing the retrieved image to [the inside/outside] analysis model (Col. 6, lines 18-21, Returning to the example 100 illustrated in FIG. 1, image recognition models 120 in the edge platform 101 can process the image data produced by the camera 110 to produce classification output data 125. Examiner considers the image recognition model using the image data to indicate the data has been “routed” to the model) [to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure]; receiving an output from the [inside/outside] analysis model indicating that the retrieved image is (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers the image recognition model using the image data to indicate the data has been “routed” to the model) [one of the inside image or the outside image]; [when the output indicates that the retrieved image is the inside image], routing the retrieved image to at least one model of the plurality of additional models (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Sandoval discloses routing data to additional models based on the outcome of a first model) [configured to analyze inside images]; and [when the output indicates that the retrieved image is the outside image], routing the retrieved image to one or more models of the plurality of additional models (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers activating the face detection model to indicate “routing” data to said model) [configured to analyze outside images]; output classification information for the retrieved image based upon the execution (Col. 8, lines 53-55, each sensor data classification model 220 can be configured to produce a particular type classification output data 222); and associate the classification information with the retrieved image (Col. 8 line 67 – Col. 9 line 1-3, the classification output data 222 can further include an indication of the sensor data 207, or a subset of the sensor data 207, used to produce the predicted classification. Examiner interprets indicating sensor data used to produce the classification as “associate”).
However, Sandoval fails to explicitly disclose an inside/outside analysis model; to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure; when the output indicates that the retrieved image is the inside image, route to a model configured to analyze inside images; and when the output indicates that the retrieved image is the outside image, route to a model configured to analyze outside images.
Hovden teaches an inside/outside analysis model (The image classification component 1906 can also employ one or more machine learning classification models to facilitate automatically classifying images as interior or exterior images of a structure. Examiner interprets the image classification component as the “inside/outside analysis model”); to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure (¶0141, The image classification component 1906 can also employ one or more machine learning classification models to facilitate automatically classifying images as interior or exterior images of a structure); when the output indicates that the retrieved image is the inside image (¶0141, the selection component 1904 can be configured to select exterior images for processing; ¶0146, GPS coordinates matched with capture locations of interior scan images. Examiner considers the separation of exterior images and matching of interior images to GPS coordinates to indicate separate analyses performed on exterior and interior images), [route to a model configured to analyze inside images]; and when the output indicates that the retrieved image is the outside image (¶0141, the selection component 1904 can be configured to select exterior images for processing), route to a model configured to analyze outside images (¶0142, The exterior perspective component 1908 can determine an optimal perspective of an exterior scan image. Examiner interprets the exterior perspective component as a model “configured to analyze outside images”).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
However, neither Sandoval nor Hovden, whether considered individually or in combination, explicitly disclose a model configured to analyze inside images.
Kim teaches a model configured to analyze inside images (¶0084, when the query image received from the user terminal 100 is captured for a living room and an interior style of a query image is a modern style, the information providing apparatus 200 may transmit information on an accessory included in reference image E classified as the living room and the modern style; ¶0108, the image analysis unit 1030 may store a DNN for image analysis).
Sandoval, Hovden, and Kim are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model, Hovden is directed to image classification of the inside and outside of structures, and Kim is directed to image classification of inside images. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside image model of Kim into the inside/outside analysis model of Hovden and the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to increase the likelihood of a user making a purchase, as suggested by Kim at ¶0122, it is possible to activate a user's purchase of accessories.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden and Kim.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden and Kim to obtain the invention as specified in claim 1.
Regarding claim 2, in which claim 1 is incorporated, Sandoval discloses where in the at least one processor is further programmed to generate a report (Col. 4, lines 27-28, process the data to make it recordable, reportable) for the plurality of images based upon the plurality of associated classification information (Col. 10, lines 34-37, the interaction engine 280 can provide results 285, … composite data (e.g., data aggregated from multiple detection data 232)).
Regarding claim 3, in which claim 1 is incorporated, Sandoval discloses wherein the hierarchy of models (Col. 8, lines 22-26, in addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated; Examiner interprets the activation of one model depending on the outcome of a first model as “a hierarchy of models”) includes a plurality of classification models (Col. 8, lines 14-15, such as sensor data classification models 220A, 220B, 220C), each trained to identify one or more items in an image (Col. 6, lines 21-25, the classification output data 125 can include indications of detected elements of the image data. The classification output data 125 can include an indication that a license plate, tail number or face was detected. Examiner considers license plates, tail numbers, and faces as “items”).
Regarding claim 5, in which claim 3 is incorporated, Sandoval discloses wherein the at
least one processor is further programmed to: route the retrieved image (Col. 7, line 67, examples of sensors 205 can include cameras; Col. 8, lines 38-40, the model management engine 210 can direct the sensor data 207 to the 40 active sensor data classification model. Examiner considers sensor data as “the retrieved image”) to a first classification model (Col. 8, lines 25-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers the model that detects the license plate as the “first classification model”) of the plurality of classification models (Col. 8, lines 14-15, such as sensor data classification models 220A, 220B, 220C) in the hierarchy of models (Col. 8, lines 22-26, in addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated; Examiner interprets the activation of one model depending on the outcome of a first model as “a hierarchy of models”), [the first classification model comprising the inside/outside analysis model]; execute the first classification model using the retrieved image as the input (Col. 8, 25-26, for example, if one model detects a license plate. For the model engine to be aware a license plate is detected, the license plate model must be executed); and receive one or more classifications from the first classification model based upon the retrieved image (Col. 8, lines 22-26, in addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers the license plate model as the “first classification model”. For the model engine to be aware a license plate is detected, the license plate model must be executed, and output the classification).
However, Sandoval fails to explicitly disclose the first classification model comprising the inside/outside analysis model.
Hovden teaches the first classification model comprising the inside/outside analysis model (The image classification component 1906 can also employ one or more machine learning classification models to facilitate automatically classifying images as interior or exterior images of a structure; ¶0142, The exterior perspective component 1908 can determine an optimal perspective of an exterior scan image Examiner interprets the image classification component as the “inside/outside analysis model” and the “first classification model” as its output determines the images sent to the exterior perspective component 1908).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden to obtain the invention as specified in claim 5.
Regarding claim 6, in which claim 5 is incorporated, Sandoval discloses the computer system of claim 5, wherein the at least one processor is further programmed to: determine a second classification model of the plurality of classification models in the hierarchy of models based upon the one or more classifications from the first classification model (Col. 8, 25-26, for example, if one model detects a license plate, a face detection model might be activated. Examiner interprets the face detection model as the “second classification model” and the license plate model as the “first classification model”); route the retrieved image to the second classification model (Col. 8, lines 38-40, the model management engine 210 can direct the sensor data 207 to the 40 active sensor data classification model); execute the second classification model using the retrieved image as the input (Col. 8, 25-26, for example, if one model detects a license plate, a face detection model might be activated); and receive one or more additional classifications from the second classification model based upon the retrieved image (Col. 9, lines 45-46, the augmentation engine 230 can accept sensor data 207, classification output data 222).
Regarding claim 8, Sandoval in view of Hovdel and Kim discloses the computer system of claim 1.
However, Sandoval fails to explicitly disclose wherein the plurality of images are of a property.
Hovdel teaches wherein the plurality of images are of a property (¶0050, a scan of the building in a walk-through manner while capturing images at various locations throughout the building).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden to obtain the invention as specified in claim 8.
Regarding claim 9, Sandoval in view of Hovdel and Kim discloses the computer system of claim 8.
However, Sandoval fails to explicitly disclose wherein the plurality of images include inside and outside images of the structure on the property.
Hovdel teaches wherein the plurality of images include inside and outside images of the structure on the property (¶0050, a scan of the building in a walk-through manner while capturing images at various locations throughout the building; ¶0138, the interior and exterior scan images).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden to obtain the invention as specified in claim 9.
Regarding claim 11, Sandoval discloses a computer-implemented method performed by a hierarchical model image analysis (HMIA) computer device configured to perform hierarchical model analysis (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be
activated. Examiner interprets the activation of one model depending on the outcome of a first model as “hierarchical model analysis”), the HMIA computer device comprising at least one processor in communication with at least one memory device (Col. 4, lines 24-35 processors 52 and data storage 56), the computer-implemented method comprising: receiving a plurality of images (Col. 4, lines 50-56, the camera 110, …, can collect images of the environment in the vicinity of the rig 105. The camera 110 can capture images including, in this example, a distressed child 105A, a suspicious party 105B near the distressed child 105A, and the license plate on a vehicle 107; Col. 6, lines 18-20, Returning to the example 100 illustrated in FIG. 1, image recognition models 120 in the edge platform 101 can process the image data produced by the camera 110. Examiner interprets the images being processed to indicate that they were “received”); and for each image of the plurality of images, retrieving an image of the plurality of images (Col. 6 and Col. 7, provided data, received from edge platform 101, Col. 8 lines 59-61 Sensor data classification models 220 can also provide additional information about the sensor data and include that data in the classification output data 222; processing of individual images from a plurality of images is an inherent feature of the system described in Sandoval); executing a hierarchy of models with the retrieved image as input (Col. 7, line 67, examples of sensors 205 can include cameras; Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated; Col. 8, lines 35-37, The model management engine 210 can accept sensor data 207 from sensors 205 and direct the sensor data 207 to the appropriate, active sensor data classification models 220; Examiner interprets data from cameras as “image” and the activation of one model depending on the outcome of a first model as “a hierarchy of models”), the hierarchy of models comprising [an inside/outside analysis model] and a plurality of additional models (Col. 8, lines 35-37, The model management engine 210 can accept sensor data 207 from sensors 205 and direct the sensor data 207 to the appropriate, active sensor data classification models 220. Examiner interprets the active sensor data classification models as “additional models”), wherein executing the hierarchy of models comprises: routing the retrieved image to [the inside/outside] analysis model (Col. 6, lines 18-21, Returning to the example 100 illustrated in FIG. 1, image recognition models 120 in the edge platform 101 can process the image data produced by the camera 110 to produce classification output data 125. Examiner considers the image recognition model using the image data to indicate the data has been “routed” to the model) [to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure]; receiving an output from the [inside/outside] analysis model indicating that the retrieved image is (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Sandoval discloses receiving the output from an initial analysis model, in this case the face detection model) [one of the inside image or the outside image]; [when the output indicates that the retrieved image is the inside image], routing the retrieved image to at least one model of the plurality of additional models (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers activating the face detection model to indicate “routing” data to said model) [configured to analyze inside images]; and [when the output indicates that the retrieved image is the outside image], routing the retrieved image to one or more models of the plurality of additional models (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers activating the face detection model to indicate “routing” data to said model) [configured to analyze outside images]; outputting classification information for the retrieved image based upon the execution (Col. 8, lines 53-55, each sensor data classification model 220 can be configured to produce a particular type classification output data 222); and associating the classification information with the retrieved image (Col. 8 line 67 – Col. 9 line 1-3, the classification output data 222 can further include an indication of the sensor data 207, or a subset of the sensor data 207, used to produce the predicted classification. Examiner interprets indicating sensor data used to produce the classification as “associate”).
However, Sandoval fails to explicitly disclose an inside/outside analysis model; to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure; when the output indicates that the retrieved image is the inside image, routing to a model configured to analyze inside images; and when the output indicates that the retrieved image is the outside image, routing to a model configured to analyze outside images.
Hovden teaches an inside/outside analysis model (The image classification component 1906 can also employ one or more machine learning classification models to facilitate automatically classifying images as interior or exterior images of a structure. Examiner interprets the image classification component as the “inside/outside analysis model”); to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure (¶0141, The image classification component 1906 can also employ one or more machine learning classification models to facilitate automatically classifying images as interior or exterior images of a structure); when the output indicates that the retrieved image is the inside image (¶0141, the selection component 1904 can be configured to select exterior images for processing; ¶0146, GPS coordinates matched with capture locations of interior scan images. Examiner considers the separation of exterior images and matching of interior images to GPS coordinates to indicate separate analyses performed on exterior and interior images), [routing to a model configured to analyze inside images]; and when the output indicates that the retrieved image is the outside image (¶0141, the selection component 1904 can be configured to select exterior images for processing), routing to a model configured to analyze outside images (¶0142, The exterior perspective component 1908 can determine an optimal perspective of an exterior scan image. Examiner interprets the exterior perspective component as a model “configured to analyze outside images”).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
However, neither Sandoval nor Hovden, whether considered individually or in combination, explicitly disclose a model configured to analyze inside images.
Kim teaches a model configured to analyze inside images (¶0084, when the query image received from the user terminal 100 is captured for a living room and an interior style of a query image is a modern style, the information providing apparatus 200 may transmit information on an accessory included in reference image E classified as the living room and the modern style; ¶0108, the image analysis unit 1030 may store a DNN for image analysis).
Sandoval, Hovden, and Kim are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model, Hovden is directed to image classification of the inside and outside of structures, and Kim is directed to image classification of inside images. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside image model of Kim into the inside/outside analysis model of Hovden and the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to increase the likelihood of a user making a purchase, as suggested by Kim at ¶0122, it is possible to activate a user's purchase of accessories.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden and Kim.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden and Kim to obtain the invention as specified in claim 11.
Regarding claim 12, (drawn to a computer-implemented method) the proposed reference of Sandoval explained in the rejection of claim 2 makes obvious the steps of claim 12, because these steps occur in the operation of the system claim discussed above. Thus, the arguments similar to that presented above for claim 2 are equally applicable to claim 12.
Regarding claim 13, (drawn to a computer-implemented method) the proposed reference of Sandoval explained in the rejection of claim 3 makes obvious the steps of claim 13, because these steps occur in the operation of the system claim discussed above. Thus, the arguments similar to that presented above for claim 3 are equally applicable to claim 13.
Regarding claim 15, (drawn to a computer-implemented method) the proposed reference of Sandoval and Hovden explained in the rejection of claim 5 makes obvious the steps of claim 15, because these steps occur in the operation of the system claim discussed above. Thus, the arguments similar to that presented above for claim 5 are equally applicable to claim 15. Regarding claim 16, (drawn to a computer-implemented method) the proposed reference of Sandoval explained in the rejection of claim 6 makes obvious the steps of claim 16, because these steps occur in the operation of the system claim discussed above. Thus, the arguments similar to that presented above for claim 6 are equally applicable to claim 16.
Regarding claim 18, Sandoval in view of Hovdel and Kim discloses the computer-implemented method of claim 11.
However, Sandoval fails to explicitly disclose wherein the plurality of images are of a property.
Hovdel teaches wherein the plurality of images are of a property (¶0050, a scan of the building in a walk-through manner while capturing images at various locations throughout the building).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden to obtain the invention as specified in claim 18.
Regarding claim 19, Sandoval in view of Hovdel and Kim discloses the computer-implemented method of claim 18.
However, Sandoval fails to explicitly disclose wherein the plurality of images include inside and outside images of the structure on the property.
Hovdel teaches wherein the plurality of images include inside and outside images of the structure on the property (¶0050, a scan of the building in a walk-through manner while capturing images at various locations throughout the building; ¶0138, the interior and exterior scan images).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden to obtain the invention as specified in claim 19.
Regarding claim 20, Sandoval discloses at least one non-transitory computer-readable
media having computer-executable instructions embodied thereon (Col. 10, lines 57-59,
operations of the process 300 can also be implemented as instructions stored on one or
more computer readable media which may be non-transitory) for hierarchical model analysis (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner interprets the activation of one model depending on the outcome of a first model as “hierarchical model analysis”), wherein when executed by a computing device including at least one processor in communication with at least one memory device (Col. 4, lines 24-35 processors 52 and data storage 56), wherein the at least one processor programmed to receive a plurality of images (Col. 4, lines 50-56, the camera 110, …, can collect images of the environment in the vicinity of the rig 105. The camera 110 can capture images including, in this example, a distressed child 105A, a suspicious party 105B near the distressed child 105A, and the license plate on a vehicle 107; Col. 6, lines 18-20, Returning to the example 100 illustrated in FIG. 1, image recognition models 120 in the edge platform 101 can process the image data produced by the camera 110. Examiner interprets the images being processed to indicate that they were “received”); and for each image of the plurality of images, retrieve an image of the plurality of images (Col.6 and Col. 7, provided data, received from edge platform 101, Col. 8 lines 59-61 Sensor data classification models 220 can also provide additional information about the sensor data and include that data in the classification output data 222; processing of individual images from a plurality of images is an inherent feature of the system described in Sandoval); execute a hierarchy of models with the retrieved image as input (Col. 7, line 67, examples of sensors 205 can include cameras; Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated; Col. 8, lines 35-37, The model management engine 210 can accept sensor data 207 from sensors 205 and direct the sensor data 207 to the appropriate, active sensor data classification models 220; Examiner interprets data from cameras as “image” and the activation of one model depending on the outcome of a first model as “a hierarchy of models”), the hierarchy of models comprising [an inside/outside analysis model] and a plurality of additional models (Col. 8, lines 35-37, The model management engine 210 can accept sensor data 207 from sensors 205 and direct the sensor data 207 to the appropriate, active sensor data classification models 220. Examiner interprets the active sensor data classification models as “additional models”), wherein executing the hierarchy of models comprises: routing the retrieved image to [the inside/outside] analysis model (Col. 6, lines 18-21, Returning to the example 100 illustrated in FIG. 1, image recognition models 120 in the edge platform 101 can process the image data produced by the camera 110 to produce classification output data 125. Examiner considers the image recognition model using the image data to indicate the data has been “routed” to the model) [to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure]; receiving an output from the [inside/outside] analysis model indicating that the retrieved image is (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Sandoval discloses receiving the output from an initial analysis model, in this case the face detection model) [one of the inside image or the outside image]; [when the output indicates that the retrieved image is the inside image], routing the retrieved image to at least one model of the plurality of additional models (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers activating the face detection model to indicate “routing” data to said model) [configured to analyze inside images]; and [when the output indicates that the retrieved image is the outside image], routing the retrieved image to one or more models of the plurality of additional models (Col. 8, lines 22-26, In addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated. Examiner considers activating the face detection model to indicate “routing” data to said model) [configured to analyze outside images]; output classification information for the retrieved image based upon the execution (Col. 8, lines 53-55, each sensor data classification model 220 can be configured to produce a particular type classification output data 222); and associate the classification information with the retrieved image (Col. 8 line 67 – Col. 9 line 1-3, the classification output data 222 can further include an indication of the sensor data 207, or a subset of the sensor data 207, used to produce the predicted classification. Examiner interprets indicating sensor data used to produce the classification as “associate”).
However, Sandoval fails to explicitly disclose an inside/outside analysis model; to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure; when the output indicates that the retrieved image is the inside image, route to a model configured to analyze inside images; and when the output indicates that the retrieved image is the outside image, route to a model configured to analyze outside images.
Hovden teaches an inside/outside analysis model (The image classification component 1906 can also employ one or more machine learning classification models to facilitate automatically classifying images as interior or exterior images of a structure. Examiner interprets the image classification component as the “inside/outside analysis model”); to determine whether the retrieved image is an inside image of an inside of a structure or an outside image of an outside of the structure (¶0141, The image classification component 1906 can also employ one or more machine learning classification models to facilitate automatically classifying images as interior or exterior images of a structure); when the output indicates that the retrieved image is the inside image (¶0141, the selection component 1904 can be configured to select exterior images for processing; ¶0146, GPS coordinates matched with capture locations of interior scan images. Examiner considers the separation of exterior images and matching of interior images to GPS coordinates to indicate separate analyses performed on exterior and interior images), [route to a model configured to analyze inside images]; and when the output indicates that the retrieved image is the outside image (¶0141, the selection component 1904 can be configured to select exterior images for processing), route to a model configured to analyze outside images (¶0142, The exterior perspective component 1908 can determine an optimal perspective of an exterior scan image. Examiner interprets the exterior perspective component as a model “configured to analyze outside images”).
Both Sandoval and Hovden are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model and Hovden is directed to image classification of the inside and outside of structures. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside/outside analysis model of Hovden into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to provide an optimal view of a structure, as suggested by Hovden at ¶0137, Technique for determining and defining an “optimal view” and a “reasonable frame” are described in greater detail infra with reference to the exterior perspective component 1908.
However, neither Sandoval nor Hovden, whether considered individually or in combination, explicitly disclose a model configured to analyze inside images.
Kim teaches a model configured to analyze inside images (¶0084, when the query image received from the user terminal 100 is captured for a living room and an interior style of a query image is a modern style, the information providing apparatus 200 may transmit information on an accessory included in reference image E classified as the living room and the modern style; ¶0108, the image analysis unit 1030 may store a DNN for image analysis).
Sandoval, Hovden, and Kim are analogous to the claimed invention because Sandoval is directed to a hierarchical image analysis model, Hovden is directed to image classification of the inside and outside of structures, and Kim is directed to image classification of inside images. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the inside image model of Kim into the inside/outside analysis model of Hovden and the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to increase the likelihood of a user making a purchase, as suggested by Kim at ¶0122, it is possible to activate a user's purchase of accessories.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Hovden and Kim.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Hovden and Kim to obtain the invention as specified in claim 20.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Sandoval et al. (US 12,288,399) (hereafter, “Sandoval”) in view of Hovden et al. (US 2019/0251352) (hereafter, “Hovden”) and further in view of Kim (US 2024/0144343) as applied to claims 1 and 11 above, and further in view of Frei et al. (US 2023/0306539) (hereafter, “Frei”).
Regarding claim 4, in which claim 3 is incorporated, Sandoval discloses wherein at least one of the plurality of classification models (Col. 8, lines 14-15, such as sensor data classification models 220A, 220B, 220C) is [trained to identify a material of an item in the image].
However, Sandoval fails to explicitly disclose trained to identify a material of an item in the image.
Frei teaches trained to identify a material of an item in the image (¶0005, the present disclosure relates to computer vision systems… and material recognition (e.g., wood, ceramic, laminate, or the like)).
Both Sandoval and Frei are analogous to the claimed invention because they are both in
the field of using machine learning for image classification. It would have been obvious to a
person of ordinary skill before the effective filing date of the claimed invention to incorporate
the material classification model of Frei into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been a simple substitution of a materials classifier for an item classifier, as suggested by the variety of engines or classifiers listed by Frei at ¶0021, a computer vision feature segmentation and material detection engine 18b, a computer vision content feature detection engine 18c, a computer vision hazard detection 18d, a computer vision damage detection engine.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Frei.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Frei to obtain the invention as specified in claim 4.
Regarding claim 14, in which claim 13 is incorporated, Sandoval discloses wherein at least one of the plurality of classification models (Col. 8, lines 14-15, such as sensor data classification models 220A, 220B, 220C) is [trained to identify a material of an item in the image].
However, Sandoval fails to explicitly disclose trained to identify a material of an item in the image.
Frei teaches trained to identify a material of an item in the image (¶0005, the present disclosure relates to computer vision systems… and material recognition (e.g., wood, ceramic, laminate, or the like)).
Both Sandoval and Frei are analogous to the claimed invention because they are both in
the field of using machine learning for image classification. It would have been obvious to a
person of ordinary skill before the effective filing date of the claimed invention to incorporate
the material classification model of Frei into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been a simple substitution of a materials classifier for an item classifier, as suggested by the variety of engines or classifiers listed by Frei at ¶0021, a computer vision feature segmentation and material detection engine 18b, a computer vision content feature detection engine 18c, a computer vision hazard detection 18d, a computer vision damage detection engine.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Frei.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Frei to obtain the invention as specified in claim 14.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Sandoval et al. (US 12,288,399) (hereafter, “Sandoval”) in view of Hovden et al. (US 2019/0251352) (hereafter, “Hovden”) and further in view of Kim (US 2024/0144343) as applied to claims 1 and 11 above, and further in view of Bufi (US 2024/0087303).
Regarding claim 7, in which claim 6 is incorporated, Sandoval discloses wherein the at least one processor is further programmed to: [determine a third classification model] of the plurality of classification models (Col. 8, lines 14-15, such as sensor data classification models 220A, 220B, 220C) in the hierarchy of models (Col. 8, lines 22-26, in addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated; Examiner interprets the activation of one model depending on the outcome of a first model as “a hierarchy of models”)) [based upon the one or more additional classifications from the second classification model]; route the retrieved image to the [third] classification model (Col. 8, lines 38-40, the model management engine 210 can direct the sensor data 207 to the 40 active sensor data classification model); execute the [third] classification model using the retrieved image as the input (Col. 8, lines 22-24, in addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated); and receive one or more further classifications from the [third] classification model based upon the retrieved image (Col. 9, lines 45-46, the augmentation engine 230 can accept sensor data 207, classification output data 222; Examiner notes that while Sandoval does not explicitly disclose a third model, the actions of routing data, executing, and receiving output are understood to be generally applicable to a third model by a person of ordinary skill in the art).
However, Sandoval fails to explicitly disclose determine a third classification model based upon the one or more additional classifications from the second classification model.
Bufi teaches determine a third classification model based upon the one or more additional classifications from the second classification model (¶0125, according to second model output data 322b, the model trigger determination module 316 may determine that the inspection image data 320 should subsequently be provided to third model 312c).
Both Sandoval and Bufi are analogous to the claimed invention because they are both in the field of using hierarchical machine learning models. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the third model of Frei into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to prevent model drift by implementing specialized models as suggested by Bufi at ¶0119 this approach advantageously ensures that each of the models 312 is able to perform specialized analysis without “drifting” from that functionality by accommodating further tasks.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Bufi.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Bufi to obtain the invention as specified in claim 7.
Regarding claim 17, in which claim 16 is incorporated, Sandoval discloses wherein the at least one processor is further programmed to: [determine a third classification model] of the plurality of classification models (Col. 8, lines 14-15, such as sensor data classification models 220A, 220B, 220C) in the hierarchy of models (Col. 8, lines 22-26, in addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated. For example, if one model detects a license plate, a face detection model might be activated; Examiner interprets the activation of one model depending on the outcome of a first model as “a hierarchy of models”)) [based upon the one or more additional classifications from the second classification model]; route the retrieved image to the [third] classification model (Col. 8, lines 38-40, the model management engine 210 can direct the sensor data 207 to the 40 active sensor data classification model); execute the [third] classification model using the retrieved image as the input (Col. 8, lines 22-24, in addition, the model management engine 210 can use the output of models to determine which sensor data classification models 220 are to be activated); and receive one or more further classifications from the [third] classification model based upon the retrieved image (Col. 9, lines 45-46, the augmentation engine 230 can accept sensor data 207, classification output data 222; Examiner notes that while Sandoval does not explicitly disclose a third model, the actions of routing data, executing, and receiving output are understood to be generally applicable to a third model by a person of ordinary skill in the art).
However, Sandoval fails to explicitly disclose determine a third classification model based upon the one or more additional classifications from the second classification model.
Bufi teaches determine a third classification model based upon the one or more additional classifications from the second classification model (¶0125, according to second model output data 322b, the model trigger determination module 316 may determine that the inspection image data 320 should subsequently be provided to third model 312c).
Both Sandoval and Bufi are analogous to the claimed invention because they are both in the field of using hierarchical machine learning models. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the third model of Frei into the hierarchical model of Sandoval. The suggestion/motivation for doing so would have been to prevent model drift by implementing specialized models as suggested by Bufi at ¶0119 this approach advantageously ensures that each of the models 312 is able to perform specialized analysis without “drifting” from that functionality by accommodating further tasks.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Bufi.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Bufi to obtain the invention as specified in claim 17.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Sandoval et al. (US 12,288,399) (hereafter, “Sandoval”) in view of Hovden et al. (US 2019/0251352) (hereafter, “Hovden”) and further in view of Kim (US 2024/0144343) as applied to claim 1 above, and further in view of Bokshi-Drotar et al. (US 12,026,786) (hereafter, “Bokshi-Drotar”).
Regarding claim 10, Sandoval in view of Hovden and Kim discloses the computer system of claim 1.
None of Sandoval, Hovden, or Kim, whether considered individually or in combinations, explicitly disclose wherein the plurality of images are of an object to be insured.
However, Bokshi-Drotar discloses wherein the plurality of images are of an object to be insured (Col. 1, lines 61-62, accessing digital image data depicting a roof of the property. Examiner is interpreting a roof to be an “object to be insured”).
Both Sandoval and Bokshi-Drotar are analogous to the claimed invention because Sandoval is in the field of hierarchical models and Bokshi-Drotar serves the application of image classification to objects to be insured. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the images of roofs from Bokshi-Drotar into hierarchical model of Sandoval. The suggestion/motivation for doing so would have been simple substitution of a set of images of roofs for a set of police camera images. One of ordinary skill in the art could have performed the substitution with predictable results.
This method of improving Sandoval was within the ordinary ability of one of ordinary skill in the art based on the teachings of Bokshi-Drotar.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Sandoval with the teachings of Bokshi-Drotar to obtain the invention as specified in claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Mahdi et al. (US 2004/0071341) discloses an indoor/outdoor model (¶0020, In this case, the image is classified as an interior image).
Kim et al. (US 2021/0361136) discloses a classifier configured for the inside of a building (¶0014, The extracting of the cleaning space may include extracting an image characteristic value from the image photographed while the cleaner traveling inside the building; inputting the image characteristic value into the artificial neural network classifier).
Rakha et al. (US 2025/0014161) discloses a classification method for the outside of a building (¶0020, exterior building envelope inspection, the method comprising: obtaining, by a processor, image data of an unmanned aerial system, wherein the image data are acquired from one or more first visual sensors of the unmanned aerial system; detecting objects, including doors and windows, within the obtained image data; identifying the detected objects via one or more classification operation).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMAO DING whose telephone number is (571)272-7237. The examiner can normally be reached Mon-Fri 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIAOMAO DING/Examiner, Art Unit 2676
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676
Prosecution Insights