DETAILED ACTION
This action is in response to Applicant’s response filed on 3/13/2026. Claims 1-20 are now pending in the present application. This Action is made FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 1/02/2026, including the translated copies of foreign patents previously not considered, has been made record of and considered by the examiner.
Response to Arguments
Applicant’s arguments filed on 3/13/2026 have been entered and fully considered.
With respect to the 35 USC 101 rejection for claims 1-20, Applicant argues that the claims are not directed to a judicial exception because “When properly considered as a whole, independent claim 1 is not directed to a mental process, but instead to a specific closed loop architecture.” Applicant further argues the amended claim 1 “presents a computer centric improvement. The method requires continually updating the final ROI used by the thermal camera, which cannot practically be performed in the human mind.”; thus, the claims integrate the allege judicial exception into a practical application into a practical application.
The Examiner agrees; therefore, the previous 35 USC 101 rejection has been withdrawn.
With respect to the 35 USC 103 rejection for claims 1, 11, and 16, and their dependent claims,
Applicant argues that while Bargoti describes “techniques for processing image data to identify specific physical features of objects. In particular, Bargoti describes receiving image data containing one or more objects, applying an image segmentation process to detect predetermined physical features, and identifying regions of the image that are likely to contain those features. Bargoti then outputs the regions identified as corresponding to the predetermined physical characteristics.”
Bargoti fails to disclose “determining a final ROI based on selection rules including comparing a confidence level of suggested ROI with a threshold and determining the final ROI based on the comparison.”
Applicant further argues, “in cited portion of Bargoti, "outputting" the regions refers to providing the detected regions as analytic results. There is no disclosure in Bargoti that the identified regions are communicated to a thermal camera. Consequently, Bargoti is silent regarding providing a closed loop that continually updates the final ROI used by the thermal camera, as recited by amended claim 1.”
Applicant’s arguments have been considered but are moot in view of the new ground(s) of rejection in view of Lee (“Dynamic Belief Fusion for Object Detection”; copy provided by Examiner).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 11, and 16, are rejected under 35 U.S.C. 103 as being unpatentable over Bargoti (US 20230052727), in view of Cruz (US 20140244439), in view of Balasubramanian (US 20220171995), and in further view of Lee (“Dynamic Belief Fusion for Object Detection”; copy provided by Examiner).
Bargoti teaches: A system (claim 1: method, claim 11: computer program product) comprising: a processor set, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media (Abstract; ¶ [0080] “The processing system may also be a distributed system. In general, processing/computing systems may include one or more processors (e.g. CPUs, GPUs), memory componentry, and an input/output interface connected by at least one bus. They may further include input/output devices (e.g. keyboard, displays, etc.). It will also be appreciated that processing/computing systems are typically configured to execute instructions and process data stored in memory (i.e. they are programmable via software to perform operations on data).”; ¶ [0081] “The memory 204 can be any form of memory device, for example, volatile or non-volatile memory, solid state storage devices, magnetic devices, etc.”), the program instructions executable to:
receive camera data from a thermal camera (¶ [0090] “Data capture 320 may involve … using cameras “; Claim 3 “The method of claim 1, wherein the image data is visual image data, thermal image data…);
determine a suggested region of interest (ROI) using the camera data (¶ [0095] “Image processing module 330 may be configured to gain an understanding of images captured during the data capture process 320. This understanding can include extracting regions of interest (ROIs) in an automated way, for example, by obtaining an understanding of what is occurring and where it is occurring in an image…”; See ¶ [0057], Fig. 1, and element 120, representative of the broadest reasonable interpretation of suggested region of interest being claimed as taught by Bargoti, where Bargoti teaches detecting predetermined physical features of objects and the corresponding identified one or more regions of the image data determined to have a likelihood of showing the predetermined physical features (i.e. suggested region(s) of interest (ROI)), which is used in subsequent operations found in, but not limited to, ¶¶[0058]-[0065], where Bargoti teaches defining confidence factors for each region and determining output regions corresponding to object features. Thus, examiner interprets the disclosed teachings of Bargoti discussed herein to be equivalent to determining a suggested ROI using the camera data.),
(¶¶ [0057] - [0058] “image segmentation process identifies one or more regions of the image data determined to have a likelihood of showing, indicating, or having a visual indication of one or more of the predetermined physical features. In some examples, step 120 involves the detection, identification, and categorisation of predetermined physical features in the image data. A physical feature may be a colour, texture, shape, or characteristic of an object…; ¶ [0013] “… the image segmentation process is implemented by a region-based segmentation process, a mathematical morphology segmentation, a genetic algorithm-based segmentation, an artificial neural network-based segmentation, or a deep learning structure.”; ¶ [0099] “Image segmentation may be performed by recognising one or more characteristics or features of any number of pixels in the image. Image segmentation may refer to “recognition”, “classification”, “extraction”, “prediction”, “regression”, or any other process whereby some ROIs or some level of understanding is extracted automatically from regions in an image.”);
determine a final [region] using the suggested ROI and a selection rule (¶¶ [0058]-[0065] teaches using the suggested ROI taught by Bargoti in ¶ [0057] (i.e. identified one or more regions of the image data determined to have a likelihood of showing one or more predetermined physical features (e.g. shapes)) and a technique of determining a confidence factor for each pixel or data point in a region or in the image data that represents a likelihood of the presence of one or more of the predetermined physical features in the region and a technique automatically tagging regions as not having one or more predetermined physical features that have a confidence factor below a predetermined probability threshold and outputting the remaining identified region(s). Examiner interprets the remaining identified region(s) that are output to be equivalent to a final region because the outputted regions are the final regions identified in the process. Further, the process of assigning confidence factors to the interpreted suggested ROIs and comparing the confidence factors to a predetermined threshold is interpreted as equivalent to a selection rule that determines a final region using the suggested ROI.)
wherein the selection rule comprises comparing a confidence level of the suggested ROI to a threshold and determining the final [region] based on the comparison (see (¶¶ [0057]-[0065] and the explanation provided in the previous paragraph of the present office action),
and communicate the final [region] (Bargoti discloses outputting identified regions and communicating such outputs to devices via wired or wireless communication and processing thermal image data captured by a camera (¶ [0079] “The at least one processing system may further be configured to output the identified one or more regions.”; ¶¶ [0079]-[0084]; ¶ [0090]) and processing thermal image data captured by a camera (¶ [0051]).)
Although Bargoti teaches a system capable of communication, Bargoti fails to explicitly disclose a camera system capable of communication (i.e. communicate the final ROI to the thermal camera).
In a related art, Cruz teaches a camera system capable of communication (Abstract; ¶ [0017]; ¶ [0022]-[0024]).
It would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to provide the identified final regions taught by Bargoti to the camera system of Cruz so that the final regions may be presented in association with the captured thermal image data, as integrating analysis results with the imaging device enables real-time feedback and improves situational awareness and processing efficiency. Both inventions lie in the field of endeavor of image analysis with applications in thermal imaging.
In the embodiments described above, Bargoti and Cruz fail to explicitly disclose a final ROI and providing a closed loop that continually updates the final ROI used by the thermal camera.
However, Bargoti does explicitly teach regions of interest (ROIs) (¶ [0095] “Image processing module 330 may be configured to gain an understanding of images captured during the data capture process 320. This understanding can include extracting regions of interest (ROIs); ¶ [0099] “Image processing module 330 may further perform scene understanding during which an image potentially including any number of ROIs may be identified based on an image segmentation technique”) and in another embodiment Bargoti teaches a quality assurance module (QA) that continuously updates the model to improve task performance based on scene understanding (FIG. 15, ¶¶ [0127]-[0129] “QA module 370 is configured to identify areas/tasks in which system 300 performs in a suboptimal manner, and to adjust existing processes such that system 300 improves its performance on tasks. This process may be continual over the lifetime of system 300….Scene understanding techniques in the image processing module 330 may be continually updated throughout the operation and lifetime of system 300. ROIs may be assessed by their performance on the original tasks or derivative tasks. For example, predicted “defect” class ROIs may be selected by their ability to predict regions of “cracking” class. ROIs may be selected if their predictions are incorrect, or they may be identified by a low “confidence factor”.”). Refer to FIG. 15 (seen below) and the continuous loop (i.e. closed loop) functionality being performed by the “Image Processing Module” that starts at the “Intelligent Sampling” step of the QA Module, proceeds to the “Review” step, the “Integration” step, the “Retraining” step, the “Reprocessing” step, then loops back to the “Image Processing Module” to restart and continuously perform the respective process.
PNG
media_image1.png
842
612
media_image1.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teaching of Bargoti, detailed in the first embodiments of the present office action, and Cruz to incorporate Bargoti’s further teachings of ROIs in order to provide a final ROIs, rather than a final region, and to incorporate the further teachings of Bargoti’s continuous loop for updating the model. Doing so would increase the accuracy of the model by providing an output that is more specific to the location by providing a final ROI, rather than a final region being output. Subsequently, the ability to continuously update the framework’s final ROI through a closed loop would provide increased accuracy to the thermal camera over the course of subsequent interactions within the system by enabling the final ROI to be adjusted for differences in process results (e.g. an object and its corresponding final ROI changing from image to the next).
Bargoti and Cruz further fail to explicitly disclose an object detection model, an anomaly detection model, and wherein the confidence level is aggregated from confidence levels determined by the object detection model and the anomaly detection model.
In a related art, Balasubramanian teaches analyzing images from thermal cameras through the use of anomaly models and object detection models (Abstract; ¶ [0044] “The image/video capture device 141 captures still or video images to be analyzed for anomalies. Exemplary image/video capture devices include, … thermal cameras….”; ¶ [0149] “an anomaly detection model is applied to an unlabeled dataset to look for anomalies and provide a confidence in its predictions.“; ¶ [0038] “Examples of models 103 used by the anomaly detection service/component 102 may include, but are not limited to models to perform anomaly detection, object detection, and/or alignment (orientation) detection.”; ¶ [0064].). Balasubramanian further teaches a threshold used for anomaly detection (FIG. 11, element 117, ¶ [0086]), using confidence scores and/or rankings to provide a confidence in the anomaly detection model’s predictions (¶¶ [0147]-[0152]), and using a machine learning model for providing a feedback loop to re-train and improve the original model for accuracy (¶ [0036]; FIG 3, ¶¶ [0058]-[0062]; ¶¶ [00143]-[0146]).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti and Cruz to incorporate the teachings of Balasubramanian’s anomaly detection model, anomaly detection confidence scores, and object detection models to provide the thermal camera with data relating to anomalies and objects found on an image, thereby increasing the camera’s accuracy by identifying anomalies, and their corresponding objects, in order to provide a more accurate final ROI in subsequent tasks and eliminating inconsistent performance associated with human inspection (Balasubramanian identifies problems associated with human inspection, ¶ [0012] “Human inspection is slow, and errors often get missed due to inconsistency in judgment, limited experience and training, availability and cost of labor, and even factors like poor eyesight and being distracted while working.”).
Bargoti, Cruz, and Balasubramanian fail to explicitly disclose wherein the confidence level is aggregated from confidence levels determined by the object detection model and the anomaly detection model.
In a related art, Lee et al. teaches: object detection confidence scores and aggregating confidence levels from multiple models. Specifically, Lee et al. discloses combining multiple detectors output, i.e. model output, (Abstract “..effectively integrate the outputs of multiple detectors..”), having confidence levels in the detection results (Abstract “…based on confidence levels in the detection results conditioned on the prior performance of individual detectors”) and further teaches fusion of detection scores from disparate object detection methods (p. 2, left column, last 5 lines, “The proposed fusion approach can robustly extract complementary information from multiple disparate detection approaches…”) wherein the fused result is reduced to a single fused detection score (Abstract “…optimally fusing information from all detectors, is determined by the Dempster’s combination rule, and is easily reduced to a single fused detection score.”; Figure 1 description “Dempster’s combination rule combines the basic probabilities of each detector and returns a fused confidence score.”). Lee further teaches object detection and corresponding detection scores based on confidence (see Figure 3, found below, which shows an example of Dempster’s combination rule, including “Detection score (confidence)” based on a detected “car in a given image” (i.e. an object detected and a corresponding confidence score) that derives a detector for each model and is further combined for a “Fusion score” (i.e. aggregated confidence score based on multiple detectors (i.e. models)). Lee also teaches there are different principles of detecting objects of interest and states “Given, an image, the detection score indicates a degree of confidence about the decision.” (p. 5, right column, subsection 5.1, “Individual Detectors”). Thus, Lee et al. teaches object detection and corresponding confidence scores and aggregating confidence levels from multiple models to produce a unified confidence score.
PNG
media_image2.png
810
921
media_image2.png
Greyscale
It would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of a selection rule comparing a confidence level of the suggested ROI to a threshold and determining the final ROI based on the comparison as taught by Bargoti, modified by Cruz and Balasubramanian, to substitute the structure of the confidence level previously taught by Bargoti and Balasubramanian with the structure of the confidence level taught by Lee, which uses the aggregate of multiple models’ confidence scores. Using an aggregate structure of an object detection model confidence level (taught by Lee) and a second confidence level corresponding to an anomaly detection model (taught by Balasubramanian) would provide the predictable result of an increase in detection accuracy, greater than approaches using a single detector or model (see Lee Abstract and p. 8, section “7. Conclusions”). All inventions lie in the same field of endeavor of image analysis and using detection models to analyze image data.
Claims 2-6, 8-9, 12-14, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Bargoti (US 20230052727), in view of Cruz (US 20140244439), in view of Balasubramanian (US 20220171995), in view of Lee (“Dynamic Belief Fusion for Object Detection”; copy provided by Examiner), and in further view of Lichtensztein (US 20220313095).
Regarding Claims 2, 3, 4, 12, and 17:
Bargoti in view of Cruz, Balasubramanian, and Lee teach the limitations of claims 1, 11, and 16.
Bargoti further teaches: the camera data includes optical image data of an asset, thermal image data corresponding to the optical image data, and metadata corresponding to the optical image data and the thermal image data (¶ [0012] “In certain embodiments, the image data is visual image data, thermal image data, hyperspectral image data, or 2D depth image data.“; [0015] “In certain embodiments, the method further comprises receiving metadata of the one or more objects, wherein the metadata is associated with the image data.”);
and the program instructions are executable to: detect an object [physical features] in the optical image data [physical features]; (¶ [0009] “According to an example aspect, there is provided a computer-implemented method comprising: receiving image data of one or more objects; applying an image segmentation process to the image data to detect predetermined physical features of the one or more objects, wherein the image segmentation process identifies one or more regions of the image data determined to have a likelihood of showing one or more of the predetermined physical features; and outputting the identified one or more regions”. Examiner interprets identified one or more regions of the image data determined to have a likelihood of showing one or more of the predetermined physical features (corresponding to an object) to be equivalent to generating an initial ROI based on a detected object’s physical features.).
While Bargoti teaches detecting objects physical’s features, and generating an initial ROI based on the physical features of the object, Bargoti fails to explicitly disclose using an object detection model to detect an object in the optical image data and generating an initial ROI based on the detection model’s detected object.
However, in a related art Balasubramanian does teach an object detection model (Abstract; ¶ [0038]), as detailed in rejection of claims 1, 11, and 16 of the present office action.
As detailed in regards to claims 1, 11, and 16 of the present office action, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti, previously modified by Cruz, Balasubramanian, and Lee, to incorporate the object detection model taught by Balasubramanian, as previously asserted in the 35 U.S.C. 103 rejections for claims 1, 11, and 11 because it would improve the accuracy of the model by detecting for objects when determining the initial ROI, not just features of the object and their corresponding ROI.
Bargoti, Cruz, Balasubramanian, and Lee fail to teach extracting temperature information from the thermal image data in the initial ROI.
In a related art, Lichtensztein teaches to extract temperature information from thermal image data in an initial ROI (¶ [0024] “The sensor system(s) 204 may include one or more optical sensors 206 (e.g., cameras) configured to capture and analyze visual data, and/or thermal sensors 208 configured to capture and analyze temperature data within the physical environment 202 using infrared or thermal scanning technology. As noted above, the temperature assessment system 102 may use a combination of optical data and thermal data to perform the temperature assessments and/or mask assessments described herein.”; ¶ [0025] “When the temperature assessment system 102 detects the individual via facial detection, within the ROI, and/or within the predetermined distance from the sensor systems 204, the temperature assessment system 102 may automatically trigger one or more assessments on the individual.”).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti, previously modified by Cruz, Balasubramanian and Lee, to incorporate the teachings of Lichtensztein (Abstract) and extract temperature information from an initial image’s ROI thermal image data corresponding to the optical image data, and metadata corresponding to the optical image data and the thermal image data by completing a temperature assessment. Doing so would provide improve the quality of the assessments relating to analyzing optical and thermal data and reduce the number of errors (Lichtensztein ¶ [0013] “The temperature assessment and mask detection techniques described herein also may use a combination of machine learning and heuristics-based operations, which analyze both the optical and thermal data to improve the quality of assessments and reduce errors. For example, the temperature assessment system may execute machine learning models for facial detection and bounding box generation, and a heuristics-based distance verification component, to detect and track individuals as they move through the perception field of the thermal sensors. For individuals within an optimal temperature scanning range, the temperature assessment system may execute machine learning and/or heuristics-based operations to perform temperature assessments, thermal uniformity assessments, and/or mask detection.”). Using the known technique of a temperature assessment, taught by Lichtensztein, to analyze an initial ROI provided by the program instructions, taught by Bargoti in view of Cruz, Balasubramanian, and Lee, would have been obvious to one of ordinary skill in the art.
Regarding Claims 5, 13, and 18:
Bargoti in view of Cruz, Balasubramanian, Lee, and Lichtensztein teach the limitations of claims 3, 12, and 17.
Balasubramanian further teaches (Previously referenced in regards to claims 1, 11, and 16 (Abstract; ¶ [0044]); [0149]))
Bargoti, Cruz, Balasubramanian, and Lee fail to teach wherein the program instructions are executable to generate an intermediate suggested ROI
In a related art Lichtensztein further teaches to generate an intermediate suggested ROI by scanning an image of a physical environment or initial ROI to extract temperature information and additional information (¶ [0018] “In some cases, the temperature assessment system 102 may determine and label a region of interest (ROI) 108 within the user interface 100. The temperature assessment system 102 may detect and perform assessments for individuals within the ROI…. Additionally or alternatively, the temperature assessment system 102 may configure the ROI 108 based on the size and layout of the physical environment being scanned. For example, the size, shape, and location of an ROI 108 may be determined by the temperature assessment system”), and to identify new (or intermediate) suggested ROIs based on the image data (¶ [0025] “Prior to performing an assessment of an individual, in some instances the temperature assessment system 102 may identify one or more ROIs within the physical environment 202, and may use facial detection to detect individuals within the ROIs. In some cases, the temperature assessment system 102 also may use data from distance sensors 210 to determine when an individual in the ROI is within a predetermined distance from the sensor systems 204, such as an optical scanning range of the thermal sensor(s) 208.”; ¶ [0066] “For instance, when performing multiple assessments/reassessments, the temperature assessment system 102 may determine different ROIs, different bounding boxes, and/or may use different combinations of sensor data for the different assessments.”).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti, Cruz, Balasubramanian, Lee, and Lichtensztein to incorporate the further teachings of Lichtensztein and generate a suggested intermediate ROI based on anomalies found in the initial ROI the extracted temperature information, and additional information. Using the known technique of an anomaly detection model, taught by Balasubramanian, to analyze an initial ROI provided by the program instructions, taught by Lichtensztein, would have improved the efficiency in identifying temperature outliers’ ROI and would have been obvious to one of ordinary skill in the art.
Regarding Claim 6:
Bargoti in view of Cruz, Balasubramanian, Lee, and Lichtensztein teach the limitations of claims 5.
Balasubramanian further teaches the anomaly detection model comprises an ensemble of plural anomaly detection models; and Balasubramanian teaches an anomaly detection model comprises an ensemble of plural anomaly detection models scoring anomalies over a span of time and the scores may be ranked by highest (Abstract; ¶¶ [0149]-[0153]; ¶ [0063] In some embodiments, the “model” that is a trained consists of multiple models with each model addressing a different problem type. Exemplary problem types may include, but are not limited to: i) object detection, ii) alignment, iii) unsupervised anomaly detection, and iv) supervised anomaly detection.”; ¶ [0067] “The supervised anomaly detector (the entire graph) is evaluated on testing data at 409 and metrics shown. The evaluation may include several actions such as collection exploration, anomaly labeling (annotating), etc. Collection exploration means that after having all inference outputs (and logged image metadata) users are to explore the collection of captured images using multiple ranking and filtering criteria.”)
Bargoti, previously modified by Cruz, Balasubramanian, Lee and Lichtensztein, fails to teach the intermediate suggested ROI is generated using the highest-ranked of the plural anomaly detection models.
In a related art, Lichtensztein further teaches thermal image data from sensors may be used to determine different ROIs (Abstract; ¶ [0053] “In some examples, the thermal sensors 208 may use thermal and/or infrared technology that permits the temperature assessment system 102 to receive multiple temperature measurements, based on the same optical/thermal images, for different target locations on or around the individuals depicted in the images.”; ¶ [0066] “For instance, when performing multiple assessments/reassessments, the temperature assessment system 102 may determine different ROIs, different bounding boxes, and/or may use different combinations of sensor data for the different assessments.”).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti, Cruz, Balasubramanian and Lichtensztein to incorporate the further teachings of Lichtensztein and generate a suggested intermediate ROI based on a plurality of anomaly detection models (e.g. unsupervised anomaly detection, and iv) supervised anomaly detection) and the respective highest ranked anomaly. Using the known technique of implementing anomaly detection models over a span of time to provide data on thermal images’ anomaly scores, taught by Balasubramanian, to generate an intermediate ROI based on thermal image data, taught by Lichtensztein, would have improved the quality of assessments and reduced errors and would have been obvious to one of ordinary skill in the art.
Regarding Claims 8, 9, 14, and 19
Bargoti in view of Cruz, Balasubramanian, Lee, and Lichtensztein teach the limitations of claims 5, 13, and 18.
Lichtensztein further teaches the program instructions are executable to determine a shape of the suggested ROI using the intermediate suggested ROI and the shape recommendation model; and the shape recommendation model determines the shape of the suggested ROI based on one or more optimization rules. By using the intermediate suggested ROI, established in claims 5, 13, and 18, Lichtensztein teaches to determine a shape (e.g. a particular body part or foreign object) using optimization or business rules (e.g. identifying parts from a point and detection algorithms and/or machine learning models) (¶ [0057] “In some examples, the temperature assessment system 102 may be configured to identify the measurement points on the skin of the individual, and/or at particular locations on the individual that provide the most accurate body temperature readings, such as the forehead, neck, and upper torso. For instance, the temperature assessment system 102 may use facial and body detection algorithms, and/or machine-learning models, to determine a precise size, shape, and location of the face and neck of the individual. Additional algorithms and/or machine learning model may be used to identify the individual's hair, clothing, facial hair, jewelry, or other foreign objects, etc. Based on these locations, the temperature assessment system 102 may determine certain temperature measurement points (e.g., on the individual's forehead or neck) that are likely to provide more accurate body temperatures for the individual, and other measurement points (e.g., on the individual's hair, nose, mask, or clothing, etc.) that are unlikely to provide accurate body temperatures. Whenever the temperature assessment system 102 determines measurement points that are more or less likely to provide accurate body temperatures, it may bias or weight these temperature measurements, and/or exclude other temperature measurements, in any of temperature assessment techniques described herein, to account for the higher or lower quality of the particular temperature measurement points.”).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Bargoti (US 20230052727), in view of Cruz (US 20140244439), in further view of Balasubramanian (US 20220171995), in further view of Lee (“Dynamic Belief Fusion for Object Detection”; copy provided by Examiner), and in further view of Lichtensztein (US 20220313095)..
Regarding Claim 7:
Bargoti in view of Cruz, Balasubramanian, Lee, and Lichtensztein teach the limitations of claims 5.
Lichtensztein further teaches the collaborative data comprises data from a same family of asset as the detected object; the content data comprises one or more selected from a group consisting of (Lichtensztein teaches a system to analyze only objects of the same family assets (people) by generating bounding boxes around individuals and performing individual assessments for different people (Abstract; ¶ [0026] “store the results of individual assessments and related data within a log data store 218.”):
historical temperature data of the asset (¶ [0025] “In some examples, the temperature assessment system 102 also may receive additional data, such as the previous temperature readings from a temperature log, and/or other data associated with an individual, to use as input data to the temperature assessments and/or mask assessments.”);
and real-time condition data of the asset (¶ [0027] In some examples, the monitoring system 214 may provide a user interface based on the assessment data from temperature assessment system 102, that allows an administrator, security officer, or other user to monitor the physical environment 202 in real-time or near real-time.”);
and the context data comprises events data associated with the asset (¶ [0016] “The temperature assessment system also may output alerts and other notifications to the monitoring system and/or other user devices, including alerts for the detection of abnormally high body temperatures or non-compliance with masking requirements.”; [0030] “As another example, in response to assessment failures or alerts generated by the temperature assessment system 102, the monitoring system 214 may record the alert/failure in a log data store 218, along with data identifying the individual that triggered the alert/failure and the assessment results.”).
Bargoti, Balasubramanian, Lee, Lichtensztein and prior teachings of Cruz described in the present office action fail to teach a system where content data comprises manufacturer information or specifications of the asset and maintenance history of the asset.
However, Cruz further teaches scanning a thermal image data for objects (Abstract; [0015]) and detected content data consisting of:
manufacturer information or specifications of the asset (¶ [0032] “components in the data repository 69A is displayed in FIG. 5. In the illustrated example, the list 500 includes a manufacturer 502 and model number 504 of the matched object as well as a description 506 and a listing 508 of the individual component parts associated with the matched object.”);
maintenance history data of the asset (¶ [0039] “data repositories integrate user specific information like repair schedules, lifespan of finished good, lifespan of a component of the finished good (in the case of replacement parts), etc.”).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti, Cruz, Balasubramanian, Lee, and Lichtensztein to incorporate the further teachings of Cruz to include manufacturer information or specifications data of the asset and maintenance history data of the asset in the collaborative data, along with the historical temperature data of the asset, real-time condition data of the asset, and the context data comprising events data associated with the asset. Doing so would decrease time and human errors associated with manually identifying objects and retrieving data (Cruz [004]).
Claims 10, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Bargoti (US 20230052727), in view of Cruz (US 20140244439), in further view of Balasubramanian (US 20220171995), in further view of Lee (“Dynamic Belief Fusion for Object Detection”; copy provided by Examiner), and in further view of Tan (US 20210049776).
Regarding Claims 10, 15, and 20:
Bargoti in view of Cruz, Balasubramanian, and Lee teach the limitations of claims 1, 11, and 16.
Balasubramanian further teaches wherein the comparing the confidence level of the suggested ROI to the threshold (¶ [0114]; ¶ [0149]; ¶ [0205]) comprises:
in response to the confidence level of the suggested ROI being less than the threshold providing the suggested ROI to a user via a user device and determining the final ROI based on input from the user (As detailed in claims 1, 11, and 16, Balasubramanian teaches regions of interest and confidence level scores based on anomalies (¶ [0114]; ¶ [0149] ) “Note that in some embodiments, the creation of the dataset(s) includes… identification of regions of interest in an image to look for anomalies, identification of objects to detect”; ¶ [0149] “At 2402 a trained model is applied to a dataset of items to generate predictions (e.g., score) and, in some embodiments, confidences in those predictions. For example, an anomaly detection model is applied to an unlabeled dataset to look for anomalies and provide a confidence in its predictions.”). Balasubramanian further teaches an embodiment for users to enter input when a confidence level of an anomaly (an their corresponding suggested region of interest) is below a threshold to adjust results of model (e.g. anomaly models) and subsequent regions of interest, ¶ [0205] “For example, the model metrics can indicate that the machine learning model is performing poorly… has a confidence level below a threshold value)) and/or is performing progressively worse …In response, in some embodiments, the user, via the user device, can transmit a request to the model training system 120 to modify the machine learning model being trained (for example, transmit a modification request). The request can include a new or modified container image, a new or modified algorithm, new or modified hyperparameter(s), and/or new or modified information describing the computing machine on which to train a machine learning model.”; FIG. 18 depicts a GUI where the user is provided a suggested area (i.e. ROI) and where the user can reselect or correct areas when anomalous areas are wrong (i.e. confidence level is below a threshold). The reselected or corrected areas taught by Balasubramanian is interpreted as a final ROI.).
It would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti, Balasubramanian and Lee to incorporate the teachings of the further teachings of Lee’s framework of providing the suggested ROI to a user via a user device and determining the final ROI based on input from the user when a confidence level in response to the confidence level of the suggested ROI being less than the threshold, while using the previously modified aggregated confidence level (detailed in claim 1, 11, and 16 rejected of the present office action), taught by Bargoti, Balasubramanian, and Lee, when comparing confidence level to a threshold. Doing so would increase the accuracy of the system by providing a method of user interaction to correct poorly derived suggested ROIs, rather than outputting a final ROI, despite a low level of confidence in the object and/or anomaly being present in the image.
Bargoti, Cruz, Balasubramanian and Lee fail to teach in response to the confidence level of the suggested ROI being greater than the threshold, automatically selecting the suggested ROI as the final ROI.
In a related art, Tan teaches in response to the confidence level of the suggested ROI being greater than the threshold, automatically selecting the suggested ROI as the final ROI (Tan refers to final ROI as output, ¶ [0046] “final (output) ROI(s)”; ¶ [0078] “may output an ROI of any location referenced in the current detection candidate map 402 that is associated with a confidence score (of the received feature map) that meets or exceeds a confidence score threshold.”). Examiner notes, Tan teaches confidence score based on object detections (¶ [0018] and ¶ [0022]).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the teachings of Bargoti, Balasubramanian and Lee to incorporate the teachings of Tan and automatically select a suggested ROI with a confidence level greater than the threshold as the final ROI. Doing so would provide the thermal camera, and subsequent user, with an increased number of metrics (i.e. level of confidence in objects and anomalies) for evaluating and identifying ROIs found in the image data.
Examiner Notes
The Examiner cites particular paragraphs in the references as applied to the claims above
for the convenience of the Applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, the Applicant fully considers the references in its entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or as disclosed by the Examiner.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMUEL DAVID BAYNES whose telephone number is (571)272-0607. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.D.B./
Samuel D. Baynes
Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665