Detailed Action
1. Claims 1-20 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to amendment
3. Applicant’s response to the last Office Action filed on 11/05/2025 has been entered and made of record.
4. Claims1-15 have been amended, and new claims 16-20 have been added.
.
Response to Argument
5. The Applicant’s argument filed 02/04/2026 is fully consider. For Examiner response see discussion below.
6 The Applicant’s has substantially amended claims 1 and 12. Based on the Applicant amendment and argument the 35 U.S.C 102 rejection is expressly withdrawn. However, further search and consideration new prior arts that teaches the amended claims 1 and 12 found.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1, 2, 12, 16-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gitzel; Ralf (hereafter Gitzel), US 20230342906 A1, filed 04/20/2023, in view of ZHANG et al., (hereafter ZHANG), CN 107341947 A, pub. 11/10/2017.
As to claim 1, Gitzel teaches A method (a method and system for monitoring a switchgear.) comprising:
generating a temperature image of an area ([0011] In one embodiment, the infrared camera is configured to acquire a first infrared image of the switchgear, and a total number of pixels in the first infrared image is equal to a first number)claim 11, [0011]the processing unit is configured to determine a pixel in the first infrared image with a maximum temperature in the first infrared image. The processing unit is configured to utilize a second number of pixels less than the first number of pixels to determine a temperature interval (range) for the first infrared image equal to a difference between the maximum temperature in the first infrared image and a threshold temperature in the first infrared image.);
generating a visual image of the area using sensor data from a visual camera ([0034] According to an example, the system comprises a visible camera configured to acquire a visible image of the switchgear);
determining a transformation between the temperature image and the visual([0098]-[0100], the processing unit is configured to overlay a location of a maximum
temperature in an infrared image onto a corresponding location in the visible image. In other words the location of hot pixels can be mapped to a visible image in order to allow a human to review the situation to determine if there is a fault); and
generating, based on the transformation, a combined image of the area by at least overlaying the temperature image onto the visual image(Claim 11, wherein the system comprises a visible camera configured to acquire a visible image of the switchgear, and wherein the processing unit is configured to overlay a location of a maximum
temperature in an infrared image onto a corresponding location in the visible image).
It is noted that Gitzel does not specifically teaches “by using a gradient-based algorithm to assign one or more colors to a portion of the temperature image based on information from an infrared sensor”
On the other hand ZHANG teaches using a gradient-based algorithm to assign one or more colors to a portion of the temperature (Abstract, page 2 invention content section, par. 4, ZHANG teaches advance transplantation vision algorithm (equivalent to a gradient-based algorithm) applies one or more colors to sections of a thermal image based on infrared sensor data. This data includes the specific temperature range for the corresponding area. Specifically ZHANG teaches the mini PC in advance transplantation vision algorithm processing the image from the infrared imager. Displaying the highest temperature value and lowest temperature value and colour-temperature gradient of the temperature interval on the first display screen of the infrared thermal imager. Vision algorithm starting from extracting temperature value in the digital area and gradient
region to identify the extreme temperature value).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate advance transplantation vision algorithm designed to display highest temperature value and lowest temperature value and colour-temperature gradient of the temperature interval taught by ZHANG into Gitzel
The suggestion/motivation for doing so would have been allows user of Gitzel to display the maximum and minimum temperature of infrared image and also display the colour-temperature gradient of the infrared image.
As to claim 12, Gitzel teaches A system comprising an infrared sensors a visual camera and memory comprising instructions that when executed cause processing circuitry ([0078], the computing unit can be configured to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method according to one of the preceding embodiment); regarding the remaining limitation 0f claim 12, the remaining limitations are rejected the same as claim 1 except claim 12 is directed to a system claim. Thus , the analysis applied to claim 1 above is also applicable to claim 12.
As to claim 2, Gitzel teaches determining the transformation further comprises determining a difference between the visual image and the temperature image([0099]-[0010], the processing unit is configured to overlay locations of pixels having a temperature between a threshold temperature and the maximum temperature in an infrared image
onto corresponding locations in the visible image. In other words, the location of hot pixels can be mapped to a visible image in order to allow a human to review the situation to determine if there is a fault).
As to claim 17, ZHANG teaches based on the gradient-based algorithm determining a rate of change of temperature within the temperature range, a boundary between a first temperature range and a second temperature range within the temperature range of the portion of the area; and assign, based on the boundary, a first color to the first temperature range and a second color to the second temperature range ( page 2 5th par., the mini PC in advance transplantation vision algorithm processing the image from the infrared thermal imager, the infrared thermal imager is a first display screen for displaying the highest
temperature value and lowest temperature value and the temperature interval of the colour temperature gradient; vision algorithm starting from extracting temperature value in the digital area and gradient region to identify the extreme temperature value according to the digital pixel characteristic using template matching, and returning the value of stored in the two variables defined in advance, lower the preset target temperature condition. Extracting gradient in target temperature corresponding to all rows of pixel values, the pixel calculating average value and taking the value as the threshold parameter of the thresholded image to extract target temperature over the area, when the area reaches a certain limit, returning to a danger sig).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate advance transplantation vision algorithm designed to display highest temperature value and lowest temperature value and colour-temperature gradient of the temperature interval taught by ZHANG into Gitzel
The suggestion/motivation for doing so would have been allows user of Gitzel to display the maximum and minimum temperature of infrared image and also display the colour-temperature gradient of the infrared image.
Claim 18 is rejected the same as claim 17 except claim 18 is directed to method claim . Thus , the analysis applied to claim 17above is also applicable to claim 18.
As to claim 16, Gitzel teaches the memory includes instructions that when executed further cause the processing circuitry to: determine a position of the system; determine a first difference between a first image frame from the visual camera and the position of the system);
determine a second difference between a second image frame from the infrared sensor and the position of the system (claims 11-12, visible camera configured to acquire a visible image
of the switchgear, and wherein the processing unit is configured to overlay a location of a maximum temperature in an infrared image onto a corresponding location in the visible image. wherein the processing unit is configured to overlay locations of pixels having a temperature between a threshold temperature and the maximum temperature in an infrared image onto corresponding locations in the visible image.); and determine the transformation based on one or more of the first difference and the second difference ( [0009]-[0010], the location of hot pixels can be mapped to a visible image in order to allow a human to review the situation to determine if there is a fault. based on a temperature between a threshold temperature and the maximum temperature ).
As to claim 20, the combination of Gitzel and ZHANG teaches obtaining a user input labeling a feature within the combined image (Gitzel: the determination that the hot spot exists comprises utilization of a machine learning algorithm., see claim 9. ) and updating, based on the user input, the gradient-based algorithm(ZHANG: transplanting vision algorithm processing the image from the infrared imager in advance in said step S2 specifically is: micro PC. the first display screen of the infrared thermal imager for displaying highest temperature value and lowest temperature value and the temperature interval of the colour-temperature gradient strip…., see page 3 2nd par., )
8. Claims 3 and 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Gitzel, US 20230342906 A1, in view of ZHANG, CN 107341947A, further in view of LIU, Xiao-lin (hereafter LIU), CN 113130028 A, pub., 07/16/2021.
As to claim 3, Gitzel teaches generating temperature image as discussed in claim 1 above, but fails to teach “enhancing a contrast of the temperature image based on the information from the infrared sensor using histogram equalization.”
On the other hand in the same field of endeavor a method of generating a thermal infrared medical image and a visible light medical image of LIU (see Abstract) teaches enhancing a contrast of the temperature image based on the information from the infrared sensor using histogram equalization (Claim 4, page5 last par.,- page 6 2nd par., obtaining basic medical image and detail medical image, then performing histogram equalization operation on the basic medical image).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate applying a well- known histogram equalization method taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of modified Gitzel automatically enhance image contrast, thus improve the quality of infrared image.
As to claim 5, LIU teaches generating the temperature image comprises filtering noise from the information from the infrared sensor(Claim 4, page 6 2nd par., Step S320: using Gaussian filtering to obtain the thermal infrared enhanced medical image and the visible light enhanced medical image respectively obtaining the noise reduction medical image).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate applying a well- known Gaussian filtering configure to remove noises from both infrared image visible image taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of modified Gitzel to generate high quality image by removing noised from both infrared image visible image .
As to claim 6, LIU teaches generating the visual image comprises enhancing contrast of the visual image using histogram equalization( Claim 4, page5 last par.,- page 6 2nd par., obtaining basic medical image and detail medical image, then performing histogram equalization operation on the basic medical image, and performing gamma conversion operation to the detail medical image).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate applying a well- known histogram equalization method taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of modified Gitzel automatically enhance image contrast, thus improve the quality of infrared image
As to claim 7, LIU teaches creating a visual image comprises enhancing the visual image using dynamic range compression(Claim 4, page5 last par.,- page 6 2nd par performing dynamic range compression on the obtained basic medical image and detail medical image, obtaining visible light enhanced medical image).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate applying a well- known histogram equalization method taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of modified Gitzel automatically enhance image contrast, thus improve the quality of infrared image
9. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Gitzel, US20230342906 A1, in view of ZHANG, CN 107341947A, further in view of Hogasten; Nicholas (hereafter Hogasten), US 8515196 B, pub. 08/20/2013
As to claim 4, modified Gitzel teaches creating an infrared image as discussed in claim 1 above, but fails to teach “enhancing the infrared image from raw infrared data using dynamic range compression”
On the other hand Hogasten teaches enhancing the infrared image from raw infrared data using dynamic range compression( col.1 lines 39-45, Systems and methods disclosed herein, in accordance with one or more embodiments, relate to dynamic range
compression of infrared images and real time enhancement of infrared images
(e.g., infrared video images) from an infrared )
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate applying a well- known histogram equalization method taught by Hogasten into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of MODIFIDE Gitzel automatically enhance image contrast, thus improve the quality of invisible image.
10.. Claims 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Gitzel, US20230342906 A1, in view of ZHANG, CN 107341947A, further in view McNichols et al., (hereafter McNichols), US 6542767 B1, pub. 04/01/2003
As to claim 9, Gitzel teaches “the combined image” as discussed in claim 1 above, but fails to teach “calibrating the combined image based on user preferences and presenting the combined image on a visual display”
On the other hand McNichols teaches calibrating the combined image based on user preferences and presenting the combined image on a visual display (Fig.4 co.9 lines30-45, Image adjustment display 410 may be used to modify the display of images 235, 240, and/or 245 by GUI 250 based on user preference ).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a graphic user interface that allow user to modify a display of images manually based on user preference taught McNichols into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of modified Gitzel to magnification a display image as user desire using the adjusts the magnification factors of the Zoom adjustment 430 unit.
As to claim 10, McNichols teaches providing controls to a user of the visual display; and altering, via the controls, a field of view of the combined image (Fig.4, co.9 lines 30 -45, Image adjustment display 410 may be used to modify the display of images 235, using Zoom adjustment 430 , where the Zoom adjustment 430 adjusts the magnification factor of images 235, 240, and 245 displayed).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a graphic user interface that allow user to modify a display of images manually based on user preference taught McNichols into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of modified Gitzel to magnification a display image as user desire using the adjusts the magnification factors of the Zoom adjustment 430 unit.
As to claim 11, McNichols teaches o alter the field of view of the combined image includes one or more of identifying regions of interest, zooming with respect to the combined image, and panning with respect to the combined image (Fig.4, co.9 lines 30 -45, Image adjustment display 410 may be used to modify the display of images 235, using Zoom adjustment 430 , where the Zoom adjustment 430 adjusts the magnification factor of images 235, 240, and 245 displayed).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a graphic user interface that allow user to modify a display of images manually based on user preference taught McNichols into modified Gitzel.
The suggestion/motivation for doing so would have been allows user of modified Gitzel to magnification a display image as user desire using the adjusts the magnification factors of the Zoom adjustment 430 unit.
11. Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Gitzel, US20230342906 A1, in view of ZHANG, CN 107341947A, further in view of WANG et al., (hereafter WANG), CN 104361595 A, pub. 02/18/2015
As to claim 8, modified Gitzel teaches “the combined image using a color mapping of the infrared image” as discussed in claim 1 above, but fails to teach the underline limitation of “combined image using a color mapping of the infrared image compared to a standard color mapping.”
On the other hand WANG teaches combined image using a color mapping of the infrared image compared to a standard color mapping ([0076], infrared thermal
image mapping to RGB colour space the colour fusion. working view of the working field of view of color blending and near infrared intensity image and micro light image is consistent infrared thermal image working view part of a color selected in fusion with near infrared intensity image working view part of infrared thermal image field which is corresponding to the colour fusion. wherein the mapping relation of the colour fusion by established comparing with the standard colour palette, it uses standard plate as the target, respectively obtaining standard color palette of near infrared intensity image,)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method mapping infrared thermal image to RGB colour space of standard color palette the colour taught WANG into modified Gitzel.
The suggestion/motivation for doing so would have been enhancing human perception, improving data interpretation, and standardizing communication of thermal data:
12. Claim 13-15 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Gitzel, US 20230342906 A1, in view of ZHANG, CN 107341947A, further in view of in view of LIU et al., (hereafter LIU), CN 110969659 A, pub. 04/07/2020
Regarding claim 13, while Gitzel teaches a memory includes instructions and processing circuitry (as discussed in claim 12 above), but files to teach “ a memory includes instructions and processing circuitry when executed further cause the processing circuitry to recognize a feature of the area using a bounding box and class probability of the feature”
On the other hand LIU teaches the controller is further configured to recognize a feature of the area using a bounding box and class probability of the feature (page 3, last paragraphs, the flame color statistical model obtained by dividing the suspected flame area, and extracting the LBP first moment. circularity, area change rate characteristic value, finally selecting sample data for training SVM classifier learning model for judging whether there is video flame).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method b dividing the suspected flame area, and extracting the LBP first moment. circularity, area change rate characteristic value taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allow user of modified Gitzel.
to generate a training data based on the LBP first moment, circularity, and area change rate characteristic values of infrared image.
As to claim 14, LIU teaches the c processing circuitry uses a convolution neural network to recognize the feature of the area (page 3, last paragraph, selecting sample data for training SVM classifier learning model for judging whether there is video flame based on color the LBP texture).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method b dividing the suspected flame area, and extracting the LBP first moment. circularity, area change rate characteristic value taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allow user of modified Gitzel.
to generate a training data based on the LBP first moment, circularity, and area change rate characteristic values of infrared image.
As to claim 15, LIU teaches a memory includes instructions that when executed further cause the processing circuitry to map the feature to a position within the area( claim 6, detecting robot platform for detecting a certain obstacle information in the range, and judging whether obstacle,.).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method b dividing the suspected flame area, and extracting the LBP first moment. circularity, area change rate characteristic value taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allow user of modified Gitzel.
to generate a training data based on the LBP first moment, circularity, and area change rate characteristic values of infrared image.
As to claim 19, LIU teaches mapping the combined image to a three-dimensional coordinate system to correlate the combined image with a depth of the area(claim 8, first infrared image to be matched electrode line in the third infrared image to be matched, using polar line constraint obtaining the each mark point set in the first infrared image to be matched of the matching point in the third infrared image to be matched, and the matching point pair; calculating all matching point three-dimensional coordinate of the space point corresponding to, and space geometric verification. )
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method b dividing the suspected flame area, and extracting the LBP first moment. circularity, area change rate characteristic value taught by LIU into modified Gitzel.
The suggestion/motivation for doing so would have been allow user of modified Gitzel.
to generate a training data based on the LBP first moment, circularity, and area change rate characteristic value of infrared image.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday -Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR.
Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217-919 (tool-free)
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699
Prosecution Insights