Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s response received has been fully considered and entered.
Response to Arguments
Applicant argues: Hwang and KVIST are not analog art, because Hwang and Kvist works off different principle see pg.8-12 of the argument.
Examiner respectfully disagrees. Hwang and KVIST both works on the object detection using the thermal property of the object, which separates the object from other objects and background: receive the temperature data generated by the temperature sensor, the temperature data (KVIST, ¶ 64) t being indicative of a plurality of average (KVIST, ¶ 63) temperatures (KVIST, ¶ [0034], i.e. With reference to FIGS. 1 to 6, system 1 comprises an IR sensor 2 which is arranged to receive IR radiation in a field of view 4. The field of view 4 has a direction of detection 5. The IR sensor 2 may be any useful type of IR sensor for example a thermopile or a bolometer, where a thermopile is preferred because of the short response time. It is preferred that the IR sensor is of a type that provides absolute temperature values, such as a thermopile. The angle α of the field of view 4 may be from 5° to 120°, where 6° to 45° is more preferred and 7° to 15° is even more preferred. The field of view 4 may have any suitable shape but is preferably circular as shown in FIGS. 3, 5-7, 9-11.), each of the plurality of average temperatures corresponding to a respective one of the plurality of images (¶ [0035]); and determine, based at least in part on the plurality of images and the plurality of average temperatures, a temperature of a desired object (KVIST, ¶41, i.e., [t]he direction 5 of detection of the field of view 4 is preferably fixed in relation to device 7 The position determining means 6 is able to determine the direction 5 of detection of the field of view 4 and the position of the IR sensor 2. The position and the direction 5 are preferably determined in relation to object in the surroundings such as objects in the area of interest 3. The position and direction may preferably be determined in relation to a stationary object. The direction of detection 5 may be the same as a line of observation perpendicular and directed towards surface of a display 11 on device 7 (see FIGS. 5 and 6). A user that holds device 7 with display 11 in the hand will then observe the display 11 in the same direction as the direction of detection 5 of IR sensor 2, which is an advantage during real time display of thermal images (see below).) identified in the image data (as cited above (KVIST, ¶ 63),
Applicant argues Hwang does not teach average temperature and Kvist does not teach object recognition, see pg. 11 of the argument. Examiner relies on Hwang for object detection and Kvist for teaching the average temperature of pixeled temperature; see citation above.
Applicant further argues combination of teaching failed because Kvist teaching IR sensor is one value, see pg. 13-14 of the arguments. However, (KVIST, ¶ 63-64 specifically teaches how find the temperatures of the object by average the temperature from the thermal sensors: i.e.: Thus, for each pixel 13 in the image 12, the temperature value may be the result of using temperature data for a plurality of temperature values, for example the average of the plurality of temperature values. Therefore, the combined teaching of the two disclose wherein the infrared sensor is a single-pixel (Kvist, para. 34) infrared sensor (HWANG, section 1, Fig. 3, both of them teach the IR sensors could be one sensor, or a few more, and one can form the images by scanning the area using the sensors).
Applicant further argue claim 3 see pg. 14 of the argument about Hwang did not discloses the temperature detection of the object. This is not found to be persuasive, because Hwang Fig. 1 shows a thermal images with different temperature shown. IR camera essentially shows the temperature of the objects.
Applicant further argues claim 12, arguing Hwang is teaching more than one sensor. This is not found to be persuasive. Because Both of the two references noticed that one can use less sensors, which scanning to expand the pixel based: wherein the infrared sensor is a single-pixel (Kvist, para. 34) infrared sensor (HWANG, section 1, Fig. 3, both of them teach the IR sensors could be one sensor, or a few more, and one can form the images by scanning the area using the sensors).
Applicant’s amendment and the related arguments have been considered. It is noted that the Examiner did consider the applicants amendment; however, the amendment does not require further search since the prior art of record do teach the additional limitations. The application is rejected while correlating the newly amended limitations.
Applicant’s arguments have been considered, but the arguments are not deemed to be persuasive. The application is rejected while correlating the newly amended limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang et. Al. “Multispectral pedestrian detection: Benchmark dataset and baseline”, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 1037-1045, doi: 10.1109/CVPR.2015.7298706, “Hwang”, in view of KVIST US 2022349757 “KVIST”.
Regarding claim 1, HWANG discloses a system for determining a temperature of an object (HWANG, abstract), the system comprising: multispectral ACF, which is an extension of aggregated channel features (ACF) to simultaneously handle color-thermal image pairs (HWANG, abstract).
It is noted that HWANG is silent about the device that receive the image data generated by the camera, the image data being indicative of a plurality of images, each of the plurality of images being associated with a different respective position of the mobile device such that the first field of view in each of the plurality of images only partially spatially overlaps the first field of view in each other image of the plurality of images; receive the temperature data generated by the temperature sensor, the temperature data being indicative of a plurality of average temperatures, each of the plurality of average temperatures corresponding to a respective one of the plurality of images; and determine, based at least in part on the plurality of images and the plurality of average temperatures, a temperature of a desired object, the desired object being at least partially within both the first field of view and the second field of view during generation of the image data and the temperature data.as claimed.
However, KVIST discloses the system including a mobile device (KVIST, see element 7, and ¶ [0040] Preferably the IR sensor 2 and positioning determining means 6 are comprised in a device 7, which preferably is a device that can be handheld. The device 7 may be a mobile phone, such as an iPhone or an Android phone. Nevertheless, parts of system 1 may located outside device 7, such as on a server. In particular, memory 8 and processor 9 or parts of or memory 8 and processor 9 may be located on a server which is in digital communication, for example wireless communication, with device 7.), comprising: a camera configured to generate image data across a first field of view (see ¶ [0012]: the position and orientation of the device are determined using tracking cameras, i.e. the camera of the mobile phone); and a temperature sensor (see KVIST, element 2; ¶ [0009]) configured to generate temperature data across a second field of view (KVIST ¶ [0034]), the first field of view and the second field of view at least partially overlapping (KVIST, as cited above about the mobile device with cameras, such cameras within the mobile have the FOVs partially overlapping, for example, as long as the FOV is more than zero, then eventually the FOVs will overlap for the plurality of the cameras); and a computing system (KVIST, abstract) configured to: receive the image data generated by the camera, the image data being indicative of a plurality of images (as cited below, ¶ 11), each of the plurality of images being associated with a different respective position (as cited below, ¶ 11) of the mobile device such that the first field of view in each of the plurality of images only partially spatially overlaps the first field of view in each other image of the plurality of images (see KVIST, where the camera takes a series of images as the device is being swept over the area of interest, see i.e. ¶ [0011]); receive the temperature data generated by the temperature sensor, the temperature data (KVIST, ¶ 64) t being indicative of a plurality of average (KVIST, ¶ 63) temperatures (KVIST, ¶ [0034], i.e. With reference to FIGS. 1 to 6, system 1 comprises an IR sensor 2 which is arranged to receive IR radiation in a field of view 4. The field of view 4 has a direction of detection 5. The IR sensor 2 may be any useful type of IR sensor for example a thermopile or a bolometer, where a thermopile is preferred because of the short response time. It is preferred that the IR sensor is of a type that provides absolute temperature values, such as a thermopile. The angle α of the field of view 4 may be from 5° to 120°, where 6° to 45° is more preferred and 7° to 15° is even more preferred. The field of view 4 may have any suitable shape but is preferably circular as shown in FIGS. 3, 5-7, 9-11.), each of the plurality of average temperatures corresponding to a respective one of the plurality of images (¶ [0035]); and determine, based at least in part on the plurality of images and the plurality of average temperatures, a temperature of a desired object (KVIST, ¶41, i.e., [t]he direction 5 of detection of the field of view 4 is preferably fixed in relation to device 7 The position determining means 6 is able to determine the direction 5 of detection of the field of view 4 and the position of the IR sensor 2. The position and the direction 5 are preferably determined in relation to object in the surroundings such as objects in the area of interest 3. The position and direction may preferably be determined in relation to a stationary object. The direction of detection 5 may be the same as a line of observation perpendicular and directed towards surface of a display 11 on device 7 (see FIGS. 5 and 6). A user that holds device 7 with display 11 in the hand will then observe the display 11 in the same direction as the direction of detection 5 of IR sensor 2, which is an advantage during real time display of thermal images (see below).) identified in the image data (as cited above (KVIST, ¶ 63), the desired object being at least partially within both the first field of view and the second field of view during generation of the image data and the temperature data (see ¶ 14, i.e. d) using the determined IR sensor values, together with their respective detected positions and orientations, to determine the thermal image of the area, where a temperature for a pixel in the image is determined by determining temperatures for two partially overlapping fields of view, such that an overlap between two fields of view are associated with fields not common for the two overlapping fields of view (non-common fields), and where a temperature value for a non-common field is determined by using the temperature difference between the two partially overlapping fields of view and the proportion of the area of the non-common field in relation to the area of the field of view, and where the temperature for a pixel in the thermal image data is determined by using the temperature values thus determined.).
Both HWANG and KVIST teach systems with thermal images with object recognition, and those systems are comparable to that of the instant application. Because the two cited references are analogous to the instant application, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains, to include in the HWANG’s disclosure, overlapping the two cameras’ FOVs, as taught by KVIST. Such inclusion would have increased the usefulness of the system by a simple and low-cost thermographic camera, and would have been consistent with the rationale of combining prior art elements according to known methods to yield predictable results to show a prima facie case of obviousness (MPEP 2143(I)(A)) under KSR International Co. v. Teleflex Inc., 127 S. Ct. 1727, 82 USPQ2d 1385, 1395-97 (2007).
Regarding claim 2, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 1, wherein the desired object extends across only part of the second field of view during generation of the temperature data (see KVIST, ¶ 34, as cited above).
Regarding claim 3, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 1, wherein the computing system is further configured to: determine, for each of the plurality of images (HWANG, pg. 1038, sect. 2), an overlap region where the second field of view overlaps the first field of view (as cited above HWANG, pg. 1038, sect. 2); identify, for each of the plurality of images, at least one object within the overlap region (HWANG, pg. 1044, Fig. 11), the at least one object including the desired object (HWANG, pg. 1044, Fig. 11); and determine, for each respective object of the at least one object in each of the plurality of images, a respective portion of the overlap region that includes the respective object (HWANG, pg. 1044, Fig. 11), wherein the computing system is configured to determine the temperature of the desired object based at least in part on the respective portion of the overlap region for each respective object in each of the plurality of images and the average temperature of the plurality of average temperatures associated with each of the plurality of images (HWANG, pg. 1042, sec. 3).
Regarding claim 4, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 3, wherein the computing system is further configured to divide the overlap region for each of the plurality of images into a plurality of zones, each zone of the plurality of zones being associated with a different weight (HWANG, Fig. 9, pg. 1042, section 2), wherein the computing system is configured to determine, for each respective object of the at least one object in each of the plurality of images, the respective portion of the overlap region that includes the respective object by determining, for each respective object of the at least one object in each of the plurality of images, a respective portion of each zone of the plurality of zones in the overlap region that includes the respective object (HWANG, as cited above), wherein the computing system is configured to determine the temperature of the desired object based at least in part on the respective portion of each zone of the plurality of zones for each respective object of the at least one object in each of the plurality of images, the weight of each zone of the plurality of zones, and the average temperature associated with each of the plurality of images (HWANG, Fig. 9, pg. 1042, section 2).
Regarding claim 5, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 4, wherein the respective portion of each zone of the plurality of zones for one or more of the at least one object varies across each of the plurality of images (HWANG, Fig. 9, pg. 1042, section 2).
Regarding claim 6, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 3, wherein the computing system is configured to determine the temperature of the desired object by performing linear analysis on the respective portion of the overlap region for each respective object in each of the plurality of images and the average temperature associated with each of the plurality of images (HWANG, Fig. 10, pg. 1043, section 3).
Regarding claim 7, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 1, wherein the computing system is further configured to: receive an initial request via a user interface indicative of a request to begin using the temperature sensor of the mobile device; and control the camera to begin generating the image data in response to receiving the initial request (HWANG, pg. 1039, section 2).
Regarding claim 8, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 1, wherein the computing system is further configured to: control an operation of a user interface to display at least a portion of the image data generated by the camera (HWANG, Fig. 4, pg. 1039); receive a selection input via the user interface indicative of a selection of the desired object from the at least the portion of the image data (HWANG, Fig. 4); and control the operation of the user interface to provide the temperature of the desired object (KVIST, controlling the operation of the user interface to provide the temperature of the desired object is intended usage, since providing such a temperature is the purpose of these devices. Further, controlling an operation of the user interface to display at least a portion of the image data generated by the camera is. Finally, the step of receiving a selection input via the user interface indicative of a selection of the desired object from the at least the portion of the image data cannot be seen to solve any particular technical problem, and can therefore not involve an inventive step.)
Regarding claim 9, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 8, wherein the computing system is further configured to: identify at least one other object within the image data aside from the desired object (D3, Fig. 7, pg. 1040); and control the operation of the user interface to request an operator move the mobile device based on a number of the at least one other object (KVIST, Fig. 15).
Regarding claim 10, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 8, wherein the computing system is further configured to control the temperature sensor to begin generating the temperature data in response to receiving the selection input (HWANG, Fig. 9, i.e. generating the temperature after annotation).
Regarding claim 11, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 1, wherein the temperature sensor is an infrared sensor (HWANG, pg. 1037).
Regarding claim 12, HWANG /KVIST, for the same motivation of combination, further discloses the system of claim 11, wherein the infrared sensor is a single-pixel (Kvist, para. 34) infrared sensor (HWANG, section 1, Fig. 3, both of them teach the IR sensors could be one sensor, or a few more, and one can form the images by scanning the area using the sensors).
Regarding claim 13, HWANG /KVIST, for the same motivation of combination, discloses a method for determining a temperature of an object, the method comprising: receiving, with a computing system, image data generated by a camera of a mobile device, the camera having a first field of view, the image data being indicative of a plurality of images, each of the plurality of images being associated with a different respective position of the mobile device such that the first field of view in each of the plurality of images only partially spatially overlaps the first field of view in each other image of the plurality of images (see rejection of claim 1); receiving, with the computing system, temperature data generated by a temperature sensor of the mobile device, the temperature sensor having a second field of view, the first field of view and the second field of view at least partially overlapping, the temperature data being indicative of a plurality of average temperatures, each of the plurality of average temperatures corresponding to a respective one of the plurality of images (see rejection of claim 1); determining, with the computing system, a temperature of a desired object based at least in part on the plurality of images and the plurality of average temperatures, the desired object being at least partially within both the first field of view and the second field of view during generation of the image data and the temperature data (see rejection of claim 1); and controlling, with the computing system, a user interface to provide the temperature of the desired object (see rejection of claim 1); the desired object being identified in the image data see rejection of claim 1).
Regarding claim 14, HWANG /KVIST, for the same motivation of combination, further discloses the method of claim 13, further comprising: determining, with the computing system for each of the plurality of images, an overlap region where the second field of view overlaps the first field of view; identifying, with the computing system for each of the plurality of images, at least one object within the overlap region, the at least one object including the desired object; and determining, with the computing system, for each respective object of the at least one object in each of the plurality of images, a respective portion of the overlap region that includes the respective object, wherein determining, with the computing system, the temperature of the desired object comprises determining the temperature of the desired object based at least in part on the respective portion of the overlap region for each respective object in each of the plurality of images and the average temperature of the plurality of average temperatures associated with each of the plurality of images (see rejection of claim 3).
Regarding claim 15, HWANG /KVIST, for the same motivation of combination, further discloses the method of claim 14, further comprising dividing, with the computing system, the overlap region for each of the plurality of images into a plurality of zones, each zone of the plurality of zones being associated with a different weight, wherein determining, with the computing system, for each respective object of the at least one object in each of the plurality of images, the respective portion of the overlap region that includes the respective object comprises determining, for each respective object of the at least one object in each of the plurality of images, a respective portion of each zone of the plurality of zones in the overlap region that includes the respective object, wherein determining, with the computing system, the temperature of the desired object comprises determining the temperature of the desired object based at least in part on the respective portion of each zone of the plurality of zones for each respective object of the at least one object in each of the plurality of images, the weight of each zone of the plurality of zones, and the average temperature associated with each of the plurality of images (see rejection of claim 4).
Regarding claim 16, HWANG /KVIST, for the same motivation of combination, further discloses the method of claim 14, wherein determining, with the computing system, the temperature of the desired object comprises determining the temperature of the desired object by performing linear analysis on the respective portion of the overlap region for each respective object in each of the plurality of images and the average temperature associated with each of the plurality of images (see rejection of claim 6).
Regarding claim 17, HWANG /KVIST, for the same motivation of combination, further discloses the method of claim 13, further comprising: receiving, with the computing system, an initial request via a user interface indicative of a request to begin using the temperature sensor of the mobile device; and controlling, with the computing system, the camera to begin generating the image data in response to receiving the initial request (see rejection of claim 7).
Regarding claim 18, HWANG /KVIST, for the same motivation of combination, further discloses the method of claim 13, further comprising: controlling, with the computing system, an operation of a user interface to display at least a portion of the image data generated by the camera; receiving, with the computing system, a selection input via the user interface indicative of a selection of the desired object from the at least the portion of the image data; and controlling, with the computing system, the operation of the user interface to provide the temperature of the desired object (see rejection of claim 8).
Regarding claim 19, HWANG /KVIST, for the same motivation of combination, further discloses the method of claim 18, further comprising: identifying, with the computing system, at least one other object within the image data aside from the desired object; and controlling, with the computing system, the operation of the user interface to request an operator move the mobile device based on a number of the at least one other object (see rejection of claim 9).
Regarding claim 20, HWANG /KVIST, for the same motivation of combination, further discloses the method of claim 18, further comprising: controlling, with the computing system, the temperature sensor to begin generating the temperature data in response to receiving the selection input (see rejection of claim 10).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 20210393137 A1 SKIN TONE CORRECTION FOR BODY TEMPERATURE ESTIMATION
US 20200309604 A1 SELF-REFERENCED AMBIENT RADIATION THERMOMETER AND PROCESS FOR DETERMINING A TEMPERATURE OF A BLACKBODY OBJECT
US 20190182439 A1 THERMAL IMAGE PROCESSING SYSTEM AND METHOD
US 9534958 B1 Monitoring using passive infra-red sensing
US 20160027171 A1 Method of Monitoring the Status of a Wound
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK F HUANG whose telephone number is (571)272-0701. The examiner can normally be reached Monday-Friday, 8:30 am - 6:00 pm (Eastern Time), Federal Alternative First Friday Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached at (571)272-2988.. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK F HUANG/ Primary Examiner, Art Unit 2485
Prosecution Insights