DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1, 11, and 20 are amended. Claims 2, 4-10, 12, and 14-19 are as previously presented. Claims 3 and 13 are cancelled. Therefore, claims 1-2, 4-12, and 14-20 are currently pending and have been considered below.
Response to Amendment
The amendment filed on November 19, 2025 has been entered.
Response to Arguments
Applicant's arguments regarding the lack of “a location of the fire relative to cookware within the image” have been fully considered but they are not persuasive.
Applicant argues that combination of the prior art does not disclose where the location of a fire relative to cookware within an image is determined.
The primary reference Livchak includes imaging of a cookware that includes a heat source during normal operations, Page 9, Para. 3 from end, “spectral information of the time-varying signal and the steady-state signal during operation are relevant independent DOF signals that can help identify normal processes and events with fires in the cooking environment… Can produce Changes and fluctuations in radiation temperature of the cooking surface may indicate flipping or placing of food on the grill or movement of hands or kitchen utensils or cooking vessels on the cooking surface”, where the cooking vessel or cookware is imaged along with a heat source at a normal operating position.
The secondary reference Mori then discloses where the heat source location is predetermined and registered, where a risk value is assigned depending on the distance that the fire is from a predetermined location, Page 8, Para. 1, “To calculate the fire risk, it is necessary to give information on the heat source whose location is specified. Therefore, the average center of gravity (X .sub.i , Y .sub.i ) and the size S .sub.i of the heat source are registered in advance. Here, i represents the registration number of the heat source (i is i> 0). Further, regarding these registered heat sources, the risk degrees .sub.Atl , .sub.i with respect to the heat source generation time that is assumed to be normally generated are registered in advance.”.
It is the Examiner’s position that combining Livchak with Mori would result in the heat source located at the cooktop with a cooking vessel being placed on top from Livchak be set as the predetermined heat source location for Mori, where this heat source position would be relative to a cooking vessel and where the fire position determined from Mori would continue to be relative to a position of a cooking vessel as it is relative to the predetermined heat source position. It is for this reason that the Examiner’s finds applicant’s argument to not be persuasive.
Applicant’s arguments, see Page 8, filed on 11/19/2025, with respect to the rejection(s) of claim(s) 1-2, 4-12, and 14-20 under U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of applicant’s amendment regarding the type of food being prepared and newly found prior art regarding that feature.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 4-6, 9-11, 14-16, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Livchak et al. (KR 20190139229 A, hereinafter Livchak) in view of Gurajapu et al. (EP 3748444 A1, hereinafter Gurajapu) and Mori et al. (JP 2006072863 A, hereinafter Mori) and Babu et al. (US 20180003392 A1, hereinafter Babu) and Bailey (WO 2020144445 A1).
Regarding claim 1, Livchak discloses a method for controlling a stovetop system (Page 2, Para. 2, lines 1-2, “…fire suppression systems for use in cooktops or hoods located above the range allow the fire retardant material to be transferred over the cooking surface”), the method comprising:
accessing a set of training data, the training data comprising image data of stovetops systems labeled based on presence of smoke (Page 5, Para. 3, lines 2-5, “The camera 599 may be selected based on a wide range of optical and near infrared frequencies. The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood. Smoke, and of course fire, should not be visible from above the hood 621 in normal circumstances.”, where there is a camera takes images of areas around a stovetop with the presence of smoke, where training is done, Page 19, Para. 3 from end, lines 1-3, “Devices and methods for receiving sensor signals and outputting information identifying various possible situations, such as a fire situation or a smoke load situation, indicated by the values of the sensor signals, can generally be identified with classifiers…”, and Page 20, Para. 1, lines 3-4, “…use machine learning to construct detectors or filters through supervised learning from training images.”);
training, using the set of training data, a machine learning model configured to detect smoke in images of a stovetop system (Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the training data is used during the recognition algorithm, Para. 20, last Para., lines 1-5, “Recognizing objects such as people in an image using a pattern recognition approach is a known technique called computer vision and includes face recognition. Known technology may use 3D scanners (infrared, autonomous vehicles, and product inspection systems such as Microsoft Kinect). Examples include face recognition and pedestrian detection. Many such approaches are known. Some use machine learning to construct detectors or filters through supervised learning from training images.”);
applying the machine learning model to received images to determine a likelihood that the received images include smoke (Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the recognition algorithm is able to determine the likelihood that smoke is present through confidence estimates, Page 20, Para. 1, lines 6-9, “Algorithm optimization of a fit (such as one type of regression) can produce an estimate of the fit's excellence with the best-fitting pattern, so these pattern matching systems provide classification (best pattern fit) and confidence estimates (Measure both the fit).”); and
responsive to determining that the received images include smoke, sending an alert indicative of smoke at the stovetop (Page 5, Para. 2 from end, last 8 lines, “Thus, in general, significant leakage of all smoke will be the result of unusual equipment misuse, fire, or failure of the exhaust system…According to a method embodiment, wherein the processor is encoded by executable instructions, the controller provides a warning signal indicating that a preliminary fire detection has occurred, thereby alerting the operator to the need to prevent an alarm or suppressive output with an override input.”).
Livchak does not disclose:
training data with a first set of images including the presence of smoke, a second set of images including the presence of steam, each image in the first of images and the second set of images further labeled with a measure of risk posed to a kitchen, wherein the measure of risk associated with each image is determined based on an amount of the image associated with fire, a type of food being prepared, and a location of the fire relative to cookware within the image, and training based on the training data a machine learning model to differentiate between smoke and steam within the images and to identify a risk posed to the kitchen;
receiving real time images of stovetops and determining whether there is smoke;
specifically controlling a stove where after determining images to be smoke, the stovetop is disabled.
However, Gurajapu discloses, in the similar field of machine learning models for analyzing features of flames (Abstract, “A method and system for advanced flare analytics in a flare operation monitoring and control system… A machine learning-based self-adaptive industrial automation system…”), where training data for machine learning models can further include images that are labelled to have smoke and steam sections (Para. 0015, “The machine learning may begin by training the model with images that have been annotated as portions that are black smoke, steam, and flame. In one embodiment, the annotation may be manual markings as to which portions are black smoke, which are steam, and which are flame. The annotated images are input into the machine learning/ deep learning core and then the system may be fine-tuned further to arrive at the best results. The training data may then be separated from the test data to be analyzed.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model regarding smoke from stovetops in Livchak to include additional features of distinguishing between smoke, steam, and flame, as taught by Gurajapu, which can be applied to images of the stovetops from Livchak. Examiner notes that although Gurajapu focuses on industrial plants, the machine learning model for detecting different particles could easily be applied to any other image, such as steam/smoke from stovetops.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of machine learning to provide more reliable and consistent detection of regions of smoke, steam, and flame down to the pixel level, as stated by Gurajapu, Para. 0014, “The regions of smoke, steam, and flame are not defined shape objects as they are nonuniform and therefore previous image processing techniques were not able to sufficiently distinguish the specific categories. The machine learning/ deep learning core provides a step forward in that the deep learning core is able to distinguish these categories in a significantly more reliable and consistent manner.”,
where being able to detect steam from smoke allows for more accurate alert detection of the system being monitored, as smoking, excessive steam, and other tailored alerts can only be done if the system can be detect what type of particle is being emitted, as stated by Gurajapu, Para. 0022, “Alerts manager 138 which communicates with and may be controlled by DVM 126, may be used to inform operators regarding out of limits situations in any flare. Smoking, excessive steam and the like are examples of alerts.”.
Mori discloses, in the similar field of fire prevention systems (Abstract, “a fire prevention system quantitatively computing a fire danger degree”), where images can be provided with the risk posed to a kitchen (Page 4, Para. 2, “An image processing unit that extracts a feature quantity of the heat source from the image information obtained by the heat source detection unit, an information storage unit that pre-registers information about the heat source and information about the environment around the heat source, and an image processing unit…An analysis unit that calculates the fire risk of the heat source using an image processing unit, and an output unit that outputs a calculation result relating to the feature amount of the heat source obtained by the image processing unit and / or the fire risk obtained by the analysis unit”, where images have a feature quantity extracted and compared to predetermined value, where that comparison outputs the amount of risk posed to the kitchen, Page 6, Para. 3 from end, “In this environment, the cooking site in the kitchen is assumed”), where the risk or danger is determined based on the amount of the image associated with fire and the location/position of the fire relative to its normal position (Page 7, Para. 5 from end, “That is, in the present embodiment, what is extracted as the feature amount is the position of the center of gravity and the size of the cluster assigned 1 by labeling.”, and Page 11, Para. 3 from end, “The information transmission unit 17 receives the position and size of the heat source obtained by the image processing unit 2 and the fire risk A obtained by the signal processing unit from the result output unit 6”, where the feature amount extracted from each image includes the amount/size of the fire and location/position of the fire). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model in modified Livchak to include the size and location determination of each image to output a risk associated with each image as taught by Mori;
where the both steam and smoke images could be used in the risk determination process from Mori as the feature quantity is calculated after a light intensity conversion, Page 6, Para. 5 from end, “The image processing unit 2 in FIG. 1 performs A / D conversion on, for example, two-dimensional analog image data from the image sensor 1C, converts it into, for example, an 8-bit signal according to the light intensity, and performs signal processing in the subsequent stage.”, and Page 6, last Para., “Here, the processing after A / D conversion will be further described as follows. First, the binarization threshold V .sub.th is determined so that the heat source to be monitored can be sensed.”, where binarization occurs after light intensity processing to determine the size of the fire, where for steam images no risk would occur as no pixels would satisfy the light intensity filtering;
where a cooking vessel located with a heat source is imaged from Livchak, Page 9, Para. 3 from end, “spectral information of the time-varying signal and the steady-state signal during operation are relevant independent DOF signals that can help identify normal processes and events with fires in the cooking environment… Can produce Changes and fluctuations in radiation temperature of the cooking surface may indicate flipping or placing of food on the grill or movement of hands or kitchen utensils or cooking vessels on the cooking surface”, where Mori discloses where a heat source location is predetermined and registered and where a risk value is assigned depending on the distance that the fire is from a predetermined location, Page 8, Para. 1, “To calculate the fire risk, it is necessary to give information on the heat source whose location is specified. Therefore, the average center of gravity (X .sub.i , Y .sub.i ) and the size S .sub.i of the heat source are registered in advance. Here, i represents the registration number of the heat source (i is i> 0). Further, regarding these registered heat sources, the risk degrees .sub.Atl , .sub.i with respect to the heat source generation time that is assumed to be normally generated are registered in advance.”, where combining Livchak with Mori would result in the heat source located at the cooktop with a cooking vessel being placed on top from Livchak be set as the predetermined heat source location for Mori, where this heat source position would be relative to a cooking vessel and where the fire position determined from Mori would continue to be relative to a position of a cooking vessel as it is relative to the predetermined heat source position.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of using quantitative measurements in order to determine the risk posed by fires within images, which can improve fire detection accuracy and allow a determination into the severity of the fire occur earlier than a conventional fire alarm, as stated by Mori, Page 4, Para. 1, “potential properties of the heat source is quantitatively used. It is an object of the present invention to provide a fire prevention system capable of preventing a fire in advance by calculating the degree of danger so as to be highly accurate and capable of making a determination at an early stage.”.
Bailey discloses, in the similar field of determining risk of fires from images (Page 3, lines 21-23, “method to detect pre-ignition conditions on a cooking hob comprising: analysing an infrared image of a hob to locate hot plates, burners or pans on the image”), where a risk of fire from an image can be dependent on the type of food being prepared (Page 13, lines 8-11, “As an example, the criteria may reflect the nature of the foodstuff being cooked and indicate an acceptable maximum temperature, an acceptable time at maximum temperature, and an acceptable total time of cooking. Deviation from these criteria may, for example, indicate: preignition conditions, overcooked food, or an unattended hob respectively.”, where depending on the type of food being prepared or the nature of the foodstuff being cooked, there can be different acceptable criteria used for determining the risk of fire or the preignition conditions, Page 28, lines 8-10, “Further heating without attention from the user is likely to result in a fire in the food, so an alert is set for a pre-ignition risk”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the fire risk determination in modified Livchak to further include a fire risk dependent on the type of food being prepared as taught by Bailey; where Mori teaches fire risk determination based on the position of the heat source from a predetermined location and where Bailey improves upon the fire risk determination through setting different fire risk thresholds at the predetermined location based on the food type being cooked.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to take into consideration the different cooking conditions related to different food items, which can alter the thresholds for determining risk of a fire, as stated by Bailey, Page 13, lines 8-11, “As an example, the criteria may reflect the nature of the foodstuff being cooked and indicate an acceptable maximum temperature, an acceptable time at maximum temperature, and an acceptable total time of cooking. Deviation from these criteria may, for example, indicate: preignition conditions, overcooked food, or an unattended hob respectively.”.
Further, Babu discloses, in the similar field of images for detecting smoke from a stovetop (Para. 0170, lines 17-19 from end, “…camera in conjunction with a controller may utilize one or more lenses to detect smoke or flame emitting from a stove top.”), where once smoke is determined to occur from real time images, the stovetop can be turned off (Para. 0209, lines 4-end, “In block 2820, the apparatus receives one or more parameters of one or more monitoring signals from one or more sensors…If the detected parameter exceeds the predetermined threshold, the apparatus will send a control signal to the motor, which in turn drives the control knob and/or operational shaft to turn off the burner…”, where such parameters include smoke, Para. 0212, last 4 lines, “…smoke sensor 1304a may send a monitoring signal which indicates that a certain level of particulate smoke (a parameter) has been detected by the sensor.”, where the camera is also able to detect smoke as stated above). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the controller and smoke threshold from images in modified Livchak to include disabling the stovetop when smoke exceeds a threshold as taught by Babu.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to automatically turn off the power to the stovetop when there is no one present, as stated by Babu, Para. 0004, lines 1-4, “For the foregoing reasons, there is a need for a safety device for automatically rotating an operational shaft of a burner to an Off position upon the occurrence of a safety event for shutting off power to the burner.”.
Regarding claim 4, modified Livchak teaches the method according to claim 1, as set forth above, discloses wherein the received images are determined to include smoke in response to the machine learning model determining that there is an likelihood that the received images include smoke (Livchak, Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the recognition algorithm is able to determine the likelihood that smoke is present through confidence estimates, Page 20, Para. 1, lines 6-9, “Algorithm optimization of a fit (such as one type of regression) can produce an estimate of the fit's excellence with the best-fitting pattern, so these pattern matching systems provide classification (best pattern fit) and confidence estimates (Measure both the fit).”).
Modified Livchak does not disclose:
where the likelihood from the machine learning model includes an above-threshold value.
However, Babu discloses the ability to compare a monitored signal to a predetermined threshold, where parameters can include a range of sensor data (Para. 0209, lines 6-14, “Such parameters may include, but not limited to, motion, temperature, humidity, CO level, CO2 level, natural gas level, propane level, butane level, and/or pollution levels. Upon receiving a parameter of a monitoring signal, the apparatus compares the parameter with the predetermined threshold stored in the database 2830 which may reside on the memory of the controller of the safety device or may be stored externally from the safety device and accessed.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model in Livchak to further include applying threshold comparisons for smoke sensor data, as taught by Babu.
Regarding the use of a machine learning model to be applied for determining if there is smoke within an image, it is the Examiner's position that one of ordinary skill in the art would have found it obvious to try as Livchak implies the use of thresholds through confidence estimates in using the recognition algorithm. Thus, smoke from images is determined by the recognition algorithm by comparing the image with training data, then an estimate to the likelihood that smoke is within the produced through the confidence estimate. Babu discloses a similar feature, but through the use of sensor data values that are compared with specific thresholds, where the sensor data values must be higher than the thresholds. Since both Livchak and Babu compare sensor data with stored data to reach conclusions, the end results are similar and it would be obvious to try implementing a specific confidence threshold (like the threshold in Babu that needs to be exceeded) that Livchak’s recognition algorithm would need to exceed in order to determine if smoke is present in the image.
Regarding claim 5, modified Livchak teaches the method according to claim 1, as set forth above, discloses wherein determining that the received images include smoke comprises determining that the received images include an above-threshold amount of smoke (Livchak, Page 8, Para. 2 from end, lines 3-8 from end, “Various devices such as photocells, charge coupled devices, optical multipliers, and camera sensors can be used as the optical sensors. The controller can be set to detect a duration of high smoke levels and trigger on or based on both duration and opacity or a combination thereof. As discussed, this signal may be combined with other signals to identify the fire, specify the location of the fire, and in combination with other factors to determine the size of the fire.”, where cameras can take images of smoke and high smoke levels are a threshold that the controller determines to mean that smoke is present).
Modified Livchak does not disclose:
wherein the threshold amount of smoke may be adjusted via a user interface presented to a client device of a user of the stovetop.
However, Babu discloses where the user is able to control the threshold amount of smoke for triggering an alert by being able to turn off the smoke sensor via a user device (Para. 0206, last 7 lines, “The user may also be able to override or temporarily turn one or more of the sensors in the sensor/relay module 1304 off. For example, if a user is cooking and producing large amounts of smoke, but knows there is no fire, the user may utilize user device 1306 to temporality turn off a smoke sensor in the sensor/relay device.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the smoke detection algorithm in modified Livchak to include the ability to adjust the threshold values as taught by Babu.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to cook foods that have a high smoke output and not cause the alarm system to be triggered, as stated by Babu, Para. 0206, last 7 lines, “The user may also be able to override or temporarily turn one or more of the sensors in the sensor/relay module 1304 off. For example, if a user is cooking and producing large amounts of smoke, but knows there is no fire, the user may utilize user device 1306 to temporality turn off a smoke sensor in the sensor/relay device.”.
Regarding claim 6, modified Livchak teaches the method according to claim 1, as set forth above, discloses herein the alert is sent via a user interface presented to a client device of a user of the stovetop (Livchak, Page 6, Para. 2, last 3 lines, “The controller can trigger an alarm in
response to a danger signal. As described elsewhere, the controller can accept an override command from the user interface and disable the alarm in response to the override command.”, where a client device that connects to the internet can also issue the alarm, Page 6, Para. 3, lines 9-12, “Another way in which an override command can be applied to the controller 600 is via the smartphone 668 interface or other portable interface. In one embodiment, the controller (or service connected by network to the user interface) is programmed to output information about the preliminary fire detection…”).
Regarding claim 9, modified Livchak teaches the method according to claim 1, as set forth above.
Modified Livchak does not disclose:
wherein disabling operation of the stovetop comprises actuating a mechanical stovetop controller configured to turn off the stovetop.
However, Babu discloses where turning off the stovetop includes actuating a mechanical controller (Para. 0040, lines 5-end, “…the knob adapter member is structured to mimic an attachment part of the operational shaft, such that the top knob can be attached to the top of the safety device module. In some examples, the top knob comprises an original knob for operational control of the burner. In some examples, the controller is configured to cause the motor to turn the operational shaft of the burner to an Off position.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the turn off mechanism in modified Livchak to include the safety device module that attaches to the stovetop burner controller as taught by Babu.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to automatically turn off the power to the stovetop when there is no one present, as stated by Babu, Para. 0004, lines 1-4, “For the foregoing reasons, there is a need for a safety device for automatically rotating an operational shaft of a burner to an Off position upon the occurrence of a safety event for shutting off power to the burner.”.
Regarding claim 10, modified Livchak teaches the method according to claim 1, as set forth above, discloses wherein the machine learning model is further configured to detect fire in images of a stovetop that include smoke (Livchak, Page 12, Para. 2 from end, lines 2-7, “Classifiers that use supervised learning to generate classifications and associated confidence levels are known. They can be used to detect fires. Sensors actively monitor different indicators. The controller stores data samples from several sensors. The system determines the likelihood that an indicator reduced from the sensor output, such as a single sensor output or a combination of single sensor outputs, or operational statistics, alone or in combination with other data, represents a fire.”, where such sensor outputs includes images of smoke; Teaching from Cha regarding the imaging of stovetops, Para. 0180, last 4 lines, “The camera may photograph a cooking situation ( e.g., overheating of a cooking container ( or object to be cooked), over cook, or whether or not smoke is generated) as a still image or a video.”, where the cooking container is located on the stovetop and thus the camera would capture images of the stovetop).
Regarding claim 11, Livchak discloses a non-transitory computer-readable storage medium comprising instructions (Page 12, last Para., lines 2-4, “…algorithms and models
are provided in the data store and And / or stored in memory to provide access to them by the
classifier.”) executable by a processor (Page 12, Para. 3 from end, lines 2-3 from end, “The classifier and upstream and downstream processing may be performed by controller 600…”), the instructions comprising:
instructions for accessing a set of training data, the training data comprising image data of stovetop systems labeled based on presence of smoke (Page 5, Para. 3, lines 2-5, “The camera 599 may be selected based on a wide range of optical and near infrared frequencies. The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood. Smoke, and of course fire, should not be visible from above the hood 621 in normal circumstances.”, where there is a camera takes images of areas around a stovetop with the presence of smoke, where training is done, Page 19, Para. 3 from end, lines 1-3, “Devices and methods for receiving sensor signals and outputting information identifying various possible situations, such as a fire situation or a smoke load situation, indicated by the values of the sensor signals, can generally be identified with classifiers…”, and Page 20, Para. 1, lines 3-4, “…use machine learning to construct detectors or filters through supervised learning from training images.”);
instructions for training, using the set of training data, a machine learning model configured to detect smoke in images of a stovetop system (Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the training data is used during the recognition algorithm, Para. 20, last Para., lines 1-5, “Recognizing objects such as people in an image using a pattern recognition approach is a known technique called computer vision and includes face recognition. Known technology may use 3D scanners (infrared, autonomous vehicles, and product inspection systems such as Microsoft Kinect). Examples include face recognition and pedestrian detection. Many such approaches are known. Some use machine learning to construct detectors or filters through supervised learning from training images.”);
instructions for applying the machine learning model to the received images to determine a likelihood that the received images include smoke (Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the recognition algorithm is able to determine the likelihood that smoke is present through confidence estimates, Page 20, Para. 1, lines 6-9, “Algorithm optimization of a fit (such as one type of regression) can produce an estimate of the fit's excellence with the best-fitting pattern, so these pattern matching systems provide classification (best pattern fit) and confidence estimates (Measure both the fit).”); and
responsive to determining that the received images include smoke, instructions for sending an alert indicative of smoke at the stovetop (Page 5, Para. 2 from end, last 8 lines, “Thus, in general, significant leakage of all smoke will be the result of unusual equipment misuse, fire, or failure of the exhaust system…According to a method embodiment, wherein the processor is encoded by executable instructions, the controller provides a warning signal indicating that a preliminary fire detection has occurred, thereby alerting the operator to the need to prevent an alarm or suppressive output with an override input.”).
Livchak does not disclose:
training data with a first set of images including the presence of smoke, a second set of images including the presence of steam, each image in the first of images and the second set of images further labeled with a measure of risk posed to a kitchen, wherein the measure of risk associated with each image is determined based on an amount of the image associated with fire, a type of food being prepared, and a location of the fire relative to cookware within the image, and training based on the training data a machine learning model to differentiate between smoke and steam within the images and to identify a risk posed to the kitchen;
receiving real time images of stovetops and determining whether there is smoke;
specifically controlling a stove where after determining images to be smoke, the stovetop is disabled.
However, Gurajapu discloses, in the similar field of machine learning models for analyzing features of flames (Abstract, “A method and system for advanced flare analytics in a flare operation monitoring and control system… A machine learning-based self-adaptive industrial automation system…”), where training data for machine learning models can further include images that are labelled to have smoke and steam sections (Para. 0015, “The machine learning may begin by training the model with images that have been annotated as portions that are black smoke, steam, and flame. In one embodiment, the annotation may be manual markings as to which portions are black smoke, which are steam, and which are flame. The annotated images are input into the machine learning/ deep learning core and then the system may be fine-tuned further to arrive at the best results. The training data may then be separated from the test data to be analyzed.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model regarding smoke from stovetops in Livchak to include additional features of distinguishing between smoke, steam, and flame, as taught by Gurajapu, which can be applied to images of the stovetops from Livchak. Examiner notes that although Gurajapu focuses on industrial plants, the machine learning model for detecting different particles could easily be applied to any other image, such as steam/smoke from stovetops.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of machine learning to provide more reliable and consistent detection of regions of smoke, steam, and flame down to the pixel level, as stated by Gurajapu, Para. 0014, “The regions of smoke, steam, and flame are not defined shape objects as they are nonuniform and therefore previous image processing techniques were not able to sufficiently distinguish the specific categories. The machine learning/ deep learning core provides a step forward in that the deep learning core is able to distinguish these categories in a significantly more reliable and consistent manner.”,
where being able to detect steam from smoke allows for more accurate alert detection of the system being monitored, as smoking, excessive steam, and other tailored alerts can only be done if the system can be detect what type of particle is being emitted, as stated by Gurajapu, Para. 0022, “Alerts manager 138 which communicates with and may be controlled by DVM 126, may be used to inform operators regarding out of limits situations in any flare. Smoking, excessive steam and the like are examples of alerts.”.
Mori discloses, in the similar field of fire prevention systems (Abstract, “a fire prevention system quantitatively computing a fire danger degree”), where images can be provided with the risk posed to a kitchen (Page 4, Para. 2, “An image processing unit that extracts a feature quantity of the heat source from the image information obtained by the heat source detection unit, an information storage unit that pre-registers information about the heat source and information about the environment around the heat source, and an image processing unit…An analysis unit that calculates the fire risk of the heat source using an image processing unit, and an output unit that outputs a calculation result relating to the feature amount of the heat source obtained by the image processing unit and / or the fire risk obtained by the analysis unit”, where images have a feature quantity extracted and compared to predetermined value, where that comparison outputs the amount of risk posed to the kitchen, Page 6, Para. 3 from end, “In this environment, the cooking site in the kitchen is assumed”), where the risk or danger is determined based on the amount of the image associated with fire and the location/position of the fire relative to its normal position (Page 7, Para. 5 from end, “That is, in the present embodiment, what is extracted as the feature amount is the position of the center of gravity and the size of the cluster assigned 1 by labeling.”, and Page 11, Para. 3 from end, “The information transmission unit 17 receives the position and size of the heat source obtained by the image processing unit 2 and the fire risk A obtained by the signal processing unit from the result output unit 6”, where the feature amount extracted from each image includes the amount/size of the fire and location/position of the fire). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model in modified Livchak to include the size and location determination of each image to output a risk associated with each image as taught by Mori;
where the both steam and smoke images could be used in the risk determination process from Mori as the feature quantity is calculated after a light intensity conversion, Page 6, Para. 5 from end, “The image processing unit 2 in FIG. 1 performs A / D conversion on, for example, two-dimensional analog image data from the image sensor 1C, converts it into, for example, an 8-bit signal according to the light intensity, and performs signal processing in the subsequent stage.”, and Page 6, last Para., “Here, the processing after A / D conversion will be further described as follows. First, the binarization threshold V .sub.th is determined so that the heat source to be monitored can be sensed.”, where binarization occurs after light intensity processing to determine the size of the fire, where for steam images no risk would occur as no pixels would satisfy the light intensity filtering;
where a cooking vessel located with a heat source is imaged from Livchak, Page 9, Para. 3 from end, “spectral information of the time-varying signal and the steady-state signal during operation are relevant independent DOF signals that can help identify normal processes and events with fires in the cooking environment… Can produce Changes and fluctuations in radiation temperature of the cooking surface may indicate flipping or placing of food on the grill or movement of hands or kitchen utensils or cooking vessels on the cooking surface”, where Mori discloses where a heat source location is predetermined and registered and where a risk value is assigned depending on the distance that the fire is from a predetermined location, Page 8, Para. 1, “To calculate the fire risk, it is necessary to give information on the heat source whose location is specified. Therefore, the average center of gravity (X .sub.i , Y .sub.i ) and the size S .sub.i of the heat source are registered in advance. Here, i represents the registration number of the heat source (i is i> 0). Further, regarding these registered heat sources, the risk degrees .sub.Atl , .sub.i with respect to the heat source generation time that is assumed to be normally generated are registered in advance.”, where combining Livchak with Mori would result in the heat source located at the cooktop with a cooking vessel being placed on top from Livchak be set as the predetermined heat source location for Mori, where this heat source position would be relative to a cooking vessel and where the fire position determined from Mori would continue to be relative to a position of a cooking vessel as it is relative to the predetermined heat source position.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of using quantitative measurements in order to determine the risk posed by fires within images, which can improve fire detection accuracy and allow a determination into the severity of the fire occur earlier than a conventional fire alarm, as stated by Mori, Page 4, Para. 1, “potential properties of the heat source is quantitatively used. It is an object of the present invention to provide a fire prevention system capable of preventing a fire in advance by calculating the degree of danger so as to be highly accurate and capable of making a determination at an early stage.”.
Bailey discloses, in the similar field of determining risk of fires from images (Page 3, lines 21-23, “method to detect pre-ignition conditions on a cooking hob comprising: analysing an infrared image of a hob to locate hot plates, burners or pans on the image”), where a risk of fire from an image can be dependent on the type of food being prepared (Page 13, lines 8-11, “As an example, the criteria may reflect the nature of the foodstuff being cooked and indicate an acceptable maximum temperature, an acceptable time at maximum temperature, and an acceptable total time of cooking. Deviation from these criteria may, for example, indicate: preignition conditions, overcooked food, or an unattended hob respectively.”, where depending on the type of food being prepared or the nature of the foodstuff being cooked, there can be different acceptable criteria used for determining the risk of fire or the preignition conditions, Page 28, lines 8-10, “Further heating without attention from the user is likely to result in a fire in the food, so an alert is set for a pre-ignition risk”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the fire risk determination in modified Livchak to further include a fire risk dependent on the type of food being prepared as taught by Bailey; where Mori teaches fire risk determination based on the position of the heat source from a predetermined location and where Bailey improves upon the fire risk determination through setting different fire risk thresholds at the predetermined location based on the food type being cooked.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to take into consideration the different cooking conditions related to different food items, which can alter the thresholds for determining risk of a fire, as stated by Bailey, Page 13, lines 8-11, “As an example, the criteria may reflect the nature of the foodstuff being cooked and indicate an acceptable maximum temperature, an acceptable time at maximum temperature, and an acceptable total time of cooking. Deviation from these criteria may, for example, indicate: preignition conditions, overcooked food, or an unattended hob respectively.”.
Further, Babu discloses, in the similar field of images for detecting smoke from a stovetop (Para. 0170, lines 17-19 from end, “…camera in conjunction with a controller may utilize one or more lenses to detect smoke or flame emitting from a stove top.”), where once smoke is determined to occur from real time images, the stovetop can be turned off (Para. 0209, lines 4-end, “In block 2820, the apparatus receives one or more parameters of one or more monitoring signals from one or more sensors…If the detected parameter exceeds the predetermined threshold, the apparatus will send a control signal to the motor, which in turn drives the control knob and/or operational shaft to turn off the burner…”, where such parameters include smoke, Para. 0212, last 4 lines, “…smoke sensor 1304a may send a monitoring signal which indicates that a certain level of particulate smoke (a parameter) has been detected by the sensor.”, where the camera is also able to detect smoke as stated above). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the controller and smoke threshold from images in modified Livchak to include disabling the stovetop when smoke exceeds a threshold as taught by Babu.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to automatically turn off the power to the stovetop when there is no one present, as stated by Babu, Para. 0004, lines 1-4, “For the foregoing reasons, there is a need for a safety device for automatically rotating an operational shaft of a burner to an Off position upon the occurrence of a safety event for shutting off power to the burner.”.
Regarding claim 14, modified Livchak teaches the apparatus according to claim 11, as set forth above, discloses wherein the received images are determined to include smoke in response to the machine learning model determining that there is an likelihood that the received images include smoke (Livchak, Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the recognition algorithm is able to determine the likelihood that smoke is present through confidence estimates, Page 20, Para. 1, lines 6-9, “Algorithm optimization of a fit (such as one type of regression) can produce an estimate of the fit's excellence with the best-fitting pattern, so these pattern matching systems provide classification (best pattern fit) and confidence estimates (Measure both the fit).”).
Modified Livchak does not disclose:
where the likelihood from the machine learning model includes an above-threshold value.
However, Babu discloses the ability to compare a monitored signal to a predetermined threshold, where parameters can include a range of sensor data (Para. 0209, lines 6-14, “Such parameters may include, but not limited to, motion, temperature, humidity, CO level, CO2 level, natural gas level, propane level, butane level, and/or pollution levels. Upon receiving a parameter of a monitoring signal, the apparatus compares the parameter with the predetermined threshold stored in the database 2830 which may reside on the memory of the controller of the safety device or may be stored externally from the safety device and accessed.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model in Livchak to further include applying threshold comparisons for smoke sensor data, as taught by Babu.
Regarding the use of a machine learning model to be applied for determining if there is smoke within an image, it is the Examiner's position that one of ordinary skill in the art would have found it obvious to try as Livchak implies the use of thresholds through confidence estimates in using the recognition algorithm. Thus, smoke from images is determined by the recognition algorithm by comparing the image with training data, then an estimate to the likelihood that smoke is within the produced through the confidence estimate. Babu discloses a similar feature, but through the use of sensor data values that are compared with specific thresholds, where the sensor data values must be higher than the thresholds. Since both Livchak and Babu compare sensor data with stored data to reach conclusions, the end results are similar and it would be obvious to try implementing a specific confidence threshold (like the threshold in Babu that needs to be exceeded) that Livchak’s recognition algorithm would need to exceed in order to determine if smoke is present in the image.
Regarding claim 15, modified Livchak teaches the apparatus according to claim 11, as set forth above, discloses wherein the instructions for determining that the received images include smoke comprises determining that the received images include an above-threshold amount of smoke (Livchak, Page 8, Para. 2 from end, lines 3-8 from end, “Various devices such as photocells, charge coupled devices, optical multipliers, and camera sensors can be used as the optical sensors. The controller can be set to detect a duration of high smoke levels and trigger on or based on both duration and opacity or a combination thereof. As discussed, this signal may be combined with other signals to identify the fire, specify the location of the fire, and in combination with other factors to determine the size of the fire.”, where cameras can take images of smoke and high smoke levels are a threshold that the controller determines to mean that smoke is present).
Modified Livchak does not disclose:
wherein the threshold amount of smoke may be adjusted via a user interface presented to a client device of a user of the stovetop.
However, Babu discloses where the user is able to control the threshold amount of smoke for triggering an alert by being able to turn off the smoke sensor via a user device (Para. 0206, last 7 lines, “The user may also be able to override or temporarily turn one or more of the sensors in the sensor/relay module 1304 off. For example, if a user is cooking and producing large amounts of smoke, but knows there is no fire, the user may utilize user device 1306 to temporality turn off a smoke sensor in the sensor/relay device.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the smoke detection algorithm in modified Livchak to include the ability to adjust the threshold values as taught by Babu.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to cook foods that have a high smoke output and not cause the alarm system to be triggered, as stated by Babu, Para. 0206, last 7 lines, “The user may also be able to override or temporarily turn one or more of the sensors in the sensor/relay module 1304 off. For example, if a user is cooking and producing large amounts of smoke, but knows there is no fire, the user may utilize user device 1306 to temporality turn off a smoke sensor in the sensor/relay device.”.
Regarding claim 16, modified Livchak teaches the apparatus according to claim 11, as set forth above, discloses wherein the alert is sent via a user interface presented to a client device of a user of the stovetop (Livchak, Page 6, Para. 2, last 3 lines, “The controller can trigger an alarm in response to a danger signal. As described elsewhere, the controller can accept an override command from the user interface and disable the alarm in response to the override command.”, where a client device that connects to the internet can also issue the alarm, Page 6, Para. 3, lines 9-12, “Another way in which an override command can be applied to the controller 600 is via the smartphone 668 interface or other portable interface. In one embodiment, the controller (or service connected by network to the user interface) is programmed to output information about the preliminary fire detection…”).
Regarding claim 19, modified Livchak teaches the apparatus according to claim 11, as set forth above.
Modified Livchak does not disclose:
wherein instructions for disabling operation of the stovetop comprises actuating a mechanical stovetop controller configured to turn off the stovetop.
However, Babu discloses where turning off the stovetop includes actuating a mechanical controller (Para. 0040, lines 5-end, “…the knob adapter member is structured to mimic an attachment part of the operational shaft, such that the top knob can be attached to the top of the safety device module. In some examples, the top knob comprises an original knob for operational control of the burner. In some examples, the controller is configured to cause the motor to turn the operational shaft of the burner to an Off position.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the turn off mechanism in modified Livchak to include the safety device module that attaches to the stovetop burner controller as taught by Babu.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to automatically turn off the power to the stovetop when there is no one present, as stated by Babu, Para. 0004, lines 1-4, “For the foregoing reasons, there is a need for a safety device for automatically rotating an operational shaft of a burner to an Off position upon the occurrence of a safety event for shutting off power to the burner.”.
Regarding claim 20, Livchak discloses a computer system (Page 19, Para. 4, “It will be appreciated that the modules, processes, systems, and sections described above may be implemented in hardware, hardware programmed in software, software instructions stored on a nontransitory computer readable medium, or a combination thereof.”) comprising:
a computer processor (Page 12, Para. 3 from end, lines 2-3 from end, “The classifier and upstream and downstream processing may be performed by controller 600…”); and
a non-transitory computer-readable storage medium storage instructions (Page 12, last Para., lines 2-4, “…algorithms and models are provided in the data store and And / or stored in memory to provide access to them by the classifier.”) that when executed by the computer processor (Page 12, Para. 3 from end, lines 2-3 from end, “The classifier and upstream and downstream processing may be performed by controller 600…”) perform actions comprising:
accessing a set of training data, the training data comprising image data of stovetop systems labeled based on presence of smoke (Page 5, Para. 3, lines 2-5, “The camera 599 may be selected based on a wide range of optical and near infrared frequencies. The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood. Smoke, and of course fire, should not be visible from above the hood 621 in normal circumstances.”, where there is a camera takes images of areas around a stovetop with the presence of smoke, where training is done, Page 19, Para. 3 from end, lines 1-3, “Devices and methods for receiving sensor signals and outputting information identifying various possible situations, such as a fire situation or a smoke load situation, indicated by the values of the sensor signals, can generally be identified with classifiers…”, and Page 20, Para. 1, lines 3-4, “…use machine learning to construct detectors or filters through supervised learning from training images.”);
training, using the set of training data, a machine learning model configured to detect smoke in images of a stovetop system (Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the training data is used during the recognition algorithm, Para. 20, last Para., lines 1-5, “Recognizing objects such as people in an image using a pattern recognition approach is a known technique called computer vision and includes face recognition. Known technology may use 3D scanners (infrared, autonomous vehicles, and product inspection systems such as Microsoft Kinect). Examples include face recognition and pedestrian detection. Many such approaches are known. Some use machine learning to construct detectors or filters through supervised learning from training images.”);
applying the machine learning model to the received images to determine a likelihood that the received images include smoke (Page 5, Para. 2 from end, lines 3-4, “The recognition algorithm implemented by the controller 108 may recognize that a fire or smoke escapes from the hood.”, where the recognition algorithm is able to determine the likelihood that smoke is present through confidence estimates, Page 20, Para. 1, lines 6-9, “Algorithm optimization of a fit (such as one type of regression) can produce an estimate of the fit's excellence with the best-fitting pattern, so these pattern matching systems provide classification (best pattern fit) and confidence estimates (Measure both the fit).”); and
responsive to determining that the received images include smoke, sending an alert indicative of smoke at the stovetop (Page 5, Para. 2 from end, last 8 lines, “Thus, in general, significant leakage of all smoke will be the result of unusual equipment misuse, fire, or failure of the exhaust system…According to a method embodiment, wherein the processor is encoded by executable instructions, the controller provides a warning signal indicating that a preliminary fire detection has occurred, thereby alerting the operator to the need to prevent an alarm or suppressive output with an override input.”).
Livchak does not disclose:
training data with a first set of images including the presence of smoke, a second set of images including the presence of steam, each image in the first set of images and the second set of images further labeled with a measure of risk posed to a kitchen, wherein the measure of risk associated with each image is determined based on an amount of the image associated with fire, a type of food being prepared, and a location of the fire relative to cookware within the image, and training based on the training data a machine learning model to differentiate between smoke and steam within the images and identify a risk posed to the kitchen;
receiving real time images of stovetops and determining whether there is smoke;
specifically controlling a stove where after determining images to be smoke, the stovetop is disabled.
However, Gurajapu discloses, in the similar field of machine learning models for analyzing features of flames (Abstract, “A method and system for advanced flare analytics in a flare operation monitoring and control system… A machine learning-based self-adaptive industrial automation system…”), where training data for machine learning models can further include images that are labelled to have smoke and steam sections (Para. 0015, “The machine learning may begin by training the model with images that have been annotated as portions that are black smoke, steam, and flame. In one embodiment, the annotation may be manual markings as to which portions are black smoke, which are steam, and which are flame. The annotated images are input into the machine learning/ deep learning core and then the system may be fine-tuned further to arrive at the best results. The training data may then be separated from the test data to be analyzed.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model regarding smoke from stovetops in Livchak to include additional features of distinguishing between smoke, steam, and flame, as taught by Gurajapu, which can be applied to images of the stovetops from Livchak. Examiner notes that although Gurajapu focuses on industrial plants, the machine learning model for detecting different particles could easily be applied to any other image, such as steam/smoke from stovetops.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of machine learning to provide more reliable and consistent detection of regions of smoke, steam, and flame down to the pixel level, as stated by Gurajapu, Para. 0014, “The regions of smoke, steam, and flame are not defined shape objects as they are nonuniform and therefore previous image processing techniques were not able to sufficiently distinguish the specific categories. The machine learning/ deep learning core provides a step forward in that the deep learning core is able to distinguish these categories in a significantly more reliable and consistent manner.”,
where being able to detect steam from smoke allows for more accurate alert detection of the system being monitored, as smoking, excessive steam, and other tailored alerts can only be done if the system can be detect what type of particle is being emitted, as stated by Gurajapu, Para. 0022, “Alerts manager 138 which communicates with and may be controlled by DVM 126, may be used to inform operators regarding out of limits situations in any flare. Smoking, excessive steam and the like are examples of alerts.”.
Mori discloses, in the similar field of fire prevention systems (Abstract, “a fire prevention system quantitatively computing a fire danger degree”), where images can be provided with the risk posed to a kitchen (Page 4, Para. 2, “An image processing unit that extracts a feature quantity of the heat source from the image information obtained by the heat source detection unit, an information storage unit that pre-registers information about the heat source and information about the environment around the heat source, and an image processing unit…An analysis unit that calculates the fire risk of the heat source using an image processing unit, and an output unit that outputs a calculation result relating to the feature amount of the heat source obtained by the image processing unit and / or the fire risk obtained by the analysis unit”, where images have a feature quantity extracted and compared to predetermined value, where that comparison outputs the amount of risk posed to the kitchen, Page 6, Para. 3 from end, “In this environment, the cooking site in the kitchen is assumed”), where the risk or danger is determined based on the amount of the image associated with fire and the location/position of the fire relative to its normal position (Page 7, Para. 5 from end, “That is, in the present embodiment, what is extracted as the feature amount is the position of the center of gravity and the size of the cluster assigned 1 by labeling.”, and Page 11, Para. 3 from end, “The information transmission unit 17 receives the position and size of the heat source obtained by the image processing unit 2 and the fire risk A obtained by the signal processing unit from the result output unit 6”, where the feature amount extracted from each image includes the amount/size of the fire and location/position of the fire). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the machine learning model in modified Livchak to include the size and location determination of each image to output a risk associated with each image as taught by Mori;
where the both steam and smoke images could be used in the risk determination process from Mori as the feature quantity is calculated after a light intensity conversion, Page 6, Para. 5 from end, “The image processing unit 2 in FIG. 1 performs A / D conversion on, for example, two-dimensional analog image data from the image sensor 1C, converts it into, for example, an 8-bit signal according to the light intensity, and performs signal processing in the subsequent stage.”, and Page 6, last Para., “Here, the processing after A / D conversion will be further described as follows. First, the binarization threshold V .sub.th is determined so that the heat source to be monitored can be sensed.”, where binarization occurs after light intensity processing to determine the size of the fire, where for steam images no risk would occur as no pixels would satisfy the light intensity filtering;
where a cooking vessel located with a heat source is imaged from Livchak, Page 9, Para. 3 from end, “spectral information of the time-varying signal and the steady-state signal during operation are relevant independent DOF signals that can help identify normal processes and events with fires in the cooking environment… Can produce Changes and fluctuations in radiation temperature of the cooking surface may indicate flipping or placing of food on the grill or movement of hands or kitchen utensils or cooking vessels on the cooking surface”, where Mori discloses where a heat source location is predetermined and registered and where a risk value is assigned depending on the distance that the fire is from a predetermined location, Page 8, Para. 1, “To calculate the fire risk, it is necessary to give information on the heat source whose location is specified. Therefore, the average center of gravity (X .sub.i , Y .sub.i ) and the size S .sub.i of the heat source are registered in advance. Here, i represents the registration number of the heat source (i is i> 0). Further, regarding these registered heat sources, the risk degrees .sub.Atl , .sub.i with respect to the heat source generation time that is assumed to be normally generated are registered in advance.”, where combining Livchak with Mori would result in the heat source located at the cooktop with a cooking vessel being placed on top from Livchak be set as the predetermined heat source location for Mori, where this heat source position would be relative to a cooking vessel and where the fire position determined from Mori would continue to be relative to a position of a cooking vessel as it is relative to the predetermined heat source position.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of using quantitative measurements in order to determine the risk posed by fires within images, which can improve fire detection accuracy and allow a determination into the severity of the fire occur earlier than a conventional fire alarm, as stated by Mori, Page 4, Para. 1, “potential properties of the heat source is quantitatively used. It is an object of the present invention to provide a fire prevention system capable of preventing a fire in advance by calculating the degree of danger so as to be highly accurate and capable of making a determination at an early stage.”.
Bailey discloses, in the similar field of determining risk of fires from images (Page 3, lines 21-23, “method to detect pre-ignition conditions on a cooking hob comprising: analysing an infrared image of a hob to locate hot plates, burners or pans on the image”), where a risk of fire from an image can be dependent on the type of food being prepared (Page 13, lines 8-11, “As an example, the criteria may reflect the nature of the foodstuff being cooked and indicate an acceptable maximum temperature, an acceptable time at maximum temperature, and an acceptable total time of cooking. Deviation from these criteria may, for example, indicate: preignition conditions, overcooked food, or an unattended hob respectively.”, where depending on the type of food being prepared or the nature of the foodstuff being cooked, there can be different acceptable criteria used for determining the risk of fire or the preignition conditions, Page 28, lines 8-10, “Further heating without attention from the user is likely to result in a fire in the food, so an alert is set for a pre-ignition risk”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the fire risk determination in modified Livchak to further include a fire risk dependent on the type of food being prepared as taught by Bailey; where Mori teaches fire risk determination based on the position of the heat source from a predetermined location and where Bailey improves upon the fire risk determination through setting different fire risk thresholds at the predetermined location based on the food type being cooked.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to take into consideration the different cooking conditions related to different food items, which can alter the thresholds for determining risk of a fire, as stated by Bailey, Page 13, lines 8-11, “As an example, the criteria may reflect the nature of the foodstuff being cooked and indicate an acceptable maximum temperature, an acceptable time at maximum temperature, and an acceptable total time of cooking. Deviation from these criteria may, for example, indicate: preignition conditions, overcooked food, or an unattended hob respectively.”.
Further, Babu discloses, in the similar field of images for detecting smoke from a stovetop (Para. 0170, lines 17-19 from end, “…camera in conjunction with a controller may utilize one or more lenses to detect smoke or flame emitting from a stove top.”), where once smoke is determined to occur from real time images, the stovetop can be turned off (Para. 0209, lines 4-end, “In block 2820, the apparatus receives one or more parameters of one or more monitoring signals from one or more sensors…If the detected parameter exceeds the predetermined threshold, the apparatus will send a control signal to the motor, which in turn drives the control knob and/or operational shaft to turn off the burner…”, where such parameters include smoke, Para. 0212, last 4 lines, “…smoke sensor 1304a may send a monitoring signal which indicates that a certain level of particulate smoke (a parameter) has been detected by the sensor.”, where the camera is also able to detect smoke as stated above). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the controller and smoke threshold from images in modified Livchak to include disabling the stovetop when smoke exceeds a threshold as taught by Babu.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of being able to automatically turn off the power to the stovetop when there is no one present, as stated by Babu, Para. 0004, lines 1-4, “For the foregoing reasons, there is a need for a safety device for automatically rotating an operational shaft of a burner to an Off position upon the occurrence of a safety event for shutting off power to the burner.”.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Livchak et al. (KR 20190139229 A, hereinafter Livchak) in view of Gurajapu et al. (EP 3748444 A1, hereinafter Gurajapu) and Mori et al. (JP 2006072863 A, hereinafter Mori) and Babu et al. (US 20180003392 A1, hereinafter Babu) and Bailey (WO 2020144445 A1) in further view of Bolz-Mendel (DE 102019113457 A1).
Regarding claim 2, modified Livchak teaches the method according to claim 1, as set forth above, discloses wherein the training data further comprises one or more of heat signature data (Livchak, Page 5, Para. 2 from end, lines 12-15, “The density of the smoke can be measured by subtracting the background pattern and level of the infrared illumination, if the detected smoke is obscure, for example carbon dioxide. Or if incandescent gas heat (fire) is present, it can also be detected at visible and infrared wavelengths.”, where heat signature or infrared illumination is detected by the camera sensor as well, Page 5, Para. 2 from end, lines 2-3, “The camera 599 may be selected based on a wide range of optical and near infrared frequencies.”).
Modified Livchak does not disclose:
the training data includes one or more of LiDAR data.
However, Bolz-Mendel discloses, in the similar field of detecting smoke by an optical sensor (Page 2, Para .3, lines 2-3, “Detecting the presence of smoke by an optical measuring unit…”), where LiDAR data can be taken for smoke detection (Page 3, last Para., “In an embodiment of the method according to the invention it is provided that the optical measuring unit measures backscattered light originating from laser pulses, in particular carries out a LIDAR measurement. In the case of a LIDAR measuring station, smoke detection can only be traced back to a simple transit time measurement and can therefore be carried out particularly easily and with fewer errors.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the sensor data for training in modified Livchak to include the LiDAR sensors and data as taught by Bolz-Mendel.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of reducing errors in smoke detection, as stated by Bolz-Mendel, Page 3, last Para., “In the case of a LIDAR measuring station, smoke detection can only be traced back to a simple transit time measurement and can therefore be carried out particularly easily and with fewer errors.”.
Regarding claim 12, modified Livchak teaches the apparatus according to claim 11, as set forth above, discloses wherein the training data further comprises one or more of heat signature data (Livchak, Page 5, Para. 2 from end, lines 12-15, “The density of the smoke can be measured by subtracting the background pattern and level of the infrared illumination, if the detected smoke is obscure, for example carbon dioxide. Or if incandescent gas heat (fire) is present, it can also be detected at visible and infrared wavelengths.”, where heat signature or infrared illumination is detected by the camera sensor as well, Page 5, Para. 2 from end, lines 2-3, “The camera 599 may be selected based on a wide range of optical and near infrared frequencies.”).
Modified Livchak does not disclose:
the training data includes one or more of LiDAR data.
However, Bolz-Mendel discloses, in the similar field of detecting smoke by an optical sensor (Page 2, Para .3, lines 2-3, “Detecting the presence of smoke by an optical measuring unit…”), where LiDAR data can be taken for smoke detection (Page 3, last Para., “In an embodiment of the method according to the invention it is provided that the optical measuring unit measures backscattered light originating from laser pulses, in particular carries out a LIDAR measurement. In the case of a LIDAR measuring station, smoke detection can only be traced back to a simple transit time measurement and can therefore be carried out particularly easily and with fewer errors.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the sensor data for training in modified Livchak to include the LiDAR sensors and data as taught by Bolz-Mendel.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of reducing errors in smoke detection, as stated by Bolz-Mendel, Page 3, last Para., “In the case of a LIDAR measuring station, smoke detection can only be traced back to a simple transit time measurement and can therefore be carried out particularly easily and with fewer errors.”.
Claims 7-8 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Livchak et al. (KR 20190139229 A, hereinafter Livchak) in view of Gurajapu et al. (EP 3748444 A1, hereinafter Gurajapu) and Mori et al. (JP 2006072863 A, hereinafter Mori) and Babu et al. (US 20180003392 A1, hereinafter Babu) and Bailey (WO 2020144445 A1) in further view of Oher et al. (JP 2020533726 A, hereinafter Oher).
Regarding claim 7, modified Livchak teaches the method according to claim 1, as set forth above.
Modified Livchak does not disclose:
wherein sending the alert comprises sending the alert to a local emergency department.
However, Oher discloses, in the similar field of smoke detection through cameras (Page 5, Para. 2, lines 1-2, “…the smoke detector 200 may further include a microprocessor 406, a smoke detector memory 407, an audio speaker 408, and a camera 409.”), where a positive smoke detection results in an alert that is sent to the local emergency department (Page 6, Para. 2 from end, lines 1-4, “If the smoke detector 200 does confirm that there is a fire in the vicinity, the smoke detector 200 may send a notification to the home networking server 101. In response, the home monitoring server 101 may notify and transmit information to the emergency response server 102 and notify a specific department to respond to a fire.”.). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the alarm alert in modified Livchak to further include a message to the local emergency department as taught by Oher.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of automatically having emergency departments notified that a fire is occurring as fires can cause destruction of communication devices, as stated by Oher, Page 2, Para. 3 from end, “…information in the network is passed to the Internet via routers (and modems). If a fire destroys a router and / or modem, if it is far away, a high-performance smoke alarm will be isolated and unable to obtain information that may be of vital importance.”, where the invention provides an improved method, Page 2, last Para., lines 1-3, “Therefore, it would be useful to have an improved system and method for transmitting smoke detector data from the smoke detector by the smoke detector. In addition, it would be advantageous to have an improved system for delivering smoke detector data using emergency personnel routers.”.
Regarding claim 8, modified Livchak teaches the method according to claim 1, as set forth above.
Modified Livchak does not disclose:
wherein the alert comprises one or more of the received images of the stovetop.
However, Oher discloses where the alert for smoke detection includes sending images of the area where smoke has occurred (Page 12, Para. 2 from end, lines 3-6, “In one embodiment, when smoke is detected, the smoke detector 200 may use the camera 409 to continuously capture an image and / or video of the area where the smoke detector is installed. In parallel, the captured image or video may be transmitted from the smoke detector 200 to the home monitoring server 101 and / or the emergency response server 102.”, where the emergency response server alerts the local emergency departments, Page 6, Para. 2 from end, lines 1-4, “If the smoke detector 200 does confirm that there is a fire in the vicinity, the smoke detector 200 may send a notification to the home networking server 101. In response, the home monitoring server 101 may notify and transmit information to the emergency response server 102 and notify a specific department to respond to a fire.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the images of smoke from the camera and the alert from modified Livchak to include sending images as part of the alert as taught by Oher.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of along for data from the emergency to be captured and stored in real time for emergency departments to review, as stated by Oher, Page 12, Para. 2 from end, last 4 lines, “In parallel, the captured image or video may be transmitted from the smoke detector 200 to the home monitoring server 101 and / or the emergency response server 102. As a result, the server may store the data in real time, and even if the smoke detector 200 is burnt by a fire, the data may be reliably acquired.”.
Regarding claim 17, modified Livchak teaches the apparatus according to claim 11, as set forth above.
Modified Livchak does not disclose:
wherein sending the alert comprises sending the alert to a local emergency department.
However, Oher discloses, in the similar field of smoke detection through cameras (Page 5, Para. 2, lines 1-2, “…the smoke detector 200 may further include a microprocessor 406, a smoke detector memory 407, an audio speaker 408, and a camera 409.”), where a positive smoke detection results in an alert that is sent to the local emergency department (Page 6, Para. 2 from end, lines 1-4, “If the smoke detector 200 does confirm that there is a fire in the vicinity, the smoke detector 200 may send a notification to the home networking server 101. In response, the home monitoring server 101 may notify and transmit information to the emergency response server 102 and notify a specific department to respond to a fire.”.). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the alarm alert in modified Livchak to further include a message to the local emergency department as taught by Oher.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of automatically having emergency departments notified that a fire is occurring as fires can cause destruction of communication devices, as stated by Oher, Page 2, Para. 3 from end, “…information in the network is passed to the Internet via routers (and modems). If a fire destroys a router and / or modem, if it is far away, a high-performance smoke alarm will be isolated and unable to obtain information that may be of vital importance.”, where the invention provides an improved method, Page 2, last Para., lines 1-3, “Therefore, it would be useful to have an improved system and method for transmitting smoke detector data from the smoke detector by the smoke detector. In addition, it would be advantageous to have an improved system for delivering smoke detector data using emergency personnel routers.”.
Regarding claim 18, modified Livchak teaches the apparatus according to claim 11, as set forth above.
Modified Livchak does not disclose:
wherein the alert comprises one or more of the received images of the stovetop.
However, Oher discloses where the alert for smoke detection includes sending images of the area where smoke has occurred (Page 12, Para. 2 from end, lines 3-6, “In one embodiment, when smoke is detected, the smoke detector 200 may use the camera 409 to continuously capture an image and / or video of the area where the smoke detector is installed. In parallel, the captured image or video may be transmitted from the smoke detector 200 to the home monitoring server 101 and / or the emergency response server 102.”, where the emergency response server alerts the local emergency departments, Page 6, Para. 2 from end, lines 1-4, “If the smoke detector 200 does confirm that there is a fire in the vicinity, the smoke detector 200 may send a notification to the home networking server 101. In response, the home monitoring server 101 may notify and transmit information to the emergency response server 102 and notify a specific department to respond to a fire.”). It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the images of smoke from the camera and the alert from modified Livchak to include sending images as part of the alert as taught by Oher.
One of ordinary skill in the art would have been motivated to make this modification in order to gain the advantage of along for data from the emergency to be captured and stored in real time for emergency departments to review, as stated by Oher, Page 12, Para. 2 from end, last 4 lines, “In parallel, the captured image or video may be transmitted from the smoke detector 200 to the home monitoring server 101 and / or the emergency response server 102. As a result, the server may store the data in real time, and even if the smoke detector 200 is burnt by a fire, the data may be reliably acquired.”.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN GUANHUA WEN whose telephone number is (571)272-9940 and whose email is kevin.wen@uspto.gov. The examiner can normally be reached Monday-Friday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ibrahime Abraham can be reached on 571-270-5569. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN GUANHUA WEN/Examiner, Art Unit 3761
01/30/2026
/IBRAHIME A ABRAHAM/Supervisory Patent Examiner, Art Unit 3761