Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 27-29 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 27 recites “determining, based on a comparison between the data associated with suspicious behavior patterns and the first data received from the first sensor…”, which lacks antecedent basis. It is unclear what the claim limitation “the data associated with suspicious behavior patterns” is meant to refer to as the claim previously introduces “data associated with safe behavior patterns” and there is no previous recitation of data representing suspicious behavior patterns. It is believed by the Examiner that Applicant intended for this limitation to correspond to data associated with suspicious behavior patterns, therefore, for examination purposes, the claim limitation will be interpreted as “determining, based on a comparison between the data associated with safe behavior patterns and the first data received from the first sensor…”.
Claims 28 and 29 are rejected as being dependent on a rejected base claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4, 7, 9-15, 17-20, and 25-29 are rejected under 35 U.S.C. 101.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of analyzing data of an environment to recognize objects and performing an action corresponding to that recognition.
The claim recites: “A computer-implemented method for securing an environment, at least a portion of the method being performed by a computing device comprising one or more processors, the method comprising: receiving first data from a first sensor, the first data including data associated with a potential object within an evaluation field of the first sensor; receiving second data from a second sensor, the second data including data associated with a potential object within an evaluation field of the second sensor, wherein the first and second sensors are different types of sensors; determining, based on the first data, a first identity classification of the potential object within the evaluation field of the first sensor; determining, based on the second data, a second identity classification of the potential object within the evaluation field of the second sensor; determining, based on at least one of the first and second identity classifications, a final identity of the potential object within at least one of the evaluation field of the first sensor and the evaluation field of the second sensor; and performing a security action based on the final identity of the potential object.”
The limitations, as drafted, are processes that, under their broadest reasonable interpretation,
cover performance of the limitation in the human mind. A person can receive data from various sensors to classify objects. For example, the person can receive visual data through their eyes and auditory data through their ears, form initial classifications from each sense, determine a final identity by comparing those classification, and decide on a mental action, such as to continue monitoring the object.
The judicial exception is not integrated into a practical application. For example, the claim recites the additional element, “A computer-implemented method for securing an environment, at least a portion of the method being performed by a computing device comprising one or more processors… ”. These additional elements are recited at a high level of generality such that they amount to a generic computer-implemented method using a processor. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial expectation. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements are recited at a high-level of generality. It is therefore a judicial exception that is not integrated into a practical application, and does not include additional elements that are sufficient to amount to significantly more than the judicial exception. This claim is not patent eligible.
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can receive visual data through their eyes and auditory data through their ears. This claim is not patent eligible.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can receive visual data through their eyes, and they can mentally infer depth data from that visual input. This claim is not patent eligible.
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can receive visual data through their eyes and auditory data through their ears. This claim is not patent eligible.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can observe sensory data to classify objects as a human, an animal, a nonliving object, or an absence of an object.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can compare and match classifications of different sensory data.
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can assign likelihood scores for classifications to make a final identification.
Claim 12 is rejected under 35 U.S.C. 101 because the claim recites additional elements recited at a high
level of generality such that they amount to merely weighting inputs based on accuracy. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. This claim is not patent eligible.
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can set and apply mental thresholds to classification likelihood scores in order to determine final identifications of objects.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can perform object classification corresponding to humans.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can recognize known and unknown people by accessing and comparing previously observed features, such as facial recognition. Further, the inclusion of “accessing a database” amounts to a generic computer-implemented step, and is nothing more than an attempt to generally link the abstract idea to a particular technology environment. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can recognize known people by accessing and comparing previously observed features, such as facial recognition. Further, similar to the analysis of claim 15 above, the inclusion of “accessing a database” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 18 is rejected under 35 U.S.C. 101 because the claim recites additional elements corresponding to the performance of a particular security actions, and is nothing more than an attempt to generally link the abstract idea to a particular technology environment. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can determine a human is unknown by accessing and comparing previously observed features, such as facial recognition. The person can further determine the human is engaging in suspicious behavior by accessing and comparing previously observed behavior patterns. Further, similar to the analysis of claim 15 above, the inclusion of “accessing a database” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can access and compare previously observed behavior patterns corresponding to any of the items listed in claim 20.
Claim 25 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can mentally set time period criteria for unknown humans to determine if security action is required.
Claim 26 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can set time period criteria for unknown humans based on the observed suspicious behavior.
Claim 27 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can determine a human is unknown by accessing and comparing previously observed features, such as facial recognition. The person can further determine the human is engaging in safe behavior by accessing and comparing previously observed behavior patterns. Further, similar to the analysis of claim 15 above, the inclusion of “accessing a database” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 28 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can access and compare previously observed behavior patterns corresponding to any of the items listed in claim 28.
Claim 29 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a further limitation of the same abstract idea identified in the analysis of claim 1. For example, the person can mentally set time period criteria for unknown humans to determine if security action is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 7, 9-11, and 13-14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ko et al. (US 20140055610 A1), (hereinafter Ko).
Regarding claim 1, Ko teaches a computer-implemented method for securing an environment, at least a portion of the method being performed by a computing device comprising one or more processors (Ko, “In a low-illumination environment, object detection is difficult, leading to a low detection rate. In an environment where illumination severely changes, like door opening or closing, an object false-detection rate increases. When a wall surface or the like is a reflective surface, the rate of false detection is high. In indoor Surveillance technology based on audio signals, detection of abnormal activity generating no audio information is difficult for some reasons such as covering the mouth and a limitation in microphone performance. Moreover, audio information is not enough to accurately ascertain indoor conditions, such as the number of people and door opening or closing. Accordingly, the indoor Surveillance system 1 may improve accuracy of context awareness by improving a monitoring function using both audio information and video information.”, pg. 3, paragraph 0056, see indoor surveillance system 1 of Fig. 1), the method comprising:
receiving first data from a first sensor, the first data including data associated with a potential object within an evaluation field of the first sensor; receiving second data from a second sensor, the second data including data associated with a potential object within an evaluation field of the second sensor, wherein the first and second sensors are different types of sensors (Ko, “The indoor surveillance system 1 may monitor enclosed and desolate spaces such as the inside of elevators, stairs, underground parking lots, senior citizens centers, play grounds, and trails around apartments or buildings. Referring to FIG. 1, the indoor surveillance system 1 includes a sensor 10, an input device 40, an audio processing device 50, a video processing device 60, a context awareness device 70, and an output device 80. The sensor 10 includes an audio sensor 20 and a video sensor 30. The audio sensor 20 collects audio signals generated in a monitoring region, in operation S21. In operation S31, the video sensor 30 captures an image of the monitoring region by using a digital and/or analog camera. The audio sensor 20 and the video sensor 30 may be installed separately from each other, or may be integrally formed with each other.”, pg. 3, paragraph 0057-0058, An audio and video sensor are used to collect data from a monitored environment. This data includes information for objects in the environment.);
determining, based on the first data, a first identity classification of the potential object within the evaluation field of the first sensor; determining, based on the second data, a second identity classification of the potential object within the evaluation field of the second sensor (Ko, “In operation S51, the audio processing device 50 determines whether an abnormal event has occurred in the monitoring region, by performing extraction of features of the audio signal and audio class recognition. The audio processing device 50 generates an audio frame of a predetermined time unit from the audio signal and extracts an audio feature from the audio frame. Next, the audio processing device 50 classifies the audio frame… In operation S61, the video processing device 60 determines whether an abnormal event has occurred in the monitoring region, by performing foreground detection and motion information extraction. The video processing device 60 detects a foreground in units of frames by using a back ground Subtraction algorithm and an optical flow technique, and identifies a video frame as a normal or abnormal event via foreground analysis.”, pg. 4, paragraphs 0060-0061, “Referring back to FIG. 5, in operation S515, the audio classifier 505 classifies an audio frame according to a hierarchical approach method. Audio frames may be roughly classified into two categories. Audio frames, such as Screaming, crying, normal conversations, and information commentaries, may be classified into a Vocal category, because they are sounds coming out of the neck of a person. Collisions, noise made by opening or closing a door, stepping Sounds, Sounds coming out of empty elevators, and alarm bell sounds may be classified into a non-vocal category.”, pg. 5, paragraph 0072, lines 1-10, “The object classification unit 621 determines whether the foreground BLOB determined to be a human being is a group of people or an individual person, by comparing a statistical feature extracted from the foreground BLOB with statistical information about an individual object and a group object via the AdaBoost training method or the like.”, pg. 9, paragraph 0103, The data collected from each sensor is analyzed to classify potential objects and events. The audio device classifies events as normal or abnormal based on determining vocal categories of the data, while the video device classifies objects as human/non-human and determines event abnormalities based on motion information in the data.);
determining, based on at least one of the first and second identity classifications, a final identity of the potential object within at least one of the evaluation field of the first sensor and the evaluation field of the second sensor (Ko, “Based on results of the abnormal event occurrence/nonoccurrence determinations that are periodically received from the audio processing device 50 and the video processing device 60, the context awareness device 70 calculates a combined probability of the audio abnormal probability P and the video abnormal probability P with regard to each of the normal and abnormal situations from the respective combined probability distribution models for the normal and abnormal situations. When a ratio between the combined probabilities for the normal and abnormal situations is greater than a threshold value T, the context awareness device 70 finally determines that an abnormal situation has occurred.”, pg. 4, paragraph, 0063, lines 1-21, These classifications from each sensor are then combined by calculating respective probabilities and using combined probability distribution models to determine a final identity corresponding to an abnormal event of an object in the environment.); and
performing a security action based on the final identity of the potential object (Ko, “In operation S81, the output device 80 may include a display and a speaker and may generate an alarm if it is determined that an abnormal situation has occurred.”, pg. 4, paragraph 0064, lines 1-3, Based on this final event identification, a security action, such as generating an alarm, is executed.).
Regarding claim 2, Ko teaches the method of claim 1, wherein the first sensor is an image sensor and the second sensor is a depth sensor or an audio sensor (Ko, “In operation S51, the audio processing device 50 determines whether an abnormal event has occurred in the monitoring region, by performing extraction of features of the audio signal and audio class recognition. The audio processing device 50 generates an audio frame of a predetermined time unit from the audio signal and extracts an audio feature from the audio frame. Next, the audio processing device 50 classifies the audio frame… In operation S61, the video processing device 60 determines whether an abnormal event has occurred in the monitoring region, by performing foreground detection and motion information extraction. The video processing device 60 detects a foreground in units of frames by using a back ground Subtraction algorithm and an optical flow technique, and identifies a video frame as a normal or abnormal event via foreground analysis.”, pg. 4, paragraphs 0060-0061).
Note that both the first and second sensor of claim 1 include analogous requirements. As indicated in the analysis of claim 1, Ko teaches both an audio and image sensor for collecting data required by the limitations of claim 1. Thus, Ko teaches embodiments where the audio and image sensor correspond to either the first or second sensors of claim 1.
Regarding claim 7, Ko teaches the method of claim 1, wherein the first sensor is an audio sensor and the second sensor is an image sensor or a depth sensor (Ko, “In operation S51, the audio processing device 50 determines whether an abnormal event has occurred in the monitoring region, by performing extraction of features of the audio signal and audio class recognition. The audio processing device 50 generates an audio frame of a predetermined time unit from the audio signal and extracts an audio feature from the audio frame. Next, the audio processing device 50 classifies the audio frame… In operation S61, the video processing device 60 determines whether an abnormal event has occurred in the monitoring region, by performing foreground detection and motion information extraction. The video processing device 60 detects a foreground in units of frames by using a back ground Subtraction algorithm and an optical flow technique, and identifies a video frame as a normal or abnormal event via foreground analysis.”, pg. 4, paragraphs 0060-0061).
Similar to the arguments made in the analysis of claim 2 above, Ko teaches embodiments where the audio and image sensor correspond to either the first or second sensors of claim 1.
Regarding claim 9, Ko teaches the method of claim 1, wherein: the first identity classification is a human, an animal, a nonliving object, or an absence of an object within the evaluation field of the first sensor, and the second identity classification is a human, an animal, a nonliving object, or an absence of an object within the evaluation field of the second sensor (Ko, “Referring back to FIG. 5, in operation S515, the audio classifier 505 classifies an audio frame according to a hierarchical approach method. Audio frames may be roughly classified into two categories. Audio frames, such as Screaming, crying, normal conversations, and information commentaries, may be classified into a Vocal category, because they are sounds coming out of the neck of a person. Collisions, noise made by opening or closing a door, stepping Sounds, Sounds coming out of empty elevators, and alarm bell sounds may be classified into a non-vocal category.”, pg. 5, paragraph 0072, lines 1-10, “The object classification unit 621 determines whether the foreground BLOB determined to be a human being is a group of people or an individual person, by comparing a statistical feature extracted from the foreground BLOB with statistical information about an individual object and a group object via the AdaBoost training method or the like.”, pg. 9, paragraph 0103, Abnormal event classification is performed corresponding to data collected for both the audio and image sensors. For collected audio data this includes classifying noises corresponding to humans and objects in the monitored environment and for collected image data this includes classifying foreground BLOBs as a human or a group of humans.).
Regarding claim 10, Ko teaches the method of claim 9, wherein determining the final identity of the potential object is based on a matching potential object being identified by both the first and second identity classifications (Ko, “Based on results of the abnormal event occurrence/nonoccurrence determinations that are periodically received from the audio processing device 50 and the video processing device 60, the context awareness device 70 calculates a combined probability of the audio abnormal probability P and the video abnormal probability P with regard to each of the normal and abnormal situations from the respective combined probability distribution models for the normal and abnormal situations. When a ratio between the combined probabilities for the normal and abnormal situations is greater than a threshold value T, the context awareness device 70 finally determines that an abnormal situation has occurred.”, pg. 4, paragraph, 0063, lines 1-21, The final identity corresponding to an abnormal event of an object in the environment is calculated by comparing probabilities from both sensors. This combination requires matching classifications, such as when the audio device identifies human vocals and the video device identifies humans with abnormal motion. Since these classifications correspond to the same monitored event, the system combines their respective probabilities to make the final determination.).
Regarding claim 11, Ko teaches the method of claim 1, wherein: the first identity classification includes a first identity likelihood score that provides a confidence level that the first identity classification is accurate, and the second identity classification includes a second identity likelihood score that provides a confidence level that the second identity classification is accurate (Ko, “In operation S71, the context awareness device 70 finally determines whether an abnormal situation has occurred in the monitoring region, based on results of the abnormal event occurrence/non-occurrence determinations that are periodically received from the audio processing device 50 and the video processing device 60. The context awareness device 70 calculates an audio abnormal probability Pa and a video abnormal probability Pv by accumulating the results of the abnormal event occurrence/non-occurrence determinations from the audio processing device 50 and the video processing device 60 for a certain period of time.”, pg. 4, paragraph 0062, lines 1-11).
Regarding claim 13, Ko teaches the method of claim 11, wherein determining the final identity of the potential object is based on: the first identity likelihood score exceeding a threshold level of confidence, or the second identity likelihood score exceeding a threshold level of confidence, or both the first and second identity likelihood scores exceeding a threshold level of confidence (Ko, “When a ratio between the combined probabilities for the normal and abnormal situations is greater than a threshold value T, the context awareness device 70 finally determines that an abnormal situation has occurred.”, pg. 4, paragraph 0063, lines 18-21, Thresholding is applied to the combined probabilities from both sensors to determine the final identity corresponding to an abnormal event of the object in the environment. This requires that the classifications from both sensors contribute sufficiently to the combined representation to satisfy this threshold.).
Regarding claim 14, Ko teaches the method of claim 1, wherein: the first identity classification of the potential object within the evaluation field of the first sensor is a human, and the final identity of the potential object is determined to be a human (Ko, “In operation S71, the context awareness device 70 finally determines whether an abnormal situation has occurred in the monitoring region, based on results of the abnormal event occurrence/non-occurrence determinations that are periodically received from the audio processing device 50 and the video processing device 60. The context awareness device 70 calculates an audio abnormal probability Pa and a video abnormal probability Pv by accumulating the results of the abnormal event occurrence/non-occurrence determinations from the audio processing device 50 and the video processing device 60 for a certain period of time.”, pg. 4, paragraph 0062, lines 1-11, “Referring back to FIG. 5, in operation S515, the audio classifier 505 classifies an audio frame according to a hierarchical approach method. Audio frames may be roughly classified into two categories. Audio frames, such as Screaming, crying, normal conversations, and information commentaries, may be classified into a Vocal category, because they are sounds coming out of the neck of a person. Collisions, noise made by opening or closing a door, stepping Sounds, Sounds coming out of empty elevators, and alarm bell sounds may be classified into a non-vocal category.”, pg. 5, paragraph 0072, lines 1-10, Abnormal event classification is performed corresponding to data collected for both the audio and image sensors. For collected audio data this includes classifying noises corresponding to humans and objects in the monitored environment and for collected image data this includes classifying foreground BLOBs as a human or a group of humans. This collected data can then be used for a final identification for abnormal event occurrence corresponding to the observed human or object.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Ko et al. (US 20140055610 A1) in view of Latham et al. (US 20230260387 A1), (hereinafter Latham).
Regarding claim 4, Ko teaches the method of claim 1, wherein the second sensor is an image sensor or an audio sensor (Ko, “In operation S51, the audio processing device 50 determines whether an abnormal event has occurred in the monitoring region, by performing extraction of features of the audio signal and audio class recognition... Next, the audio processing device 50 classifies the audio frame… In operation S61, the video processing device 60 determines whether an abnormal event has occurred in the monitoring region, by performing foreground detection and motion information extraction.”, see pg. 4, paragraphs 0060-0061).
Ko does not teach wherein the first sensor is a depth sensor.
However, Latham teaches wherein the first sensor is a depth sensor (Latham, “For example, in FIG. 1, an event detection component may use computer vision to identify train 102 approaching the station, person 104 on the tracks, and multiple other persons walking in the station. The computer vision may be applied to frames taken from one or more sensors 106, such as a thermal camera or a video camera.”, pg. 2, paragraph 0025, “At block 302, the method 300 includes collecting sensor data from a plurality of sensors located in the environment… Examples of sensors in the plurality of sensors include, but are not limited to, video/imaging cameras, thermal cameras, microphones, smart speakers, computers, moisture sensors, smoke detectors, light detection sensors, depth sensors, etc.”, pg. 3, paragraph 0031).
Ko teaches classifying abnormal events by analyzing data from audio and image sensors (Ko, see pg. 4, paragraphs 0060-0064). Ko does not teach using a depth sensor. Latham teaches implementing a depth sensor for detecting security events in an environment (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Ko’s sensor arrangement to include a depth sensor for abnormal event classification as taught by Latham (Latham, pg. 3, paragraph 0031). The motivation for doing so would have been to capture additional modality data for the environment, thereby increasing the accuracy of classification. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Ko with Latham to obtain the invention according to claim 4.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Ko et al. (US 20210221389 A1) in view of Long et al. (US 20210221389 A1), (hereinafter Long).
Regarding claim 12, Ko teaches the method of claim 11. Ko does not teach further comprising: weighting the first identity likelihood score based on an accuracy rating of the first sensor; and weighting the second identity likelihood score based on an accuracy rating of the second sensor.
However, Long teaches further comprising: weighting the first identity likelihood score based on an accuracy rating of the first sensor; and weighting the second identity likelihood score based on an accuracy rating of the second sensor (Long, “In one example approach , the sensors include a first sensor and a second sensor. The first sensor captures sensor data generated by sensing , with the first sensor, at least one of the sensor accuracy measurement features deployed along the pathway . The second sensor captures sensor data generated by sensing, with the second sensor, at least one of the sensor accuracy measurement features deployed along the pathway. Computing device 116 uses the data read from sensor accuracy measurement feature 154 generate a combined confidence score for the first and second sensors, the combined confidence score indicating perceived effectiveness of operation of the first and second sensors in combination . In one such example approach, generating a combined confidence score for the first and second sensors includes calculating a sensor accuracy for the first sensor and calculating a sensor accuracy for the second sensor , and calculating the combined confidence score for the first and second sensors as a function of a weighted version of the sensor accuracy calculated for the first sensor and a weighted version of the sensor accuracy calculated for the second sensor.”, pg. 16, paragraph 0141).
Ko teaches classifying abnormal events by calculating classification probabilities for an audio and image sensor and combining the probabilities to make final identifications (Ko, see pg. 4, paragraphs 0060-0064). Ko does not teach applying weighting to these probabilities based on an accuracy rating of the sensors. Long teaches implementing weighting for confidence scores corresponding to different sensors (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the probabilities of Ko to be weighted based on respective sensor accuracy as taught by Long (Long, pg. 16, paragraph 0141). The motivation for doing so would have been improve the reliability of the final identifications by accounting for varying detection capabilities of the sensors. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Ko with Long to obtain the invention according to claim 12.
Claims 15, 17-20, and 25-29 are rejected under 35 U.S.C. 103 as being unpatentable over Ko et al. (US 20210221389 A1) in view of Fu et al. (US 20190327448 A1), (hereinafter Fu).
Regarding claim 15, Ko teaches the method of claim 14. Ko does not teach further comprising: accessing a database containing data associated with features of known people; and determining, based on a comparison between the data associated with features of known people and the first data received from the first sensor, whether the human is a known person or an unknown person.
However, Fu teaches further comprising: accessing a database containing data associated with features of known people; and determining, based on a comparison between the data associated with features of known people and the first data received from the first sensor, whether the human is a known person or an unknown person (Fu, “One or more of the devices 102a-102n may receive
an audio input (e.g. , capture a voice) from the visitor 50 and perform a voice analysis to authenticate the visitor 50. The audio input may comprise a voice command to control the devices 102a-102n and/or other network-connected devices. In one example, the visitor 50 may approach one of the devices 102a-102n and speak, “This is Alice, 3467". The authentication may comprise recognizing the voice of the visitor 50 (e.g., recognizing the voice as Alice).”. pg. 3, paragraph 0039, lines 1-9, “Various factors may be analyzed by the device 102 to perform the authentication of the user 50. In the example shown , the signal AUTH_A may correspond to the speech 122. The device 102 may analyze the speech (e.g. , a pass phrase) and / or voice of the user 50 (e.g., vocal patterns, voice biometric markers, etc.) as one factor for authenticating the user 50. In the example shown , the signal AUTH_B may correspond to the user 50. The device 102 may analyze the characteristics of the user 50 based on video analysis (e.g., facial recognition, gait recognition, height detection, etc.).”, pgs. 4 and 5, paragraph 0044, lines 1-11).
Ko teaches classifying abnormal events by classifying objects, including multiple or individual people, in an environment (Ko, see pg. 8, paragraph 0101). Ko does not teach accessing a database corresponding to known people to determine if detected humans are known or unknown. Fu teaches performing person authentication including storing databases corresponding to known people for both audio data and image data and comparing captured data of people to these databases (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the person classification of Ko to include the person authentication as taught by Fu (Fu, pgs. 4 and 5, paragraph 0044, lines 1-11). The motivation for doing so would have been to verify people present in the environment, thereby increasing security of the system. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Ko with Fu to obtain the invention according to claim 15.
Claim 17 contains limitations found analogous to claim 15. As indicated in the analysis of claim 15, Ko in view of Fu teaches all the limitations according to claim 15. Therefore, claim 17 is rejected for the same reasons of obviousness as claim 15.
Regarding claim 18, Ko in view of Fu teaches the method of claim 17, wherein the security action includes at least one of: activating lighting, playing music, opening or closing a window covering, turning a fan on or off, locking or unlocking a door, lighting a fireplace, powering an electrical outlet, turning on or play a predefined channel or video or music on a television or other device, starting or stopping a kitchen appliance, starting or stopping a sprinkler system, opening or closing a garage door, adjusting a temperature or other function of a thermostat or furnace or air conditioning unit (Ko, “In operation S81, the output device 80 may include a display and a speaker and may generate an alarm if it is determined that an abnormal situation has occurred. The display outputs video signals received via a plurality of channels. A detected foreground region of an image displayed on the display may be marked with a box, and a user may be warned by highlighting the edge or entire region of an image of a channel determined to be an abnormal situation. The speaker outputs audio signals received via a plurality of channels. As for a Sound of a channel determined to be an abnormal situation, a warning Sound may be output via the speaker to warn the user.”, pg. 4, paragraph 0064, An alarm is generated in response to an abnormal situation occurring. This includes playing a video with labels or audio signal with warning sounds.).
Regarding 19, Ko teaches the method of claim 14. Ko does not teach further comprising: accessing a database containing data associated with features of known people; and determining, based on a comparison between the data associated with features of known people and the first data received from the first sensor, that the human is an unknown person; accessing a database containing data associated with suspicious behavior patterns; and determining, based on a comparison between the data associated with suspicious behavior patterns and the first data received from the first sensor, that the unknown person is engaging in a suspicious behavior.
However, Fu teaches further comprising: accessing a database containing data associated with features of known people; and determining, based on a comparison between the data associated with features of known people and the first data received from the first sensor, that the human is an unknown person; accessing a database containing data associated with suspicious behavior patterns; and determining, based on a comparison between the data associated with suspicious behavior patterns and the first data received from the first sensor, that the unknown person is engaging in a suspicious behavior. (Fu, “One or more of the devices 102a-102n may receive an audio input (e.g. , capture a voice) from the visitor 50 and perform a voice analysis to authenticate the visitor 50. The audio input may comprise a voice command to control the devices 102a-102n and/or other network-connected devices. In one example, the visitor 50 may approach one of the devices 102a-102n and speak, “This is Alice, 3467". The authentication may comprise recognizing the voice of the visitor 50 (e.g., recognizing the voice as Alice).”. pg. 3, paragraph 0039, lines 1-9, “Various factors may be analyzed by the device 102 to perform the authentication of the user 50. In the example shown , the signal AUTH_A may correspond to the speech 122. The device 102 may analyze the speech (e.g. , a pass phrase) and / or voice of the user 50 (e.g., vocal patterns, voice biometric markers, etc.) as one factor for authenticating the user 50. In the example shown , the signal AUTH_B may correspond to the user 50. The device 102 may analyze the characteristics of the user 50 based on video analysis (e.g., facial recognition, gait recognition, height detection, etc.).”, pgs. 4 and 5, paragraph 0044, lines 1-11, “In some embodiments, the device 102 may automatically perform a command based on the detected behavior of the visitor 50. If the visitor 50 is not detected as a person on the whitelist 360 or the blacklist 362 (e.g., an unknown visitor), the behavior may be analyzed for particular patterns. In one example, after a pre-determined amount of time of detecting the same person the device 102 may conclude that the visitor 50 is loitering and play a sound such as an alarm.”, pg. 12, paragraph 0119, lines 1-9).
Ko teaches classifying abnormal events by classifying objects, including multiple or individual people, in an environment (Ko, see pg. 8, paragraph 0101). Ko further teaches accessing a database containing data associated with suspicious behavior patterns (see pg. 9, paragraphs 0108-0109. Ko does not teach accessing a database corresponding to known people to determine if detected humans are unknown and accessing the database containing data associated with suspicious behavior patterns for the unknown person. Fu teaches performing person authentication including storing databases corresponding to known people for both audio data and image data and in response to determining a visitor is unknown analyzing suspicious behavior patterns (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the person classification of Ko to include the person authentication as taught by Fu (Fu, pgs. 4 and 5, paragraph 0044, lines 1-11). The motivation for doing so would have been to verify people present in the environment, thereby increasing security of the system. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Ko with Fu to obtain the invention according to claim 19.
Regarding claim 20, Ko in view of Fu teaches the method of claim 19, wherein the suspicious behavior includes at least one of: crawling, creeping, running, looking over a shoulder, picking up a package, touching a car, opening a car door, peaking into a car, opening a mailbox, opening a door, opening a window, holding a weapon, screaming, or throwing something (Fu, “In an example, the video streams 312a-312b may be played back simultaneously. For example, the front view video stream 312a may show a view of the face of a visitor 320 and the bottom view 312b may show the waist down view of the visitor 320 and the package 330 at the same time. For example, if the visitor 320 is a burglar stealing the package 330, the front view video 312a may provide a clear view of the face (e.g., identity) of the visitor 320 but not the package 330 and the bottom view video 312b may show the visitor 320 stealing the package 330 but not provide a view of the face to identify the thief. Similarly, if the visitor 320 is attempting to break into the home by opening the door, the front view video 312a may not provide the view of the door but the bottom view video 312b may show the visitor 320 attempting to open the door.”, pg. 9, paragraph 0097, lines 1-15, “In yet another example, a group of users on the blacklist 362 may be people exhibiting the behavior of a burglar (e.g., jiggling doorknobs, checking windows, attempting to enter multiple entrances, etc.).”, pgs. 10 and 11, paragraph 0109, lines 9-12).
Regarding claim 25, Ko in view of Fu teaches the method of claim 19, wherein the security action is performed if the unknown person remains within at least one of the evaluation field of the first sensor or the evaluation field of the second sensor for more than an identified period of time (Fu, “In some embodiments, the device 102 may automatically perform a command based on the detected behavior of the visitor 50. If the visitor 50 is not detected as a person on the whitelist 360 or the blacklist 362 (e.g., an unknown visitor), the behavior may be analyzed for particular patterns. In one example, after a pre-determined amount of time of detecting the same person the device 102 may conclude that the visitor 50 is loitering and play a sound such as an alarm.”, pg. 12, paragraph 0119, lines 1-9).
Regarding claim 26, Ko in view of Fu teaches the method of claim 25, wherein the identified period of time is adjustable based on the suspicious behavior (Fu, “In some embodiments, the device 102 may automatically perform a command based on the detected behavior of the visitor 50. If the visitor 50 is not detected as a person on the whitelist 360 or the blacklist 362 (e.g., an unknown visitor), the behavior may be analyzed for particular patterns. In one example, after a pre-determined amount of time of detecting the same person the device 102 may conclude that the visitor 50 is loitering and play a sound such as an alarm.”, pg. 12, paragraph 0119, lines 1-9, The pre-determined amount of time is dependent on repeated detections of the same person performing suspicious behaviors.).
Regarding claim 27, Ko teaches the method of claim 14. Ko does not teach further comprising: accessing a database containing data associated with features of known people; determining, based on a comparison between the data associated with features of known people and the first data received from the first sensor, that the human is an unknown person; accessing a database containing data associated with safe behavior patterns; and determining, based on a comparison between the data associated with suspicious behavior patterns and the first data received from the first sensor, that the unknown person is engaging in a safe behavior.
However, Fu teaches further comprising: accessing a database containing data associated with features of known people; determining, based on a comparison between the data associated with features of known people and the first data received from the first sensor, that the human is an unknown person; accessing a database containing data associated with safe behavior patterns; and determining, based on a comparison between the data associated with suspicious behavior patterns and the first data received from the first sensor, that the unknown person is engaging in a safe behavior (Fu, “One or more of the devices 102a-102n may receive an audio input (e.g. , capture a voice) from the visitor 50 and perform a voice analysis to authenticate the visitor 50. The audio input may comprise a voice command to control the devices 102a-102n and/or other network-connected devices. In one example, the visitor 50 may approach one of the devices 102a-102n and speak, “This is Alice, 3467". The authentication may comprise recognizing the voice of the visitor 50 (e.g., recognizing the voice as Alice).”. pg. 3, paragraph 0039, lines 1-9, “Various factors may be analyzed by the device 102 to perform the authentication of the user 50. In the example shown , the signal AUTH_A may correspond to the speech 122. The device 102 may analyze the speech (e.g. , a pass phrase) and / or voice of the user 50 (e.g., vocal patterns, voice biometric markers, etc.) as one factor for authenticating the user 50. In the example shown , the signal AUTH_B may correspond to the user 50. The device 102 may analyze the characteristics of the user 50 based on video analysis (e.g., facial recognition, gait recognition, height detection, etc.).”, pgs. 4 and 5, paragraph 0044, lines 1-11, “The example video frame 450 may comprise a delivery truck 452 and a delivery person 454. In the example video frame 450, the delivery person 454 is shown carrying the package 330. For example, the front-facing capture device 234a may capture images of the delivery person 454 approaching the premises. For example, the video frames may capture a sequence of events corresponding to the delivery truck 452 approaching and parking near the premises 402a, the delivery person 454 getting out of the truck 452 and retrieving the package 330 from the truck 452 and then carrying the package 330 up to the access point 404a (e.g., the front door). The video processor 214 may be configured to intelligently analyze the video frames to determine the behavior of the visitor (e.g., the delivery person 454) and come to the conclusion that the behavior is consistent with a package delivery behavior. Sensor fusion may be implemented for further authentication (e.g., detecting a diesel sound of the truck 452, audio of the delivery person 454 announcing themselves, etc.).”, pg. 12, paragraph 0123).
Ko teaches classifying abnormal events by classifying objects, including multiple or individual people, in an environment (Ko, see pg. 8, paragraph 0101). Ko further teaches accessing a database containing data associated with safe behavior patterns (see pg. 9, paragraphs 0108-0109). Ko does not teach accessing a database corresponding to known people to determine if detected humans are unknown and accessing the database containing data associated with safe behavior patterns for the unknown person. Fu teaches performing person authentication including storing databases corresponding to known people for both audio data and image data and in response to determining a visitor is unknown analyzing safe behavior patterns, such as a delivery driver delivery a package (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the person classification of Ko to include the person authentication as taught by Fu (Fu, pgs. 4 and 5, paragraph 0044, lines 1-11). The motivation for doing so would have been to verify the identity of people present in the environment along with their behaviors, thereby increasing security of the system. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Ko with Fu to obtain the invention according to claim 27.
Regarding claim 28, Ko in view of Fu teaches the method of claim 27, wherein the safe behavior includes at least one of: walking a dog, riding a bike, delivering a package, or performing yard work (Fu, “The example video frame 450 may comprise a delivery truck 452 and a delivery person 454. In the example video frame 450, the delivery person 454 is shown carrying the package 330. For example, the front-facing capture device 234a may capture images of the delivery person 454 approaching the premises. For example, the video frames may capture a sequence of events corresponding to the delivery truck 452 approaching and parking near the premises 402a, the delivery person 454 getting out of the truck 452 and retrieving the package 330 from the truck 452 and then carrying the package 330 up to the access point 404a (e.g., the front door). The video processor 214 may be configured to intelligently analyze the video frames to determine the behavior of the visitor (e.g., the delivery person 454) and come to the conclusion that the behavior is consistent with a package delivery behavior. Sensor fusion may be implemented for further authentication (e.g., detecting a diesel sound of the truck 452, audio of the delivery person 454 announcing themselves, etc.).”, pg. 12, paragraph 0123).
Regarding claim 29, Ko in view of Fu teaches the method of claim 27, wherein the security action is performed if the unknown person remains within at least one of the evaluation field of the first sensor and the evaluation field of the second sensor for more than an identified period of time (Fu, “In some embodiments, the device 102 may automatically perform a command based on the detected behavior of the visitor 50. If the visitor 50 is not detected as a person on the whitelist 360 or the blacklist 362 (e.g., an unknown visitor), the behavior may be analyzed for particular patterns. In one example, after a pre-determined amount of time of detecting the same person the device 102 may conclude that the visitor 50 is loitering and play a sound such as an alarm.”, pg. 12, paragraph 0119, lines 1-9).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CONNOR LEVI HANSEN whose telephone number is (703)756-5533. The examiner can normally be reached Monday-Friday 9:00-5:00 (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CONNOR L HANSEN/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights