DETAILED ACTION
This action is written in response to the application filed 6/5/23. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
Claims 1-2, 6-7, 12-18 and 21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
In determining whether the claims are subject matter eligible, the Examiner applies the 2019 USPTO Patent Eligibility Guidelines, as well as guidance from MPEP § 2106.
Step 1: Is the claim to a process, machine, manufacture, or composition of matter? Yes—claim 1 recites a method, which is a process.
Step 2A, prong one: Does the claim recite an abstract idea, law of nature or natural phenomenon? Yes—the claim recites one or more limitations which—under their broadest reasonable interpretation—covers performance of the limitation in the mind (see table below).
Claim limitation
Examiner analysis
1. A method, comprising:
using a control circuit of the camera to determine when an object has moved into the field of view of the camera;
This is a mental process akin to a human evaluation/judgment/observation.
determining a category for the object using one or more rules with criteria specifying multiple different categories of objects; and
This is a mental process akin to a human evaluation/judgment/observation.
Because the claim recites limitations which can practically be implemented as mental processes, the claim recites a mental process.
Step 2A, prong two: Does the claim recite additional elements that integrate the judicial exception into a practical application? No—the claim does not recite even generic computer hardware.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No—the additional limitations are addressed below:
obtaining image data from a camera defining a field of view, wherein the image data includes one or more separate images taken at different points in time;
This is insignificant pre-solution activity: gathering data to be processed in subsequent steps.
sending an alert to a computing device when the object moves within the field of view indicating the category of the object.
This is insignificant post-solution activity: transmitting results from a preceding step.
The only limitation on the performance of the described method is that it must be performed “using a control circuit”. The claim thus recites computing components only at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. The statement that the method is performed by computer does not satisfy the test of “inventive concept.” See Alice Corp. Pty. Ltd. v. CLS Bank Int’l, 573 U.S. 208, 134 S. Ct. 2347, 2360 (2014).
For the reasons above, claim 1 is rejected as being directed to non-patentable subject matter under §101. This rejection applies equally to dependent claims 2-22. The additional limitations of the dependent claims are addressed briefly below. Taken alone, the additional elements of the dependent claims above do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation.
Claim limitation
Examiner analysis
2. The method of claim 1, comprising:
sending the category and one or more of the separate images to a personal computing device, wherein the personal computing device is configured to present a user interface that provides access to the separate images and the category.
This is insignificant post-solution activity: transmitting results from a preceding step to a user device. See generally MPEP 2106.05(g).
4. The method of claim 1, comprising:
sending one or more of the separate images, the category, and user input indicating whether the image data matches the category to a data analytics service via a communication link.
This is insignificant post-solution activity: transmitting results from a preceding step. See generally MPEP 2106.05(g).
6. The method of claim 1, comprising:
comparing pixel data from one of the separate images to corresponding pixel data from another different one of the separate images.
This is a mental process akin to a human judgment/opinion/observation.
7. The method of claim 1, comprising:
comparing regions from one of the separate images to one or more predetermined image patterns stored in a memory of the control circuit.
This is a mental process akin to a human judgment/opinion/observation.
12 The method of claim 1, comprising:
comparing regions from sound data captured by the camera to one or more predetermined audio input patterns stored in a memory of the control circuit.
This is a mental process akin to a human judgment/opinion/observation.
13. The method of claim 1, wherein the camera is mounted adjacent to a door.
This is insignificant pre-solution information: specifying the source of received data. See generally MPEP 2106.05(g)(3) discussing “Mere Data Gathering”.
14. The method of claim 1, wherein the camera is mounted in a doorbell mechanism.
This is insignificant pre-solution information: specifying the source of received data. See generally MPEP 2106.05(g)(3) discussing “Mere Data Gathering”.
15. The method of claim 5, wherein the rule criteria include threshold values ranging between 0.0 and 1.0.
This is a mental process akin to a human judgment/opinion/observation.
16. The method of claim 1, wherein the control circuit includes a processor, memory, and communication circuits operable to establish and maintain one or more communication links.
This is insignificant pre-solution information: specifying the source of received data. See generally MPEP 2106.05(g)(3) discussing “Mere Data Gathering”.
17. The method of claim 1, wherein the control circuit is operable to maintain one or more wireless or wired communication links with one or more other computing devices via a computer network.
This is insignificant pre-solution information: specifying the source of received data. See generally MPEP 2106.05(g)(3) discussing “Mere Data Gathering”.
18. The method of claim 1, wherein the camera is an IP camera.
This is insignificant pre-solution information: specifying the source of received data. See generally MPEP 2106.05(g)(3) discussing “Mere Data Gathering”.
21. The method of claim 5, wherein the rule criteria optionally includes one or more predetermined image patterns and/or one or more predetermined audio input patterns.
This is a mental process akin to a human judgment/opinion/observation.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 6, 16-17, 20 and 22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Xu (US 11,935,377 B1).
Regarding claim 1, Xu discloses a method, comprising:
obtaining image data from a camera defining a field of view, wherein the image data includes one or more separate images taken at different points in time;
Col. 5, lines 18 et seq., “In some embodiments, the lenses 108a-108n may be directed, tilted, panned, zoomed and/or rotated to capture the environment surrounding the camera system 100 ( e.g., capture data from the field of view).”
Col. 1, lines 34 et seq., “video frames”.
using a control circuit of the camera to determine when an object has moved into the field of view of the camera;
Col. 11, lines 19 et seq., “The video pipeline may be configured to recognize objects. Objects may be recognized by interpreting numerical and/or symbolic information to determine that the visual data represents a particular type of object and/or feature. For example, the number of pixels and/or the colors of the pixels of the video data may be used to recognize portions of the video data as objects.”
Fig. 5 (reproduced below), depicting object detection functionality within a device processor.
PNG
media_image1.png
644
664
media_image1.png
Greyscale
determining a category for the object using one or more rules with criteria specifying multiple different categories of objects; and
Fig. 9, “Perform classification on objects based on characteristics detected”.
Col. 10, line 19 et seq., “may be configured to recognize gardener/pool maintenance person and inhibit triggering an alarm. …. In another example, the low cost structured light based 3D sensing system may be configured to trigger an alarm upon recognition of certain objects (e.g. restraining order is out against ex-spouse, alert 911 if that person is detected).”
sending an alert to a computing device when the object moves within the field of view indicating the category of the object.
Col. 10, line 14 et seq., “In an example, the low cost structured light based 3D sensing system may be used to lock/unlock a door, arm/disarm an alarm system, and/or allow "tripwire" control of access to a restricted region ( e.g., a garden, a vehicle, a garage, a house, etc.).”
Col. 22, line 59 et seq., “The user interface 730 may enable the users to provide 60 input to the access control device 50 and/or enable the access control device 50 to present output to the users. In one example, the user interface 730 may comprise a display and/or a keypad. For example, the user interface 730 may be one component of the access control device 50 that may be configured to receive the credentials presented by the user.”
Col. 22, line 40 et seq., “Based on the results of the computer vision operations, the processors 714 may generate a security measure ( e.g., sound an alarm, present a notification to a security guard, initiate a lockdown procedure, etc.).
Regarding claim 6, Xu discloses the further limitation comprising:
comparing pixel data from one of the separate images to corresponding pixel data from another different one of the separate images.
Col. 11, lines 23 et seq., “For example, the number of pixels and/or the colors of the pixels of the video data may be used to recognize portions of the video data as objects.”
Regarding claim 16, Xu discloses the further limitation wherein the control circuit includes a processor, memory, and communication circuits operable to establish and maintain one or more communication links.
Col. 19, line 25 ‘processor’.
Col. 20, line 35 ‘memory’.
Col. 19, lines 41 et seq., “In some embodiments, the camera system lO0i may communicate with the cloud service 702 and/or the access control device 50 wirelessly. In some embodiments, the camera system l00i may communicate with the cloud service 702 and/or the access control device 50 via the physical 45 connection 722 (e.g., an ethernet connection, a USB connection, a wire, etc.).”
Regarding claim 17, Xu discloses the further limitation wherein the control circuit is operable to maintain one or more wireless or wired communication links with one or more other computing devices via a computer network.
Col. 19, lines 41 et seq., “In some embodiments, the camera system lO0i may communicate with the cloud service 702 and/or the access control device 50 wirelessly. In some embodiments, the camera system l00i may communicate with the cloud service 702 and/or the access control device 50 via the physical 45 connection 722 (e.g., an ethernet connection, a USB connection, a wire, etc.).”
Regarding claim 20, Xu discloses the further limitation wherein a data analytics service is in communication with the control circuit and is operable to analyze image or audio data provided by the camera using a neural network.
Col. 12, line 4 et seq., “One of the hardware modules 190a-190n (e.g., 190b) may 5 implement a convolutional neural network (CNN) module. The CNN module 190b may be configured to perform the computer vision operations on the video frames.”
Regarding claim 22, Xu discloses the further limitation wherein a data analytics service is in communication with the control circuit and is operable to analyze image or audio data provided by the camera using a convolutional neural network.
Col. 12, line 4 et seq., “One of the hardware modules 190a-190n (e.g., 190b) may 5 implement a convolutional neural network (CNN) module. The CNN module 190b may be configured to perform the computer vision operations on the video frames.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The following are the references relied upon in the rejections below:
Calacci (Calacci, Dan, Jeffrey J. Shen, and Alex Pentland. "The cop in your neighbor's doorbell: Amazon ring and the spread of participatory mass surveillance." Proceedings of the ACM on Human-Computer Interaction 6.CSCW2 (2022): 1-47.)
Chandola (Chandola, Varun, Arindam Banerjee, and Vipin Kumar. "Anomaly detection: A survey." ACM computing surveys (CSUR) 41, no. 3 (2009): 1-58.)
Jin (Jin, Xiaoying, and Scott Paswaters. "A fuzzy rule base system for object-based feature extraction and classification." Signal Processing, Sensor Fusion, and Target Recognition XVI. Vol. 6567. SPIE, 2007.)
Joshi (Joshi, Ajay J., Fatih Porikli, and Nikolaos Papanikolopoulos. "Breaking the interactive bottleneck in multi-class classification with active selection and binary feedback." 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 2010.)
Mondal (Mondal, Sujoy, and Abhirup Das Barman. "Human auditory model based real-time smart home acoustic event monitoring." Multimedia Tools and Applications 81.1 (2022): 887-906. Published online 18 Sept 2021.)
Neoran (US 2010/0004926 A1)
Stolojescu-Crisan (Stolojescu-Crisan, Cristina, Calin Crisan, and Bogdan-Petru Butunoi. "Access control and surveillance in a smart home." High-Confidence Computing 2.1 (Mar 2022): 100036.)
Xu (US 11,935,377 B1)
Claims 2, 13-14 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Xu and Stolojescu-Crisan.
Regarding claim 2, Stolojescu-Crisan discloses the following further limitation which Xu does not disclose comprising:
sending the category and one or more of the separate images to a personal computing device, wherein the personal computing device is configured to present a user interface that provides access to the separate images and the category.
P. 8, fig. 12, “normal desktop view”, image captions: “Fat Cat”, “Fluffy Cat”, “The Dog”.
PNG
media_image2.png
277
974
media_image2.png
Greyscale
At the time of filing, it would have been obvious to a person of ordinary skill to combine the smart device GUI disclosed by Stolojescu-Crisan with the Xu system because home security-related alerts may be time sensitive, and homeowners are likely to have a smartphone at hand at all times. Prompt notification will provide for escalation (eg calling 911) or de-escalation (eg disarming an alarm).
Regarding claim 13, Xu discloses the further limitation wherein the camera is mounted adjacent to a door.
Col. 2, line 27, “doorbell cameras”.
Regarding claim 14, Xu discloses the further limitation wherein the camera is mounted in a doorbell mechanism.
Col. 2, line 27, “doorbell cameras”.
Regarding claim 16, Stolojescu-Crisan discloses the further limitation wherein the control circuit includes a processor, memory, and communication circuits operable to establish and maintain one or more communication links.
P. 1, introduction, “IoT uses the Internet Protocol (IP) to connect devices, which include smart- phones, tablets and digital assistants to various types of sensors, appliances, and systems such as lighting, temperature, or security. The characteristics of the IoT are very wide and include a variety of physical elements, as shown in Fig. 1 [3] .”
The Examiner notes that a processor, memory and communication circuits are inherent throughout the IoT system described above.
The obviousness analysis of claim 2 applies equally here.
Regarding claim 17, Stolojescu-Crisan discloses the further limitation wherein the control circuit is operable to maintain one or more wireless or wired communication links with one or more other computing devices via a computer network.
P. 1, introduction, “IoT uses the Internet Protocol (IP) to connect devices, which include smart- phones, tablets and digital assistants to various types of sensors, appliances, and systems such as lighting, temperature, or security. The characteristics of the IoT are very wide and include a variety of physical elements, as shown in Fig. 1 [3] .”
P. 2, fig. 2 (reproduced below).
PNG
media_image3.png
440
536
media_image3.png
Greyscale
The obviousness analysis of claim 2 applies equally here.
Regarding claim 18, Stolojescu-Crisan discloses the further limitation wherein the camera is an IP camera.
P. 1, introduction, “IoT uses the Internet Protocol (IP) to connect devices, which include smart- phones, tablets and digital assistants to various types of sensors, appliances, and systems such as lighting, temperature, or security. The characteristics of the IoT are very wide and include a variety of physical elements, as shown in Fig. 1 [3] .”
The obviousness analysis of claim 2 applies equally here.
Regarding claim 19, Stolojescu-Crisan discloses the further limitation wherein a data analytics service is in communication with the control circuit and is operable to analyze image or audio data provided by the camera using artificial intelligence.
Abstract and passim: “smart home solutions” using “smart devices”.
See also p. 1, introduction, “Intelligent Building Management Systems”.
The obviousness analysis of claim 2 applies equally here.
Claims 3-4 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Xu and Joshi.
Regarding claim 3, Joshi discloses the following further limitation which Xu does not disclose comprising:
accepting user input from a user indicating that the image data matches the category determined by the control circuit.
P. 3, first paragraph, “Based on the estimated membership probabilities for the query image, the sample selection algorithm selects a sample image from the current training set. The query-sample pair is shown to the user for feedback. If a “match” response is obtained, indicating that the query and sample images belong to the same category, the query image is added to the current training set along with its category label. If a “no-match” response is obtained, the sample selection algorithm is again invoked to ask for a different sample image.”
At the time of filing, it would have been obvious to a person of ordinary skill to combine the active learning technique disclosed by Joshi with the Xu system because this will increase the number (and reliability) of labeled samples for further training the object detection system.
Regarding claim 4, Joshi discloses the further limitation comprising:
sending one or more of the separate images, the category, and user input indicating whether the image data matches the category to a data analytics service via a communication link.
P. 3, first paragraph, “Based on the estimated membership probabilities for the query image, the sample selection algorithm selects a sample image from the current training set. The query-sample pair is shown to the user for feedback. If a “match” response is obtained, indicating that the query and sample images belong to the same category, the query image is added to the current training set along with its category label. If a “no-match” response is obtained, the sample selection algorithm is again invoked to ask for a different sample image.”
P. 2, fig. 2 (reproduced below), ‘communication links’ are indicated by the arrows between components.
PNG
media_image4.png
448
382
media_image4.png
Greyscale
The obviousness analysis of claim 3 applies equally here.
Regarding claim 10, Joshi discloses the following further limitation which Xu does not disclose comprising:
accepting user input from a user indicating that sound data captured by the camera matches a category determined by the control circuit.
P. 3, first paragraph, “Based on the estimated membership probabilities for the query image, the sample selection algorithm selects a sample image from the current training set. The query-sample pair is shown to the user for feedback. If a “match” response is obtained, indicating that the query and sample images belong to the same category, the query image is added to the current training set along with its category label. If a “no-match” response is obtained, the sample selection algorithm is again invoked to ask for a different sample image.”
The obviousness analysis of claim 3 applies equally here.
Claims 5, 8 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Xu and Chandola.
Regarding claim 5, Chandola discloses the following further limitation which Xu does not disclose comprising:
using a data analytics service to determine updated rule criteria for at least one of the rules specifying different categories of objects, wherein the data analytics service uses the image data provided by the camera, the category determined by the control circuit, and user input to determine the updated rule criteria.
P. 21, “A basic multi-class rule-based technique consists of two steps. The first step is to learn rules from the training data using a rule learning algorithm, such as RIPPER, Decision Trees, and so on. Each rule has an associated confidence value that is proportional to ratio between the number of training instances correctly classified by the rule and the total number of training instances covered by the rule. The second step is to find, for each test instance, the rule that best captures the test instance. The inverse of the confidence associated with the best rule is the anomaly score of the test instance.”
See also p. 17, sec. 3.5, discussion applications to image processing.
At the time of filing, it would have been obvious to a person of ordinary skill to apply rule-based anomaly detection (as taught by Chandola) to the intrusion detection system of Xu because intrusion detection is an anomaly detection task, meaning it is also a classification task. This combination would provide for alerts or alarms upon detection of unusual activity in the house, eg an intruder.
Regarding claim 8, Xu discloses the further limitation comprising:
obtaining sound data from an input device of the camera, wherein the sound data includes sound data obtained from the input device at different points in time; and
Col. 4, lines 44 et seq., “The circuit 114 may be implemented as one or more sensors (e.g., motion, ambient light, proximity, sound, etc.).”
Chandola discloses the following further limitation which Xu does not disclose comprising:
using a control circuit of the camera to compare the sound data with rule criteria in the control circuit to determine an event category, wherein the rule criteria specifies one or more categories of events.
P. 18, “A single sensor network might be comprised of sensors that collect different types of data, such as binary, discrete, continuous, audio, video, and so forth. The data is generated in a streaming mode. Often times the environment in which the various sensors are deployed, and the communication channel, induce noise and missing values in the collected data.”
P. 21, “Variants of the basic technique have been proposed for anomaly detection in audio signal data”.
The obviousness analysis of claim 5 applies equally here.
Regarding claim 21, Xu discloses the further limitation wherein the rule criteria optionally includes one or more predetermined image patterns and/or one or more predetermined audio input patterns.
P. 12, line 19 et seq., “The CNN module 190b may be 20 configured to implement pattern and/or image recognition using a training process through multiple layers of feature detection. The CNN module 190b may be configured to conduct inferences against a machine learning model.”
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Xu, Chandola and Calacci.
Regarding claim 9, Calacci discloses the following further limitation which Xu does not disclose comprising:
sending the event category and at least a portion of sound data captured by the camera to a personal computing device, wherein the personal computing device is configured to present a user interface that provides access to the sound data and the category.
P. 20, fig. 8 (a) and (b) (reproduced below). The Examiner notes that each video clip includes sound, as suggested by the speaker (volume) icon in the lower right corner of each video frame.
PNG
media_image5.png
290
1162
media_image5.png
Greyscale
At the time of filing, it would have been obvious to a person of ordinary skill to transmit video data including sound to a user’s personal computing device because this would provide for an immediate alert to the user / homeowner of activities taking place at their residence. Audio can give important additional information or context about persons or objects depicted in the frames.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Xu and Neoran.
Regarding claim 11, Neoran discloses the following further limitation which Xu does not disclose comprising:
using a data analytics service to determine updated rule criteria for at least one of the rules specifying different categories of objects, wherein the data analytics service uses sound data captured by the camera, the category determined by the control circuit, and user input to determine the updated rule criteria.
[0011] “In accordance with an aspect of an embodiment of the invention, classification and/or segmentation of the audio content by the apparatus includes obtaining an input audio signal; dividing the signal into one or more audio segments; classifying each segment of the audio signal, for example, using a multi-stage sieve-like approach and applying Bayesian and/or rule-based methods”.
At the time of filing, it would have been obvious to a person of ordinary skill to apply the rule-based audio classification techniques disclosed by Neoran with the intrusion detection system of Xu because audio data is captured by the Xu system, and the analysis thereof would provide for more complete and accurate classification predictions by the Xu system.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Xu and Mondal.
Regarding claim 12, Mondal discloses the further limitation which Xu does not disclose comprising:
comparing regions from sound data captured by the camera to one or more predetermined audio input patterns stored in a memory of the control circuit.
P. 892, fig. 3(c) (reproduced below).
PNG
media_image6.png
302
360
media_image6.png
Greyscale
At the time of filing, it would have been obvious to a person of ordinary skill to apply the audio pattern matching technique disclosed by Mondal with the intrusion detection system of Xu because the former can help inform better automated classification of intrusion events vs non-intrusion events.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Xu, Chandola and Jin.
Regarding claim 15, Jin discloses the following further limitation which Xu/Chandola do not disclose wherein the rule criteria include threshold values ranging between 0.0 and 1.0.
P. 9, sec. 4.1, “In the experiments, the confidence value thresholds were set slightly lower than the half magnitude of membership function which is 0.5.”
At the time of filing, it would have been obvious to a person of ordinary skill to apply the rule threshold criteria specified by Jin in the Xu/Chandola system because membership functional are commonly normalized between 0 and 1 for interpretability.
Additional Relevant Prior Art
The following references were identified by the Examiner as being relevant to the disclosed invention, but are not relied upon in any particular prior art rejection:
Edwards discloses a home / neighborhood security system featuring a plurality of linked cameras. (US 2016/0309123 A1)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vincent Gonzales whose telephone number is (571) 270-3837. The examiner can normally be reached on Monday-Friday 7 a.m. to 4 p.m. MT. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang, can be reached at (571) 270-7092.
Information regarding the status of an application may be obtained from the USPTO Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
/Vincent Gonzales/Primary Examiner, Art Unit 2124
Prosecution Insights