DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments/Amendments
2. With respect to Claim Rejection 35 U.S.C § 103 towards Claims 1-2, 7, 15-17, 21, 23-25, 27-30 and 32, Applicant’s arguments have been considered but are moot because the new ground to rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenge in the argument.
With respect to Claims 8-10, 26 and 31, all of the rejections in the most recent Office action are overcome. Thus, the rejections have been withdrawn.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1, 7, 29 are rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1.)
With respect to Claim 1, Agrawal et al. disclose
A method comprising:
receiving, by a computing device, audio data indicative of a voice command (Agrawal et al. [0013] describes receiving voice command from a user);
determining, based on receiving the audio data, a direction of a source of the audio data and a thermal signature associated with the direction of the source (Agrawal et al. [0013] describes detect the direction of the voice command and associate the thermal signature in the voice command direction with the device 100 user/owner. Therefore, if more than one person is present in proximity of the device 100, the voice command direction can be used to distinguish the user's thermal signature from thermal signatures of other persons in proximity, [0022] the voice recognition module 253 may send an indication of a valid command to the voice direction module 252 which may then communicate the voice direction information to the thermal signature detection module 251, [0034] the voice direction module 252 will determine the direction of the voice command and the thermal signature detection module 251 will attempt to obtain the thermal signature from the voice command direction. The method of operation will then proceed to decision block 507 and determine whether the thermal signature of the user is detectable in the direction of the voice command );
determining that the thermal signature is indicative of a user based on an analysis of the thermal signature (Agrawal et al. [0012] Each of four IR sensors 101 can detect the presence of a human body and take the environmental temperature versus the human body temperature into account to obtain the user’s thermal signature as referred to herein. In one example embodiment, the IR sensors 101 detect temperature within a zero to one foot distance range and can detect proximity within a one foot to eight foot range. See paragraphs [0011, 0012 and 0025]); and
sending, based on determining that the thermal signature is indicative of the user, data indicative of the voice command to a device to cause the device to execute the voice command (Agrawal et al. [0011] describes the device 100 can detect a user’s thermal signal when the user logs in or unlocks the device using an appropriate voice command. The device detects a thermal signature of a user who operates the external lockable device and send commands over the wireless interface to lock or unlock the lockable device based on the presence of the user’s thermal signature.)
Agrawal et al. disclose determining that the thermal signature is indicative of a user based on an analysis of the thermal signature in order to verify the presence of the user. However, Agrawal et al. fail to explicitly teach a spatial arrangement comprising a plurality of intensity regions within the thermal signature that are characteristic of the user.
However, Bills et al. teach
determining that the thermal signature is indicative of a user based on an analysis of the thermal signature indicating a spatial arrangement comprising a plurality of intensity regions within the thermal signature that are characteristic of the user (Bills et al. col. 6 lines 58-62 In one example, a voltage resulting from the photons collected at the infrared camera 104 during a single open exposure window 204C may be read from each pixel of the infrared camera 104 to determine the presence of the object 101 within the photon collection zone 210C, col. 17 lines 12-26 The LWIR microbolometer camera 1014 may be a thermal (e.g., infrared) camera having a sensor array configured to detect, at each of its imaging elements, thermal radiation typically associated with humans and various animals. The biological detection preprocessor 1012 may be configured to control the operation of the LWIR microbolometer camera 1014, possibly in response to commands received from the vehicle autonomy processor 1030. Additionally, the biological detection preprocessor 1012 may process the image data received from the LWIR microbolometer camera 1014 to help identify whether any particular imaged objects in the scene are human or animal in nature, as well as possibly to specifically distinguish humans from other thermal sources, such as by way of intensity, size, and/or other characteristics); and
Agrawal et al. and Bills et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources (Bills et al. col. 17 lines 12-26 The LWIR microbolometer camera 1014 may be a thermal (e.g., infrared) camera having a sensor array configured to detect, at each of its imaging elements, thermal radiation typically associated with humans and various animals. The biological detection preprocessor 1012 may be configured to control the operation of the LWIR microbolometer camera 1014, possibly in response to commands received from the vehicle autonomy processor 1030. Additionally, the biological detection preprocessor 1012 may process the image data received from the LWIR microbolometer camera 1014 to help identify whether any particular imaged objects in the scene are human or animal in nature, as well as possibly to specifically distinguish humans from other thermal sources, such as by way of intensity, size, and/or other characteristics.)
With respect to Claim 7, Agrawal et al. in view of Bills et al. teach
wherein determining the thermal signature comprises receiving, from one or more devices, data indicative of one or more infrared signals and analyzing the data indicative of the one or more infrared signals to determine the thermal signature (Agrawal et al. [0009-0010, 0012, 0024, 0027] describes using a thermal infrared sensor to detect a presence of the user.)
With respect to Claim 29, Agrawal et al. in view of Bills et al. disclose
wherein the device comprises a thermostat, television, speaker, or user device configured to execute the voice command (Agrawal et al. Fig. 1 element 103 Lockable device).
5. Claim 2 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Zhang (US 2019/0098402 A1.)
With respect to Claim 2, Agrawal et al. in view of Bills et al. teach all the limitations of Claim 1 upon which Claim 2 depends. Agrawal et al. in view of Bills et al. fail to explicitly tech
wherein receiving the audio data comprises capturing the audio data using an array of microphones, and wherein determining the direction of the source comprises determining, based on spatial processing of the audio data from the array of microphones, the direction of the source.
However, Zhang teaches
wherein receiving the audio data comprises capturing the audio data using an array of microphones, and wherein determining the direction of the source comprises determining, based on spatial processing of the audio data from the array of microphones, the direction of the source (Zhang [0007] In response to detecting a sound signal by the microphone array, the device determines a sound source direction corresponding to detected sound signal according to a sound phase difference obtained by each microphone in the microphone array.)
Agrawal et al., Bills et al. and Zhang are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of the spatial processing as taught by Zhang for the benefit of detecting direction of the sound source (Zhang [0007] In response to detecting a sound signal by the microphone array, the device determines a sound source direction corresponding to detected sound signal according to a sound phase difference obtained by each microphone in the microphone array.)
6. Claims 21, 23 are rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Kamiizumi et al. (US 2023/0137225 A1.)
With respect to Claim 21, Agrawal et al. in view of Bills et al. teach all the limitations of Claim 1 upon which Claim 21 depends. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein the plurality of intensity regions within the thermal signature comprise at least one higher thermal intensity region and at least one lower thermal intensity region, wherein:
the at least one higher thermal intensity region is associated with at least one of: a face, one or more arms, or one or more hands, or another exposed upper body area; and
the at least one lower thermal intensity region is associated with at least one of: a shirt, pants, or other leg and torso areas.
However, Kamiizumi et al. teach
wherein the plurality of intensity regions within the thermal signature comprise at least one higher thermal intensity region and at least one lower thermal intensity region, wherein:
the at least one higher thermal intensity region is associated with at least one of: a face, one or more arms, or one or more hands, or another exposed upper body area (Kamiizumi [0040] Person 30 shown in FIG. 2A wears jacket 30a and pants 30b. The surface temperatures of jacket 30a and pants 30b are close to an ambient temperature. Hence, for example, when the ambient temperature is a room temperature of about 25° C., on the surface temperatures of person 30 detected by heat source detector 20, the surface temperatures of parts of jacket 30a and pants 30b are lower than those of the other parts (the face, the neck and the arms) where skin is exposed. Therefore, as compared with the surface temperatures of the parts where the skin is exposed, the surface temperatures of jacket 30a and pants 30b are displayed to have low relative densities (colors close to the colors of the surrounding pixels). In the temperature environment described above, since the ambient temperature is lower than the temperatures of the surfaces of the clothes, when an object which has a temperature less than or equal to the ambient temperature does not exist inside the viewing angle ϕ, the region other than the person of thermal image 40 has the lowest density. For example, when the room temperature is about 25° C., the average skin temperature of the face is about 33° C., the temperature of jacket 30a is about 27° C., the temperature of the arms (exposed parts) is about 30° C., and the temperature of pants 30b is about 28° C., a temperature distribution as shown in thermal image 40 is provided. See paragraphs [0038-0039]); and
the at least one lower thermal intensity region is associated with at least one of: a shirt, pants (See paragraph [0038-0040]), or other leg and torso areas.
Agrawal et al., Bills et al. and Kamiizumi are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of the thermal image as taught by Kamiizumi for the benefit of detecting the presence of the person (Kamiizumi Fig. 2B and [0038-0040].)
With respect to Claim 23, Agrawal et al. in view of Bills and Kamiizumi teach
wherein the analysis of the thermal signature further indicates that the spatial arrangement, comprising higher and lower thermal intensity regions within the thermal signature, matches a reference spatial arrangement comprising one or more of an arm, head, leg, foot, body, or other body part pattern that is characteristic of human presence (Kamiizumi [0040] Person 30 shown in FIG. 2A wears jacket 30a and pants 30b. The surface temperatures of jacket 30a and pants 30b are close to an ambient temperature. Hence, for example, when the ambient temperature is a room temperature of about 25° C., on the surface temperatures of person 30 detected by heat source detector 20, the surface temperatures of parts of jacket 30a and pants 30b are lower than those of the other parts (the face, the neck and the arms) where skin is exposed. Therefore, as compared with the surface temperatures of the parts where the skin is exposed, the surface temperatures of jacket 30a and pants 30b are displayed to have low relative densities (colors close to the colors of the surrounding pixels). In the temperature environment described above, since the ambient temperature is lower than the temperatures of the surfaces of the clothes, when an object which has a temperature less than or equal to the ambient temperature does not exist inside the viewing angle ϕ, the region other than the person of thermal image 40 has the lowest density. For example, when the room temperature is about 25° C., the average skin temperature of the face is about 33° C., the temperature of jacket 30a is about 27° C., the temperature of the arms (exposed parts) is about 30° C., and the temperature of pants 30b is about 28° C., a temperature distribution as shown in thermal image 40 is provided. See paragraphs [0038-0039]).
7. Claims 24-25 are rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Barton et al. (US 2022/0122431 A1.)
With respect to Claim 24, Agrawal et al. in view of Bills et al. disclose all the limitations of Claim 1 upon which Claim 24 depends. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein determining that the thermal signature is indicative of the user is further based on inputting the thermal signature associated with the source into a machine learning model, wherein the machine learning model is configured to categorize the thermal signature as human or non-human.
However, Barton et al. teach
wherein determining that the thermal signature is indicative of the user is further based on inputting the thermal signature associated with the source into a machine learning model, wherein the machine learning model is configured to categorize the thermal signature as human or non-human (Barton et al. [0087, 0123 and 0123] disclose using machine learning model to classify features of detected objects into person and non-person classes.)
Agrawal et al., Bills et al. and Barton et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of the machine learning model as taught by Barton et al. for the benefit of classifying features of detected objects into person and non-person classes (Barton et al. [0087, 0123 and 0123] disclose using machine learning model to classify features of detected objects into person and non-person classes.)
With respect to Claim 25, Agrawal et al. in view of Bills et al. and Barton et al. teach
wherein the machine learning model performs automated feature recognition to identify the spatial arrangement comprising the plurality of intensity regions within the thermal signature that are characteristic of the user (Barton et al. [0087, 0123 and 0123] disclose using machine learning model to recognize and classify features of detected objects into person and non-person.)
8. Claim 28 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Malatesha et al. (US 2017/0255446 A1.)
With respect to Claim 28, Agrawal et al. in view of Bills et al. disclose all the limitations of Claim 1 upon which Claim 28 depends. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein the data indicative of the voice command comprises a packet in transit to the device, the packet comprising instructions that cause the device to execute an operation specified by the voice command.
However, Malatesha et al. teach
wherein the data indicative of the voice command comprises a packet in transit to the device, the packet comprising instructions that cause the device to execute an operation specified by the voice command (Malatesha et al. [0052] The audio packet may be one packet with metadata that contains a list of IWB appliances 102 where resulting command strings are to be sent. This way the audio packet queue 106 is not burdened with multiple audio packets if the audio packet is intended for multiple IWB appliances 102. The audio packet queue manager 204 may include instructions send the resulting command string to multiple IWB appliances 102 based on metadata included in the audio packet. This allows a user of client device 114 to control multiple IWB appliances via the interactive whiteboard app 116.)
Agrawal et al., Bills et al. and Malatesha et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of the audio packet as taught by Malatesha et al. for the benefit of sending the resulting command strings to a target IWB appliance (Malatesha et al. [0052] The audio packet may be one packet with metadata that contains a list of IWB appliances 102 where resulting command strings are to be sent. This way the audio packet queue 106 is not burdened with multiple audio packets if the audio packet is intended for multiple IWB appliances 102. The audio packet queue manager 204 may include instructions send the resulting command string to multiple IWB appliances 102 based on metadata included in the audio packet. This allows a user of client device 114 to control multiple IWB appliances via the interactive whiteboard app 116.)
9. Claim 30 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Wong et al. (US 2011/0053554 A1.)
With respect to Claim 30, Agrawal et al. in view of Bills et al. disclose all the limitations of Claim 1 upon which Claim 30 depends. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein the computing device comprises a gateway device, and wherein sending the data indicative of the voice command to the device comprises authorizing the data indicative of the voice command to be transmitted via a network to the device.
However, Wong et al. teach
wherein the computing device comprises a gateway device, and wherein sending the data indicative of the voice command to the device comprises authorizing the data indicative of the voice command to be transmitted via a network to the device (Wong et al. [0010] maintaining a record of devices participating in the session and controlling the transmission of speech data from one of the devices ("the transmitting device") participating in the communication session to the other devices ("the receiving devices") participating in the session by selectively generating a message which grants the transmitting device permission to send speech data to the receiving devices.)
Agrawal et al., Bills et al. and Wong et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of the permission as taught by Wong et al. for the benefit of granting the transmitting device permission to send speech data to the receiving devices (Wong et al. [0010] maintaining a record of devices participating in the session and controlling the transmission of speech data from one of the devices ("the transmitting device") participating in the communication session to the other devices ("the receiving devices") participating in the session by selectively generating a message which grants the transmitting device permission to send speech data to the receiving devices.)
10. Claims 8-9 are rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1).
With respect to Claim 8, Agrawal et al. disclose
A method comprising:
receiving audio data indicative of a voice command (Agrawal et al. [0013] describes receiving voice command from a user);
detecting, based on processing the audio data, a trigger word associated with a voice command (Agrawal et al. [0010] describes detecting voice password entry to unlock the device, Claim 4 determining that the user has entered a password using the user interface);
based on detection of the triggering word, determining location information associated with a source of the trigger word and a thermal signature associated with the source (Agrawal et al. [0013] describes detect the direction of the voice command and associate the thermal signature in the voice command direction with the device 100 user/owner. Therefore, if more than one person is present in proximity of the device 100, the voice command direction can be used to distinguish the user's thermal signature from thermal signatures of other persons in proximity, [0022] the voice recognition module 253 may send an indication of a valid command to the voice direction module 252 which may then communicate the voice direction information to the thermal signature detection module 251, [0034] the voice direction module 252 will determine the direction of the voice command and the thermal signature detection module 251 will attempt to obtain the thermal signature from the voice command direction. The method of operation will then proceed to decision block 507 and determine whether the thermal signature of the user is detectable in the direction of the voice command );
determining that the thermal signature is indicative of a user based on an analysis of the thermal signature (Agrawal et al. [0012] Each of four IR sensors 101 can detect the presence of a human body and take the environmental temperature versus the human body temperature into account to obtain the user’s thermal signature as referred to herein. In one example embodiment, the IR sensors 101 detect temperature within a zero to one foot distance range and can detect proximity within a one foot to eight foot range. See paragraphs [0011, 0012 and 0025]);
sending, based on the determining that the thermal signature is indicative of the user, data indicative of the voice command to a device to cause the device to execute the voice command (Agrawal et al. [0011] describes the device 100 can detect a user’s thermal signal when the user logs in or unlocks the device using an appropriate voice command. The device detects a thermal signature of a user who operates the external lockable device and send commands over the wireless interface to lock or unlock the lockable device based on the presence of the user’s thermal signature.)
Agrawal et al. disclose determining that the thermal signature is indicative of a user based on an analysis of the thermal signature in order to verify the presence of the user. However, Agrawal et al. fail to explicitly teach a spatial arrangement comprising a plurality of intensity regions within the thermal signature that are characteristic of the user.
However, Bills et al. teach
determining that the thermal signature is indicative of a user based on an analysis of the thermal signature indicating a spatial arrangement comprising a plurality of intensity regions within the thermal signature that are characteristic of the user (Bills et al. col. 6 lines 58-62 In one example, a voltage resulting from the photons collected at the infrared camera 104 during a single open exposure window 204C may be read from each pixel of the infrared camera 104 to determine the presence of the object 101 within the photon collection zone 210C, col. 17 lines 12-26 The LWIR microbolometer camera 1014 may be a thermal (e.g., infrared) camera having a sensor array configured to detect, at each of its imaging elements, thermal radiation typically associated with humans and various animals. The biological detection preprocessor 1012 may be configured to control the operation of the LWIR microbolometer camera 1014, possibly in response to commands received from the vehicle autonomy processor 1030. Additionally, the biological detection preprocessor 1012 may process the image data received from the LWIR microbolometer camera 1014 to help identify whether any particular imaged objects in the scene are human or animal in nature, as well as possibly to specifically distinguish humans from other thermal sources, such as by way of intensity, size, and/or other characteristics); and
Agrawal et al. and Bills et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources (Bills et al. col. 17 lines 12-26 The LWIR microbolometer camera 1014 may be a thermal (e.g., infrared) camera having a sensor array configured to detect, at each of its imaging elements, thermal radiation typically associated with humans and various animals. The biological detection preprocessor 1012 may be configured to control the operation of the LWIR microbolometer camera 1014, possibly in response to commands received from the vehicle autonomy processor 1030. Additionally, the biological detection preprocessor 1012 may process the image data received from the LWIR microbolometer camera 1014 to help identify whether any particular imaged objects in the scene are human or animal in nature, as well as possibly to specifically distinguish humans from other thermal sources, such as by way of intensity, size, and/or other characteristics.)
With respect to Claim 9, Agrawal et al. in view of Bills et al. disclose
wherein determining the location information comprises determining the location information based on one or more of a global positioning sensor, a mobile device, a cell phone, or a wearable device (Agrawal et al. Fig. 1 element 100.)
11. Claim 10 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Zingade et al. (US 2022/0237735 A1.)
With respect to Claim 10, Agrawal et al. in view of Bills et al. teach all the limitations of Claim 8. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein receiving the audio data comprises capturing the audio data from a plurality of devices located at a premises, wherein determining the location information associated with the source comprises triangulating, based on processing the audio data from the plurality of devices, the location information.
However, Zingade et al. teach
wherein receiving the audio data comprises capturing the audio data from a plurality of devices located at a premises, wherein determining the location information associated with the source comprises triangulating, based on processing the audio data from the plurality of devices, the location information (Zingade et al. Fig. 10 A element 1096 microphone, [0139] microphone(s) 1096, [0084] sounds originating on the left may be associated with the first conference participant 102A and sounds originating on the right may be associated with the first conference participant 102B. Further, such sound direction information may be used to improve the efficiency of the client application 134. For example, when the client application 134 detects sounds originating from the left, the client application 134 may forgo processing those image regions positioned on the right and vice versa. By way of a non-limiting example, sound direction may be determined (e.g., triangulated) using multiple microphones.)
Agrawal et al., Bills et al. and Zingade et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of triangulating the direction of sound as taught by Zingade et al. for the benefit of processing the image regions (Zingade et al. Fig. 10 A element 1096 microphone, [0139] microphone(s) 1096, [0084] sounds originating on the left may be associated with the first conference participant 102A and sounds originating on the right may be associated with the first conference participant 102B. Further, such sound direction information may be used to improve the efficiency of the client application 134. For example, when the client application 134 detects sounds originating from the left, the client application 134 may forgo processing those image regions positioned on the right and vice versa. By way of a non-limiting example, sound direction may be determined (e.g., triangulated) using multiple microphones.)
12. Claim 26 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Zhou et al. (US 2021/0393141 A1.)
With respect to Claim 26, Agrawal et al. in view of Bills et al. teach all the limitations of Claim 8 upon which Claim 26 depends. Agrawal et al. in view of Bills et al. fail to teach
wherein determining that the thermal signature is indicative of the user is further based on: the analysis of the thermal signature indicating that one or more temperature metrics match, within a threshold similarity, an expected temperature metric associated with the user, wherein the one or more temperature metrics comprise at least one of a temperature, temperature range, or an average temperature.
However, Zhou et al. teach
wherein determining that the thermal signature is indicative of the user is further based on: the analysis of the thermal signature indicating that one or more temperature metrics match, within a threshold similarity, an expected temperature metric associated with the user, wherein the one or more temperature metrics comprise at least one of a temperature (Zhou et al. Claim 12 and [0058] disclose determining that at least part of a person is in the first thermal radiation image in response to the pixels with temperatures values greater than the threshold corresponding to at least part of a person), temperature range, or an average temperature.
Agrawal et al., Bills et al. and Zhou et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of temperature threshold as taught by Zhou et al. for the benefit of a part of person is in the thermal radiation image (Zhou et al. Claim 12 and [0058] disclose determining that at least part of a person is in the first thermal radiation image in response to the pixels with temperatures values greater than the threshold corresponding to at least part of a person), temperature range, or an average temperature.
13. Claim 31 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Malatesha et al. (US 2017/0255446 A1.)
With respect to Claim 31, Agrawal et al. in view of Bills et al. disclose all the limitations of Claim 8 upon which Claim 31 depends. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein the data indicative of the voice command comprises a packet in transit to the device, the packet comprising instructions that cause the device to execute an operation specified by the voice command.
However, Malatesha et al. teach
wherein the data indicative of the voice command comprises a packet in transit to the device, the packet comprising instructions that cause the device to execute an operation specified by the voice command (Malatesha et al. [0052] The audio packet may be one packet with metadata that contains a list of IWB appliances 102 where resulting command strings are to be sent. This way the audio packet queue 106 is not burdened with multiple audio packets if the audio packet is intended for multiple IWB appliances 102. The audio packet queue manager 204 may include instructions send the resulting command string to multiple IWB appliances 102 based on metadata included in the audio packet. This allows a user of client device 114 to control multiple IWB appliances via the interactive whiteboard app 116.)
Agrawal et al., Bills et al. and Malatesha et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of the audio packet as taught by Malatesha et al. for the benefit of sending the resulting command strings to a target IWB appliance (Malatesha et al. [0052] The audio packet may be one packet with metadata that contains a list of IWB appliances 102 where resulting command strings are to be sent. This way the audio packet queue 106 is not burdened with multiple audio packets if the audio packet is intended for multiple IWB appliances 102. The audio packet queue manager 204 may include instructions send the resulting command string to multiple IWB appliances 102 based on metadata included in the audio packet. This allows a user of client device 114 to control multiple IWB appliances via the interactive whiteboard app 116.)
14. Claims 15, 17 are rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1).
With respect to Claim 15, Agrawal et al. disclose
A method comprising:
receiving, by a gateway device (Agrawal et al. Fig. 1 element 100), audio data indicative of a voice command (Agrawal et al. [0013] describes receiving voice command from a user);
determining, based on receiving the audio data, location information associated with a source of the audio data (Agrawal et al. [0013] describes detect the direction of the voice command);
causing, based on the location information, one or more devices to capture thermal data associated with the source (Agrawal et al. [0013] describes detect the direction of the voice command and associate the thermal signature in the voice command direction with the device 100 user/owner. Therefore, if more than one person is present in proximity of the device 100, the voice command direction can be used to distinguish the user's thermal signature from thermal signatures of other persons in proximity, [0022] the voice recognition module 253 may send an indication of a valid command to the voice direction module 252 which may then communicate the voice direction information to the thermal signature detection module 251, [0034] the voice direction module 252 will determine the direction of the voice command and the thermal signature detection module 251 will attempt to obtain the thermal signature from the voice command direction. The method of operation will then proceed to decision block 507 and determine whether the thermal signature of the user is detectable in the direction of the voice command);
determining that the thermal data is indicative of a person based on an analysis of the thermal data (Agrawal et al. [0012] Each of four IR sensors 101 can detect the presence of a human body and take the environmental temperature versus the human body temperature into account to obtain the user's thermal signature as referred to herein, [0013] describes detect the direction of the voice command and associate the thermal signature in the voice command direction with the device 100 user/owner. Therefore, if more than one person is present in proximity of the device 100, the voice command direction can be used to distinguish the user's thermal signature from thermal signatures of other persons in proximity. See paragraphs [0011, 0012 and 0025]);
sending, based on the thermal data being indicative of the person, data indicative of the voice command to a device to cause the device to execute the voice command (Agrawal et al. [0011] describes the device 100 can detect a user’s thermal signal when the user logs in or unlocks the device using an appropriate voice command. The device detects a thermal signature of a user who operates the external lockable device and send commands over the wireless interface to lock or unlock the lockable device based on the presence of the user’s thermal signature.)
Agrawal et al. disclose determining that the thermal signature is indicative of a user based on an analysis of the thermal signature in order to verify the presence of the user. However, Agrawal et al. fail to explicitly teach a spatial arrangement comprising a plurality of intensity regions within the thermal signature that are characteristic of the user.
However, Bills et al. teach
determining that the thermal data is indicative of a person based on an analysis of the thermal data indicating a spatial arrangement comprising a plurality of intensity regions within a thermal signature that are characteristic of the person (Bills et al. col. 6 lines 58-62 In one example, a voltage resulting from the photons collected at the infrared camera 104 during a single open exposure window 204C may be read from each pixel of the infrared camera 104 to determine the presence of the object 101 within the photon collection zone 210C, col. 17 lines 12-26 The LWIR microbolometer camera 1014 may be a thermal (e.g., infrared) camera having a sensor array configured to detect, at each of its imaging elements, thermal radiation typically associated with humans and various animals. The biological detection preprocessor 1012 may be configured to control the operation of the LWIR microbolometer camera 1014, possibly in response to commands received from the vehicle autonomy processor 1030. Additionally, the biological detection preprocessor 1012 may process the image data received from the LWIR microbolometer camera 1014 to help identify whether any particular imaged objects in the scene are human or animal in nature, as well as possibly to specifically distinguish humans from other thermal sources, such as by way of intensity, size, and/or other characteristics); and
Agrawal et al. and Bills et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources (Bills et al. col. 17 lines 12-26 The LWIR microbolometer camera 1014 may be a thermal (e.g., infrared) camera having a sensor array configured to detect, at each of its imaging elements, thermal radiation typically associated with humans and various animals. The biological detection preprocessor 1012 may be configured to control the operation of the LWIR microbolometer camera 1014, possibly in response to commands received from the vehicle autonomy processor 1030. Additionally, the biological detection preprocessor 1012 may process the image data received from the LWIR microbolometer camera 1014 to help identify whether any particular imaged objects in the scene are human or animal in nature, as well as possibly to specifically distinguish humans from other thermal sources, such as by way of intensity, size, and/or other characteristics.)
With respect to Claim 17, Agrawal et al. in view of Bills et al. teach
wherein causing the one or more devices to capture thermal data associated with the source comprises sending an instruction to emit, based on location information, an infrared signal, wherein the one or more devices receive, based on the emitted infrared signal, the thermal data and send the thermal data to the gateway device (Agrawal et al. [0009-0010, 0012, 0024, 0027] describes using a thermal infrared sensor to detect a presence of the user.)
15. Claim 16 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Zingade et al. (US 2022/0237735 A1.)
With respect to Claim 16, Agrawal et al. in view of Bills et al. disclose all the limitations of Claim 15. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein receiving the audio data comprises receiving the audio data from a plurality of devices located at a premises, wherein determining the location information associated the source comprises triangulating, based on processing the audio data from the plurality of devices, one or more of a location, a direction, a region, an area, a room, or a portion of the room.
However, Zingade et al. teach
wherein receiving the audio data comprises receiving the audio data from a plurality of devices located at a premises, wherein determining the location information associated the source comprises triangulating, based on processing the audio data from the plurality of devices, one or more of a location, a direction (Zingade et al. Fig. 10 A element 1096 microphone, [0139] microphone(s) 1096, [0084] sounds originating on the left may be associated with the first conference participant 102A and sounds originating on the right may be associated with the first conference participant 102B. Further, such sound direction information may be used to improve the efficiency of the client application 134. For example, when the client application 134 detects sounds originating from the left, the client application 134 may forgo processing those image regions positioned on the right and vice versa. By way of a non-limiting example, sound direction may be determined (e.g., triangulated) using multiple microphones), a region, an area, a room, or a portion of the room.
Agrawal et al., Bills et al. and Zingade et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of triangulating the direction of sound as taught by Zingade et al. for the benefit of processing the image regions (Zingade et al. Fig. 10 A element 1096 microphone, [0139] microphone(s) 1096, [0084] sounds originating on the left may be associated with the first conference participant 102A and sounds originating on the right may be associated with the first conference participant 102B. Further, such sound direction information may be used to improve the efficiency of the client application 134. For example, when the client application 134 detects sounds originating from the left, the client application 134 may forgo processing those image regions positioned on the right and vice versa. By way of a non-limiting example, sound direction may be determined (e.g., triangulated) using multiple microphones.)
16. Claim 27 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and George et al. (US 2018/0285650 A1.)
With respect to Claim 27, Agrawal et al. in view of Bills et al. disclose all the limitation of Claim 15 upon which Claim 27 depends. Agrawal et al. in view of Bills et al. fail to teach
wherein determining that the thermal signature is indicative of the person is further based on:
the analysis of the thermal data over a period of time, wherein the analysis indicates one or more of a change, movement, or movement pattern of a thermal signature over the period of time are characteristic of the person (George et al. [0027] detecting of one or more people entering the area by analyzing a change in thermal intensity values for a few second.)
Agrawal et al., Bills et al. and George et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of analyzing a change in the thermal intensity values over a short period of time as taught by George et al. for the benefit of detecting the human presence in the area (George et al. [0027] detecting of one or more people entering the area by analyzing a change in thermal intensity values for a few second.)
17. Claim 32 is rejected under 35 U.S.C.103 as being unpatentable over Agrawal et al. (US 2017/0024574 A1) in view of Bills et al. (US 10,656,275 B1) and Malatesha et al. (US 2017/0255446 A1.)
With respect to Claim 32, Agrawal et al. in view of Bills et al. disclose all the limitations of Claim 15 upon which Claim 32 depends. Agrawal et al. in view of Bills et al. fail to explicitly teach
wherein the data indicative of the voice command comprises a packet in transit to the device, the packet comprising instructions that cause the device to execute an operation specified by the voice command.
However, Malatesha et al. teach
wherein the data indicative of the voice command comprises a packet in transit to the device, the packet comprising instructions that cause the device to execute an operation specified by the voice command (Malatesha et al. [0052] The audio packet may be one packet with metadata that contains a list of IWB appliances 102 where resulting command strings are to be sent. This way the audio packet queue 106 is not burdened with multiple audio packets if the audio packet is intended for multiple IWB appliances 102. The audio packet queue manager 204 may include instructions send the resulting command string to multiple IWB appliances 102 based on metadata included in the audio packet. This allows a user of client device 114 to control multiple IWB appliances via the interactive whiteboard app 116.)
Agrawal et al., Bills et al. and Malatesha et al. are analogous art because they are from a similar field of endeavor in the Signal recognition algorithm and applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the steps of using the thermal signature to detect the presence of the user as taught by Agrawal et al., using teaching of intensity of thermal radiation as taught by Bills et al. for the benefit of distinguishing human from other thermal sources, using teaching of the audio packet as taught by Malatesha et al. for the benefit of sending the resulting command strings to a target IWB appliance (Malatesha et al. [0052] The audio packet may be one packet with metadata that contains a list of IWB appliances 102 where resulting command strings are to be sent. This way the audio packet queue 106 is not burdened with multiple audio packets if the audio packet is intended for multiple IWB appliances 102. The audio packet queue manager 204 may include instructions send the resulting command string to multiple IWB appliances 102 based on metadata included in the audio packet. This allows a user of client device 114 to control multiple IWB appliances via the interactive whiteboard app 116.)
Conclusion
18. The prior art made of record and not relied upon is considered pertinent to application’s disclosure. See PTO-892.
a. Nicholson et al. (US 2018/0342247 A1.) In this reference, Nicholson et al. disclose a method and/or system for activating, based on determining that the thermal data is associated with a human, at least one audio input device associated with the information handling device.
b. Laitinene et al. (US 2022/0303711 A1.) In this reference, Laitinene et al. disclose a method for using infra-red sensor to determine the at least one direction of at least sound source.
c. MacNeish et al. (US 2021/0212576 A1.) In this reference, MacNeish et al. disclose a method for detecting change of the temperature by a thermal image.
19. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
20. Any inquiry concerning this communication or earlier communications from the examiner should be directed to THUYKHANH LE whose telephone number is (571)272-6429. The examiner can normally be reached Mon-Fri: 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew C. Flanders can be reached on 571-272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THUYKHANH LE/Primary Examiner, Art Unit 2655
Prosecution Insights