DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 26 November 2025 have been fully considered but they are not persuasive.
The Applicant argues that Oami in view of Scalisi et al. fails to disclose or suggest two events as claimed (e.g., “a second event based on a first time regarding the first event in a case where the first event is detected”), and controlling playback of “a sound as a first tone based on a time when the second event occurred, wherein the second event includes an event that continues for more than a certain period of time.” The Examiner respectfully disagrees. Oami discloses in Figs. 10 and 11 an example of Abandonment and Carry Away events. Abandonment is basically sensed when the rest time of an object has exceeded a predetermined threshold, as in images 1002 and 1003 as shown in Fig. 10. That is, to issue an alert upon sensing abandonment, a value equal to or more than the time until an abandoned object is determined as time difference information, and a person near the object as in an image 1001 is searched for. The features of the person who has abandoned the object are extracted. Carry away is sensed upon finding a person has carried away an object that has remained immovable for a predetermined time, as in images 1102 and 1103 show in Fig. 11. The reliability of carry away determination may be raised by collating the person in the image 1101 with the person in the image 1103 (paragraphs [0098]-[0105]) . The first event is detected and then based on the table provided in Fig. 7, the system goes back in time to look for a second event to verify that an actual security event has occurred. For example, to identify the person who would have abandoned the item, that person would have needed to be seen with the item and then walking away from the item, which would be considered the second event. Once that information is acquired a security alert can be issued. Scalisi et al. is then relied upon to determine what kind of alert is to be issued (paragraphs [0131], [0142], and [0150]). Therefore, the combination of the Oami and Scalisi et al. references are a reasonable combination and disclose the described claim limitations, and the rejection is maintained.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 10-11, 26-27, 29-30, 32-33, and 35-37 are rejected under 35 U.S.C. 103 as being unpatentable over Oami (U.S. Patent Application Publication 2015/0254514) in view of Scalisi et al. (U.S. Patent Application Publication 2017/0085844).
Regarding claim 1, Oami discloses a monitoring device comprising: a memory storing instructions (Figs. 2 and 6; paragraph [0055] – the information processing apparatus 200 includes a CPU (Central Processing Unit), a storage, a RAM (Random Access Memory), and a communication controller as hardware components – the storage stores program modules); and at least one processor (Fig. 2; paragraph [0055] – the information processing apparatus 200 includes a CPU (Central Processing Unit), a storage, a RAM (Random Access Memory), and a communication controller as hardware components) configured to execute the instructions to perform processing comprising: detecting a first event from video data captured by an image capturing device (Figs. 2 and 6; paragraph [0038] – the information processing apparatus 200 automatically detects an event that occurs in an airport lounge by processing a video obtained by causing a surveillance camera 250 to capture people, benches, and the like in the airport lounge, and alerts a security guard or the like; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0040] – the alert targets include events that should attract attention of a guard, for example, appearance of a person included in a watch list (so-called black list), suspicious behavior, and an dangerous behavior; paragraph [0041] – the event sensor 201 also generates an alert upon sensing a specific behavior such a fall or a crouch; paragraph [0042] – the event sensor 201 also generates an alert when a person has entered a restricted area; paragraph [0043] – the event sensor 201 also generates an alert upon sensing an abandoned object or theft of an object left behind); determining a time for detecting a second event based on a first time regarding the first event in a case where the first event is detected (Figs. 2 and 6; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location); and outputting an alert in a case where the second event is detected within a period between the first time and a second time (Figs. 2, 6, and 7; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location), wherein the second time is determined by the time for detecting the second event, and wherein the second event includes an event that continues for more than a certain period of time (Figs. 2, 6, 7, 10, and 11; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location; paragraphs [0098]-[0105] – Abandoment/Carry Away - time difference – abandonment is basically sensed when the rest time of an object has exceeded a predetermined threshold, as in images 1002 and 1003 as shown in Fig. 10 – that is, to issue an alert upon sensing abandonment, a value equal to or more than the time until an abandoned object is determined as time difference information, and a person near the object as in an image 1001 is searched for – the features of the person who has abandoned the object are extracted – carry away is sensed upon finding a person has carried away an object that has remained immovable for a predetermined time, as in images 1102 and 1103 show in Fig. 11 – the reliability of carry away determination may be raised by collating the person in the image 1101 with the person in the image 1103; the first event is detected and then based on the table provided in Fig. 7, the system goes back in time to look for a second event to verify that an actual security event has occurred, for example, to identify the person who would have abandoned the item, that person would have needed to be seen with the item and then walking away from the item). However, Oami fails to disclose wherein the alert is configured to control playback of a sound as a first tone based on a time when the second event occurred.
Referring to the Scalisi et al. reference, Scalisi et al. discloses a monitoring device comprising: outputting an alert when an event is detected, wherein the alert is configured to control playback of a sound as a first tone based on a time when the event occurred (paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have had the alert be configured to control playback of a sound as a first tone based on a time when the event occurred as disclosed by Scalisi et al. in the device disclosed by Oami in order to provide an alert that can penetrate through background noise and reach people who might be distracted or unaware of visual alerts.
Regarding claim 10, Oami discloses a monitoring method performed by a monitoring device and comprising: detecting a first event from video data captured by an image capturing device (Figs. 2 and 6; paragraph [0038] – the information processing apparatus 200 automatically detects an event that occurs in an airport lounge by processing a video obtained by causing a surveillance camera 250 to capture people, benches, and the like in the airport lounge, and alerts a security guard or the like; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0040] – the alert targets include events that should attract attention of a guard, for example, appearance of a person included in a watch list (so-called black list), suspicious behavior, and an dangerous behavior; paragraph [0041] – the event sensor 201 also generates an alert upon sensing a specific behavior such a fall or a crouch; paragraph [0042] – the event sensor 201 also generates an alert when a person has entered a restricted area; paragraph [0043] – the event sensor 201 also generates an alert upon sensing an abandoned object or theft of an object left behind); determining a time for detecting a second event based on a first time regarding the first event in a case where the first event is detected (Figs. 2 and 6; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location); and outputting an alert in a case where the second event is detected within a period between the first time and a second time (Figs. 2, 6, and 7; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location), wherein the second time is determined by the time for detecting the second event, and wherein the second event includes an event that continues for more than a certain period of time (Figs. 2, 6, 7, 10, and 11; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location; paragraphs [0098]-[0105] – Abandoment/Carry Away - time difference – abandonment is basically sensed when the rest time of an object has exceeded a predetermined threshold, as in images 1002 and 1003 as shown in Fig. 10 – that is, to issue an alert upon sensing abandonment, a value equal to or more than the time until an abandoned object is determined as time difference information, and a person near the object as in an image 1001 is searched for – the features of the person who has abandoned the object are extracted – carry away is sensed upon finding a person has carried away an object that has remained immovable for a predetermined time, as in images 1102 and 1103 show in Fig. 11 – the reliability of carry away determination may be raised by collating the person in the image 1101 with the person in the image 1103; the first event is detected and then based on the table provided in Fig. 7, the system goes back in time to look for a second event to verify that an actual security event has occurred, for example, to identify the person who would have abandoned the item, that person would have needed to be seen with the item and then walking away from the item). However, Oami fails to disclose wherein the alert is configured to control playback of a sound as a first tone based on a time when the second event occurred.
Referring to the Scalisi et al. reference, Scalisi et al. discloses a monitoring method performed by a monitoring device and comprising: outputting an alert when an event is detected, wherein the alert is configured to control playback of a sound as a first tone based on a time when the event occurred (paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have had the alert be configured to control playback of a sound as a first tone based on a time when the event occurred as disclosed by Scalisi et al. in the method disclosed by Oami in order to provide an alert that can penetrate through background noise and reach people who might be distracted or unaware of visual alerts.
Regarding claim 11, Oami discloses a monitoring system comprising at least one memory storing instructions and at least one processor configured to execute instructions to: detecting a first event from video data captured by an image capturing device (Figs. 2 and 6; paragraph [0038] – the information processing apparatus 200 automatically detects an event that occurs in an airport lounge by processing a video obtained by causing a surveillance camera 250 to capture people, benches, and the like in the airport lounge, and alerts a security guard or the like; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0040] – the alert targets include events that should attract attention of a guard, for example, appearance of a person included in a watch list (so-called black list), suspicious behavior, and an dangerous behavior; paragraph [0041] – the event sensor 201 also generates an alert upon sensing a specific behavior such a fall or a crouch; paragraph [0042] – the event sensor 201 also generates an alert when a person has entered a restricted area; paragraph [0043] – the event sensor 201 also generates an alert upon sensing an abandoned object or theft of an object left behind); determine a time for detecting a second event based on a first time regarding the first event in a case where the first event is detected (Figs. 2 and 6; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location); and output an alert in a case where the second event is detected within a period between the first time and a second time (Figs. 2, 6, and 7; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location), wherein the second time is determined by the time for detecting the second event, and wherein the second event includes an event that continues for more than a certain period of time (Figs. 2, 6, 7, 10, and 11; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location; paragraphs [0098]-[0105] – Abandoment/Carry Away - time difference – abandonment is basically sensed when the rest time of an object has exceeded a predetermined threshold, as in images 1002 and 1003 as shown in Fig. 10 – that is, to issue an alert upon sensing abandonment, a value equal to or more than the time until an abandoned object is determined as time difference information, and a person near the object as in an image 1001 is searched for – the features of the person who has abandoned the object are extracted – carry away is sensed upon finding a person has carried away an object that has remained immovable for a predetermined time, as in images 1102 and 1103 show in Fig. 11 – the reliability of carry away determination may be raised by collating the person in the image 1101 with the person in the image 1103; the first event is detected and then based on the table provided in Fig. 7, the system goes back in time to look for a second event to verify that an actual security event has occurred, for example, to identify the person who would have abandoned the item, that person would have needed to be seen with the item and then walking away from the item). However, Oami fails to disclose wherein the alert is configured to control playback of a sound as a first tone based on a time when the second event occurred.
Referring to the Scalisi et al. reference, Scalisi et al. discloses a monitoring system comprising at least one memory storing instructions and at least one processor configured to execute instructions to: output an alert when an event is detected, wherein the alert is configured to control playback of a sound as a first tone based on a time when the event occurred (paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have had the alert be configured to control playback of a sound as a first tone based on a time when the event occurred as disclosed by Scalisi et al. in the system disclosed by Oami in order to provide an alert that can penetrate through background noise and reach people who might be distracted or unaware of visual alerts.
Regarding claim 26, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claim 1 including that wherein the alert is configured to control playback of a sound as a second tone in a case where the second event occurred at night, the second tone being different from the first tone (Scalisi et al.: paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Regarding claim 27, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claims 1 and 26 including that wherein the second tone is larger volume than the first tone (Scalisi et al.: paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Regarding claim 29, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claim 10 including that wherein the alert is configured to control playback of a sound as a second tone in a case where the second event occurred at night, the second tone being different from the first tone (Scalisi et al.: paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Regarding claim 30, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claims 10 and 29 including that wherein the second tone is larger volume than the first tone (Scalisi et al.: paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Regarding claim 32, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claim 11 including that wherein the alert is configured to control playback of a sound as a second tone in a case where the second event occurred at night, the second tone being different from the first tone (Scalisi et al.: paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Regarding claim 33, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claims 11 and 32 including that wherein the second tone is larger volume than the first tone (Scalisi et al.: paragraph [0131] – the speaker 404 of the alert communication system 404 can be configured to emit a sound in response to the initiation of the event – the speaker 404 can be configured to emit a wide-range of sounds and various decibel levels depending on the application – for example, a more severe alert can be louder than a less severe alert; paragraph [0142] – in some examples, the alert communication system 402 can emit a loud sound from the speaker 404 – for example, the loud sound can be alternating high and low pitch sounds that may be heard by individuals located within 300 feet of the home – the high and low pitch sounds can be of high and low frequencies that may be heard by individuals located within a neighboring home and/or a vehicle passing by – the loud sound can be a unique sound easily detectable by individuals – as well, the sound can comprise a noise that indicates danger or help is required; paragraph [0150] – in block 516, the security system 202 can determine the alarm type – for example, a non-urgent alert (such as elevated radon levels) may not warrant a high-pitched, high-volume noise in the middle of the night – the security system 202 can balance the severity of the alert with the certainty for the alert).
Regarding claim 35, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claim 1 including that wherein the at least one processor is configured to execute the instructions to perform processing comprising: detecting the second event from the video data (Oami: Figs. 2, 6, 7, 10, and 11; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location; paragraphs [0098]-[0105] – Abandoment/Carry Away - time difference – abandonment is basically sensed when the rest time of an object has exceeded a predetermined threshold, as in images 1002 and 1003 as shown in Fig. 10 – that is, to issue an alert upon sensing abandonment, a value equal to or more than the time until an abandoned object is determined as time difference information, and a person near the object as in an image 1001 is searched for – the features of the person who has abandoned the object are extracted – carry away is sensed upon finding a person has carried away an object that has remained immovable for a predetermined time, as in images 1102 and 1103 show in Fig. 11 – the reliability of carry away determination may be raised by collating the person in the image 1101 with the person in the image 1103; the first event is detected and then based on the table provided in Fig. 7, the system goes back in time to look for a second event to verify that an actual security event has occurred, for example, to identify the person who would have abandoned the item, that person would have needed to be seen with the item and then walking away from the item).
Regarding claim 36, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claim 10 including that wherein the monitoring method comprises detecting the second event from the video data (Oami: Figs. 2, 6, 7, 10, and 11; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location; paragraphs [0098]-[0105] – Abandoment/Carry Away - time difference – abandonment is basically sensed when the rest time of an object has exceeded a predetermined threshold, as in images 1002 and 1003 as shown in Fig. 10 – that is, to issue an alert upon sensing abandonment, a value equal to or more than the time until an abandoned object is determined as time difference information, and a person near the object as in an image 1001 is searched for – the features of the person who has abandoned the object are extracted – carry away is sensed upon finding a person has carried away an object that has remained immovable for a predetermined time, as in images 1102 and 1103 show in Fig. 11 – the reliability of carry away determination may be raised by collating the person in the image 1101 with the person in the image 1103; the first event is detected and then based on the table provided in Fig. 7, the system goes back in time to look for a second event to verify that an actual security event has occurred, for example, to identify the person who would have abandoned the item, that person would have needed to be seen with the item and then walking away from the item).
Regarding claim 37, Oami in view of Scalisi et al. discloses all of the limitations as previously discussed with respect to claim 11 including that wherein the at least one processor is configured to execute instructions to: detecting the second event from the video data (Oami: Figs. 2, 6, 7, 10, and 11; paragraph [0039] – the video captured by the surveillance camera 250 is stored in a video storage 210 and also sent to an event sensor 201 – the event sensor 201 analyzes the acquired video and confirms, by referring to an event database 220, whether an event as an alert target has not occurred – if an event as an alert target has occurred, alert information is generated and sent to an object type determining unit 202 – the alert information includes the time at which the alert is generated, the alert type, and information representing the position of the object as the alert target; paragraph [0053] – this makes it possible to alert a guard 262 who is watching a monitor 261 in a monitor room 260 and also offer detailed information associated with the alert; paragraph [0054] – it is possible to present the guard 262 the past location of an object by searching for the same object in videos captured by other cameras up to the time immediately before the alert generation time; paragraph [0074] – Fig. 6 is a block diagram for explaining the arrangement and use method of a search query generator – the search query generator 501 includes a time difference determining unit 613 and an object detector/tracker 611 – the rest of the arrangement and operation is the same as in the second embodiment; paragraph [0075] – the time difference determining unit 613 determines feature extraction time information serving as the reference of query feature extraction from alert information and outputs it to the object detector/tracker 611; paragraph [0076] – the alert information is input to the time difference determining unit 613 – based on the event type information included in the alert information, the time difference determining unit 613 determines time difference information that designates how long the time goes back from the alert generation time to use video data as a query – this time difference changes depending on the event type and is therefore determined based on an event table 700 set as shown in Fig. 7 – the event table 700 holds time difference information is selected in accordance with input alert information; paragraph [0077] – in addition to the time difference information, the alert information and the video are also input to the object detector/tracker 611; paragraph [0080] – in this embodiment, when an alert is generated, an image suitable for a search is selected based on tracking information at that time and alert type information, and the search is automatically executed - as a possible use, for example, when a suspicious person alert is generated, the video of a camera located near the camera that has issued the alert is searched to track the person; paragraph [0081] – when a person is tracked before and after the video designated by the guard, and the search is performed by designating features supposed to be most reliable in the video, the search accuracy is expected to be higher than when performing the search using only images designated by the guard; paragraphs [0082]-[0123] – Fig. 7 is a view showing the event table 700 used to set an object type, a predetermined time difference, a predetermined time width, and the number of object search images in accordance with an event type; the second event is the information presented to the guard regarding the object’s past or current location; paragraphs [0098]-[0105] – Abandoment/Carry Away - time difference – abandonment is basically sensed when the rest time of an object has exceeded a predetermined threshold, as in images 1002 and 1003 as shown in Fig. 10 – that is, to issue an alert upon sensing abandonment, a value equal to or more than the time until an abandoned object is determined as time difference information, and a person near the object as in an image 1001 is searched for – the features of the person who has abandoned the object are extracted – carry away is sensed upon finding a person has carried away an object that has remained immovable for a predetermined time, as in images 1102 and 1103 show in Fig. 11 – the reliability of carry away determination may be raised by collating the person in the image 1101 with the person in the image 1103; the first event is detected and then based on the table provided in Fig. 7, the system goes back in time to look for a second event to verify that an actual security event has occurred, for example, to identify the person who would have abandoned the item, that person would have needed to be seen with the item and then walking away from the item).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATHER R JONES whose telephone number is (571)272-7368. The examiner can normally be reached Mon. - Fri.: 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached on (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HEATHER R JONES/Primary Examiner, Art Unit 2481
March 24, 2026
Prosecution Insights