DETAILED ACTION
Claim Objections
Claim 20 is objected to because of the following informalities:
Claim 20 should be corrected as “The system of claim 17, ….”;
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12, 14-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cobb (US 20110064267 A1, as provided in IDS), and in view of Millar (US 20110107270 A1, as provided in IDS).
Re Claim 1, Cobb discloses a method, comprising:
determining, via a processor, a distance between (1) an object trajectory for an object in a first data stream from the plurality of data streams that tracks the object and
and (2) previously identified trajectory information; (see Cobb: e.g., --[0036] Network 110 receives video data (e.g., video stream(s), video images, or the like) from the video input source 105. The video input source 105 may be a video camera, a VCR, DVR, DVD, computer, web-cam device, or the like. For example, the video input source 105 may be a stationary video camera aimed at a certain area (e.g., a subway station, a parking lot, a building entry/exit, etc.), which records the events taking place therein.--, in [0036]; -- The estimator/identifier component 215 may receive the output of the tracker component 210 (and the BG/FG component 205) and derive feature data for each tracked foreground object. For example, the estimator/identifier component 215 may derive a variety of micro features characterizing different aspects of a tracked foreground object, e.g., size, height, width, and area (in pixels), reflectivity, shininess rigidity, etc. Each micro feature may be represented using numerical values, e.g., a normalized value between 0 and 1 or a -1 representing a null value for a given micro feature. Additionally, the estimator/identifier component 215 may derive kinematic data describing the actions of a foreground object, e.g., a spatial position, direction of movement, velocity and acceleration, etc. Like the micro features, the kinematic data may be represented using numerical values.--, in [0042]-[0045]; and, -- a complete trajectory includes the kinematic data obtained when an object is first observed in a frame of video along with each successive observation of that object up to when it leaves the scene (or becomes stationary to the point of becoming part of the scene background).--, in [0046], and,, -- the behavioral anomaly detector 225 may evaluate the emergent trajectories to identify anomalous events--, in [0049], and [0051], and, --the ART network may specify a mapping to a "closest" cluster within ART network 625 for that input data vector (determined in the first cluster layers, e.g., using a Euclidian distance measure).--, in [0069], [0087], and [0091]);
Cobb however does not explicitly teach calculating {above mentioned} the distance measure,
Millar teaches calculating, via a processor, (1) an object trajectory for an object in a first data stream from a plurality of data streams that tracks the object and (2) previously identified trajectory information (see Millar: e.g., -- the determining whether the new trajectory is a normal trajectory comprises determining the relative distance in terms of statistical feature similarities between the new trajectory and one or more of the normal trajectories stored within the database.--, in [0009]-[0011], and [0031], -- a new trajectory appearing in the scene can be classified into a cluster (a normal scene activity) or not belonging to any cluster (an abnormal scene activity) from a viewpoint of the training dataset.. A similarity score is calculated by a weighted sum of the individual scores resulted from the trajectory's primitives--, in [0070]-[0075], and [0079]);
Cobb and Millar are combinable as they are in the same field of endeavor: analyzing moving objects using statistical and semantic features learnt from object trajectory data. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Cobb’s method using Millar’s teachings by including (1) an object trajectory for an object in a first data stream from a plurality of data streams that tracks the object and (2) previously identified trajectory information to Cobb’s determining the distance measure in order to classify a new trajectory into a cluster (a normal scene activity) or not belonging to any cluster (an abnormal scene activity) from a viewpoint of the training dataset (see Millar: e.g. in [0070]-[0075], and [0079]);
Cobb as modified by Millar further disclose upon determining, based on the distance, that the object trajectory maps to the previously identified trajectory information (see Cobb: e.g., --the ART network may specify a mapping to a "closest" cluster within ART network 625 for that input data vector (determined in the first cluster layers, e.g., using a Euclidian distance measure).--, in [0069], and [0091]; and, -- the components 205, 210, 215, and 220 may be combined (or further subdivided) to suit the needs of a particular case and further that additional components may be added (or some may be removed) from a video surveillance system--, in [0040], -- At step 910, the cluster layer determines a formal language distance for each sequence received at step 905 to each cluster in the ART network of that cluster layer, e.g., based on the formal language vector corresponding to a given segment and the equations set forth above. At step 915, the sequence may be added to an existing cluster or a new cluster may be started. That is, the input is mapped into the ART network.--, in [0087]; also see: self- organized map (SOM)”, --Formal language vectors can be subtracted and absolute values can be taken, but multiplication by a scalar is not defined. This allows clustering by ART but not by SOM--, in [0081]-[0082]):
calculate a score based on (1) a prior probability of the object trajectory mapping to the previously identified trajectory information (see Cobb: e.g., --the voting experts component 760 may determine the probability of observing the sequence itself, as well as the probability of observing the particular segments induced in the sequence by the voting experts component 760 {herein “the probability of observing the particular segments “ is the score being determined, and the frequency of observed sequences is the distribution}--, in [0051], [0069], [0079], and, -- determine the mean and standard deviation for distribution of input data and the nodes of a given SOM. In one embodiment, if the mean minus two standard deviations is equal to or greater than one, then the inputs in any node whose total number of inputs is equal to or less than this number may be flagged as an anomaly.--, [0091]-[0092]; also see Millar e.g., -- a new trajectory appearing in the scene can be classified into a cluster (a normal scene activity) or not belonging to any cluster (an abnormal scene activity) from a viewpoint of the training dataset.. A similarity score is calculated by a weighted sum of the individual scores resulted from the trajectory's primitives--, in [0070]-[0075], and [0079]) and
(2) a probability of the object trajectory being at least the distance from a mean of the previously identified trajectory information, (see Cobb: e.g., --the voting experts component 760 may determine the probability of observing the sequence itself, as well as the probability of observing the particular segments induced in the sequence by the voting experts component 760 {herein “the probability of observing the particular segments “ is the score being determined, and the frequency of observed sequences is the distribution}--, in [0051], [0069], [0079], and, --the ART network may specify a mapping to a "closest" cluster within ART network 625 for that input data vector (determined in the first cluster layers, e.g., using a Euclidian distance measure).--, in [0069], [0087], and [0091]; and, -- If the distance between the input data and the closest cluster in the ART network 625 exceeds a specified amount…. an alert specifying the occurrence of an anomalous observation may be generated.--, in [0069], and [0091]-[0092]; also see: -- if the current input (e.g., a cluster label assigned to a cluster in an ART network) is not an element of a sequence having a probability of occurring above a specified threshold (relative to prior observation), a sequence anomaly may be issued--, in [0051]; and, -- determine the mean and standard deviation for distribution of input data and the nodes of a given SOM. In one embodiment, if the mean minus two standard deviations is equal to or greater than one, then the inputs in any node whose total number of inputs is equal to or less than this number may be flagged as an anomaly.--, [0091]-[0092]; also see Millar e.g., -- a new trajectory appearing in the scene can be classified into a cluster (a normal scene activity) or not belonging to any cluster (an abnormal scene activity) from a viewpoint of the training dataset.. A similarity score is calculated by a weighted sum of the individual scores resulted from the trajectory's primitives--, in [0070]-[0075], and [0079]), and
upon determining that the score exceeds a threshold value, causing an alert to be output, the alert indicating an anomaly in the object trajectory (see Cobb: e.g., --the voting experts component 760 may determine the probability of observing the sequence itself, as well as the probability of observing the particular segments induced in the sequence by the voting experts component 760 {herein “the probability of observing the particular segments “ is the score being determined, and the frequency of observed sequences is the distribution}--, in [0051], [0069], [0079], -- If the distance between the input data and the closest cluster in the ART network 625 exceeds a specified amount…. an alert specifying the occurrence of an anomalous observation may be generated.--, in [0069], and [0091]-[0092]; also see: -- if the current input (e.g., a cluster label assigned to a cluster in an ART network) is not an element of a sequence having a probability of occurring above a specified threshold (relative to prior observation), a sequence anomaly may be issued--, in [0051]; and, -- determine the mean and standard deviation for distribution of input data and the nodes of a given SOM. In one embodiment, if the mean minus two standard deviations is equal to or greater than one, then the inputs in any node whose total number of inputs is equal to or less than this number may be flagged as an anomaly.--, [0091]-[0092]; also see Millar e.g., -- a new trajectory appearing in the scene can be classified into a cluster (a normal scene activity) or not belonging to any cluster (an abnormal scene activity) from a viewpoint of the training dataset.. A similarity score is calculated by a weighted sum of the individual scores resulted from the trajectory's primitives--, in [0070]-[0075], and [0079]).
Re Claim 2, Cobb as modified by Millar further disclose generating a new trajectory including an (x,y) position of the object upon determining that the object trajectory does not map to any stored trajectory cluster from the previously identified trajectory information, and storing the new trajectory in a trajectory buffer (see Cobb: e.g., -- The estimator/identifier component 215 may receive the output of the tracker component 210 (and the BG/FG component 205) and derive feature data for each tracked foreground object. For example, the estimator/identifier component 215 may derive a variety of micro features characterizing different aspects of a tracked foreground object, e.g., size, height, width, and area (in pixels), reflectivity, shininess rigidity, etc. Each micro feature may be represented using numerical values, e.g., a normalized value between 0 and 1 or a -1 representing a null value for a given micro feature. Additionally, the estimator/identifier component 215 may derive kinematic data describing the actions of a foreground object, e.g., a spatial position, direction of movement, velocity and acceleration, etc. Like the micro features, the kinematic data may be represented using numerical values.--, in [0042]-[0045], and, --The context events provide the cortex model component with data that can describe foreground object behavior for any given scene captured by a video camera: object, time, position, velocity, and primitive features. Further, like a mammalian visual cortex, the lower layers of the cortex model component may be divided into two sections, one for object identification and one for object location, referred to as the dorsal and ventral sections, respectively. The micro features are input to the first dorsal layer of the cortex model component and the kinematic data is input to the first ventral layer of the cortex model component. At higher levels, outputs form the dorsal and ventral sides are combined so that an entire behavior-space can be represented.--, in [0028], and [0063]).
Re Claim 3, Cobb as modified by Millar further disclose wherein the prior probability of the object trajectory mapping to the previously identified trajectory information is determined based on at least a count of previous trajectories for the object based on an object identifier associated with the plurality of data streams (see Cobb: e.g., in [0051], and [0079]; similarly see Millar: in [0070]-[0075], and [0079]).
Re Claim 4, Cobb as modified by Millar further disclose the score is calculated as S=1.0-Prxfi, Pr being the probability of the object trajectory mapping to the previously identified trajectory information, and fi being the probability of the object trajectory being at least the distance from the mean of the previously identified trajectory information (see Cobb: e.g., --the voting experts component 760 may determine the probability of observing the sequence itself, as well as the probability of observing the particular segments induced in the sequence by the voting experts component 760--, in [0051], [0069], and [0079], and, -- the cluster layer combining the dorsal and ventral sides, the input to the combining cluster layer may be the cross product of segments output from a dorsal side sequence layer and segments output from a ventral side sequence layer--, in [0083],and see Millar: in [0070]-[0075], and [0079]).
Re Claim 5, Cobb as modified by Millar further disclose the previously identified trajectory information includes an ordered list of points in a two-dimensional (2D) image-pixel space (see Cobb: e.g., -- receiving a set of data inputs derived by a computer vision engine configured to analyze pixels depicting a plurality of foreground objects in the sequence of video frames and modeling behavior of the foreground objects in the scene by passing the received sensory data inputs to a first cluster layer of a plurality of layers.--, in [0010]; [0024]-[0026], and [0036], and, -- The frame itself may include a two-dimensional array of pixel values for multiple channels (e.g., RGB channels for color video or grayscale channel or radiance channel for black and white video).--, in [0041]); and,
the distance is a distance between a list of points of the object trajectory and the ordered list of points of the previously identified trajectory information, as determined based on a dynamic programming technique (see Cobb:., -- a complete trajectory includes the kinematic data obtained when an object is first observed in a frame of video along with each successive observation of that object up to when it leaves the scene (or becomes stationary to the point of becoming part of the scene background).--, in [0046], and,, -- the behavioral anomaly detector 225 may evaluate the emergent trajectories to identify anomalous events--, in [0049], and [0051], and, --the ART network may specify a mapping to a "closest" cluster within ART network 625 for that input data vector (determined in the first cluster layers, e.g., using a Euclidian distance measure).--, in [0069]).
Re Claim 6, Cobb as modified by Millar further disclose wherein the score is a first score, the threshold value is a first threshold value, and the distance is a first distance, and the alert is a first alert, the method further comprising:
upon determining that the object trajectory does not map to the previously identified trajectory information: determining a second score based on at least a cumulative probability distribution indicating a probability of the object trajectory being at least a second distance from a mean of the previously identified trajectory information that best matches the object trajectory (see Cobb: e.g., Figs. 8-9, --the voting experts component 760 may determine the probability of observing the sequence itself, as well as the probability of observing the particular segments induced in the sequence by the voting experts component 760 --, in [0051], [0069], [0079], and, -- determine the mean and standard deviation for distribution of input data and the nodes of a given SOM. In one embodiment, if the mean minus two standard deviations is equal to or greater than one, then the inputs in any node whose total number of inputs is equal to or less than this number may be flagged as an anomaly.--, [0091]-[0092]), and
upon determining that the second score exceeds a second threshold value, causing a second alert to be output (see Cobb: e.g., --In one embodiment, if the mean minus two standard deviations is equal to or greater than one, then the inputs in any node whose total number of inputs is equal to or less than this number may be flagged as an anomaly.--, [0091]-[0092]).
Re Claim 7, Cobb as modified by Millar further disclose wherein a representation of the object trajectory includes an assembly of tracked positions of the object (see Cobb: e.g., -- the components 205, 210, 215, and 220 may be combined (or further subdivided) to suit the needs of a particular case and further that additional components may be added (or some may be removed) from a video surveillance system--, in [0040], -- At step 910, the cluster layer determines a formal language distance for each sequence received at step 905 to each cluster in the ART network of that cluster layer, e.g., based on the formal language vector corresponding to a given segment and the equations set forth above. At step 915, the sequence may be added to an existing cluster or a new cluster may be started. That is, the input is mapped into the ART network.--, in [0087]).
Re Claim 8, Cobb as modified by Millar further teaches assembling the object trajectory by: clustering raw tracked object positions to form clustered raw tracked positions (see Cobb: e.g., -- the components 205, 210, 215, and 220 may be combined (or further subdivided) to suit the needs of a particular case and further that additional components may be added (or some may be removed) from a video surveillance system--, in [0040], -- At step 910, the cluster layer determines a formal language distance for each sequence received at step 905 to each cluster in the ART network of that cluster layer, e.g., based on the formal language vector corresponding to a given segment and the equations set forth above. At step 915, the sequence may be added to an existing cluster or a new cluster may be started. That is, the input is mapped into the ART network.--, in [0087]); and
combining the clustered raw tracked positions to define the object trajectory (see Cobb: e.g., -- the components 205, 210, 215, and 220 may be combined (or further subdivided) to suit the needs of a particular case and further that additional components may be added (or some may be removed) from a video surveillance system--, in [0040], -- At step 910, the cluster layer determines a formal language distance for each sequence received at step 905 to each cluster in the ART network of that cluster layer, e.g., based on the formal language vector corresponding to a given segment and the equations set forth above. At step 915, the sequence may be added to an existing cluster or a new cluster may be started. That is, the input is mapped into the ART network.--, in [0087]).
Re Claims 9-12, 14-16, claims 9-12, and 14-16 are the corresponding storage medium claim to claims 1, 3-5, and 6-8 respectively. Claims 9-12, and 14-16 thus are rejected for the similar reasons for claims 1, 3-5, and 6-8. See above discussions with regard to claims 1, 3-5, and 6-8 respectively. Further, Cobb as modified by Millar further disclose a non-transitory computer-readable storage medium storing instructions, that when executed by a processor, cause the processor to perform the operations (see Cobb: e.g., in [0009]).
Re Claims 17, 19, claims 17, and 19 are the corresponding system claim to claims 1-2 respectively. Claims 17, and 19 thus are rejected for the similar reasons for claims 1-2. See above discussions with regard to claims 1-2 respectively. Further, Cobb as modified by Millar further disclose a system, comprising: a processor; and a memory a memory in communication with the processor, the memory storing instructions to cause the processor to perform the method (see Cobb: e.g., in [0009]).
Re Claim 20, Cobb as modified by Millar further disclose calculate the score include instructions to calculate the score further based on a maturity of a stored trajectory cluster of the previously identified trajectory information, the maturity of the previously identified trajectory cluster representing a number of training trajectories that have previously mapped to the trajectory cluster (see Cobb: e.g., in [0051], and [0079]; similarly see Millar: in [0070]-[0075], and [0079]).
Claims 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Cobb as modified by Millar, and in view of Miyake (US 20120108261 A1, as provided in IDS).
Re Claim 13, Cobb as modified by Millar teaches previously identified trajectory information includes an ordered list of points in a two-dimensional (2D) image-pixel space; and the distance is a distance between a list of points of the object trajectory and of the previously identified trajectory information, the distance determined based on a dynamic programming technique, where cells in antidiagonals of a matrix and all connected sub-matrices are computed in parallel (see Cobb: e.g., -- receiving a set of data inputs derived by a computer vision engine configured to analyze pixels depicting a plurality of foreground objects in the sequence of video frames and modeling behavior of the foreground objects in the scene by passing the received sensory data inputs to a first cluster layer of a plurality of layers.--, in [0010]; [0024]-[0026], and [0036], and, -- The frame itself may include a two-dimensional array of pixel values for multiple channels (e.g., RGB channels for color video or grayscale channel or radiance channel for black and white video).--, in [0041]; and in [0083]-[0085], and, --The machine-learning engine 140 also includes an event bus 222. In one embodiment, the components of the computer vision engine 135 and machine-learning engine 140 output data to the event bus 222. At the same time, the components of the machine-learning engine 140 may subscribe to receive different events streams from the event bus 222. For example, the cortex model component 240 may subscribe to receive the kinematic data vectors and micro feature vectors output from the computer vision engine 135 and use this information to construct progressively complex abstractions representing behavioral patterns.--, in [0048], also see Millar: --three clustering runs can occur. Each of these can involve different parameters related to the definition of distance matrices of graph edges, cluster statistics updating scheme, and/or the handling of leftover trajectories (trajectories that do not belong to any clusters). Runs 1 and 2, for example, can be designed for trajectory selection in the preprocessing stage. Run 3, for example, can be designed for normal trajectories clustering. Each run, for example, can cluster trajectories using a different distance measure.--, in [0032], [0045], [0052], [0070]-[0075], and [0079]),
Cobb as modified by Millar however do not explicitly disclose the dynamic programming technique including a Needleman-Wunsch algorithm,
Miyake teaches using the dynamic programming technique including a Needleman-Wunsch algorithm in trajectory mapping/matching, where cells in antidiagonals of a matrix used in the Needleman-Wunsch algorithm and all connected sub-matrices are computed in parallel (see Miyake: --the Needleman-Wunsch algorithm, has been applied to determine the best geographic information & transport network information sequence corresponding to the trajectory samples. The basic goal of this framework is to identify a given estimated LAC trajectory sequence from various possible geographic information & transport network information sequences, and find the best sequence match.--, in [0010]).
Cobb (as modified by Millar) and Miyake are combinable as they are in the same field of endeavor: analyzing moving objects using statistical and semantic features learnt from object trajectory data. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Cobb’s method using Miyake’s teachings by including using the dynamic programming technique of a Needleman-Wunsch algorithm in trajectory mapping/matching to Cobb’s self organizing map (SOM) algorithm to mapping the received trajectory to a stored trajectory cluster in order to determine the best geographic information & transport network information sequence corresponding to the trajectory samples (see Miyake: e.g. in [0010]).
Re Claim 18, claim 18 is the corresponding system claim to claim 13 respectively. Claim 18 thus is rejected for the similar reasons for claim 13. See above discussions with regard to claim 13 respectively. Further, Cobb as modified by Millar and Miyake disclose a system, comprising: a processor; and a memory a memory in communication with the processor, the memory storing instructions to cause the processor to perform the method (see Cobb: e.g., in [0009]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Miyake (US 20120115505 A1, as provided in IDS) teaches performing travel estimation based on the mobile phone operation data, including performing interpolation on data associated with one or more individuals in a population to estimate intermediate positions of a trajectory of each of the one or more individuals for a specified time period based on a shortest path mesh sequence estimation algorithm; and discloses utilizing the set of dominant motion trajectory characteristics to limit a total of all possible locations of the target object to a subset of candidate locations for tracking the target object; Bernal (US 9213901 B2) utilizing the set of dominant motion trajectory characteristics to refine at least one tracking parameter used to track the target object; and utilizing the set of dominant motion trajectory characteristics to smooth a trajectory of the target object derived from tracking the target object (see para. 88-90).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEI WEN YANG whose telephone number is (571)270-5670. The examiner can normally be reached on 8:00 - 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEI WEN YANG/Primary Examiner, Art Unit 2662
Prosecution Insights