Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 12, 13, 16, 17, 20 is/are rejected under 35 U.S.C. 102(a)(1) as being unpatentable by Moshfeghi US2017/0053419
Regarding claim 1, Moshfeghi teaches: acquiring, using a computer system, a set of images from a plurality of cameras, wherein: each of the plurality of cameras have a different respective field of view, and (Moshfeghi see paragraphs 0029 0030 0107, computer, group of still images with images taken at different images with multiple cameras)
at least part of the fields of view are of a monitored environment; (Moshfeghi see paragraphs 0029 0030 0107 images taken at different images with multiple cameras where area or environment being captured by camera reads on monitored environment)
detecting and localizing, using the computer system, in at least some of the set of images, a first entity moving through the monitored environment;
determining, using the computer system, a first set of locations within the monitored environment of the first entity based on locations of the first entity in the set of images, wherein each of the first set of locations is associated with an image acquisition time; (Moshfeghi see paragraphs 0024 0029 0030 0032 0074 0075, images taken from different angles by camera to identify and locate human and monitor position of object as it moves with timestamps)
acquiring, using the computer system, a set of sensor measurements of the monitored environment from a plurality of sensors, the plurality of sensors being different from the plurality of cameras; (Moshfeghi see paragraphs 0033 0034 0057 0070 0082 devices with multiple sensors such as accelerometer, gyroscope, velocity sensor, infrared light sensor, detect device position using RF signals)
determining, using the computer system, a second set of locations within the monitored environment of the first entity based on the set of sensor measurements, wherein each of the second set of locations is associated with a sensor measurement time; (Moshfeghi see paragraphs 0033 0034 0070 0082 multiple sensors such as accelerometer, gyroscope, velocity sensor to determine region of presence and location estimation at time T0. Detect position of device using RF signal including estimated time of arrival)
determining, using the computer system, whether the first set of locations should be associated with the second set of locations based on a set of confidence factors calculated based on the first set of locations and the second set of locations, the set of confidence factors being indicative of the second set of locations being locations of the first entity and not another entity; (Moshfeghi see paragraphs 0046 0047 0050 0068 device accuracy range indicating high probability of device within region and calculating a region with speed limits, using accuracy range and calculated region to associat and pair object with device)
in response to determining that the first set of locations should be associated with the second set of locations, determining, using the computer system, a sequence of locations of the first entity through the monitored environment; and (Moshfeghi see paragraphs 0074 0075 0078 device and user are paired and positioning is updated every 10 seconds in database as continuous updates in the mall)
storing, using the computer system, the sequence of locations in a computer-readable media in communication with the computer system. (Moshfeghi see paragraphs 0025 0073-0075 0099 once an object and device is paired or associated previous data within timeframe is collected and stored based on the pairing and stored in database on computer readable medium)
Regarding claim 12, Moshfeghi teaches: acquiring, using the computer system, a second set of sensor measurements of the monitored environment from a second plurality of sensors, wherein the second plurality of sensors are different from the plurality of cameras, and wherein the second plurality of sensors are different from the first plurality of sensors; (Moshfeghi see paragraphs 0070 0075 images of user or position information of device over the past hour or every ten seconds to be tracked for position within a mall such that mobile device sensors include sensors for accelerometer, gyroscope, velocity sensor)
determining, using the computer system, a third set of locations within the monitored environment of the first entity based on the second set of sensor measurements, wherein each of the third set of locations is associated with the sensor measurement time, wherein determining the sequence of locations of the first entity comprises determining the sequence of locations of the first entity based on the third set of locations. (Moshfeghi see paragraphs 0070 0075 images of user or position information of device over the past hour or every ten seconds to be tracked for position within a mall such that mobile device sensors include sensors for accelerometer, gyroscope, velocity sensor which are being used to establish location estimates)
Regarding claim 13, Moshfeghi teaches: wherein the plurality of sensors comprises electronic emission sensors, and wherein determining the sequence of locations comprises determining a location based on a time of arrival of a signal from a mobile computing device or an angle of arrival of the signal from the mobile computing device. (Moshfeghi see paragraphs 0021 0030 0070 0075 device to give off RF signals indicating location and a particular time with images taken at different angles)
Regarding claim 16, Moshfeghi teaches: determining a sound of the first entity using the ultrasonic sound sensor, wherein the sound is measured at a first time of measurement; and
determining an entity location associated with the sound at the first time of measurement, wherein the second set of locations comprises the entity location. (Moshfeghi see paragraphs 0070 0092 sensor information given at a specific initial time including ultrasound used to locate a person)
Regarding claim 17, Moshfeghi teaches: wherein a sensor of the plurality of sensors is attached to a camera of the plurality of cameras. (Moshfeghi see paragraphs 0070 0073 mobile device sensors such as accelerometer, gyroscope, velocity sensor and object or person to be paired with multiple devices, mobile device or smart phone has a camera)
Regarding claim 20, see rejection of claim 1
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 4, 6, 7 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Hinton et al. US2020/0285934
Regarding claim 2, Moshfeghi teaches: wherein a first subset of the set of images is acquired by a first camera and a second subset of the set of images is acquired by a second camera, and wherein determining the first entity comprises: (Moshfeghi see paragraphs 0030 images taken from multiple cameras)
detecting, using the computer system, the first entity based on the first subset of the set of images
determining, using the computer system, a first set of attributes associated with the first entity based on the first subset of the set of images; (Moshfeghi see paragraphs 0082 0087 using for example two of three image capture types to be combined into an IR rendered image that can or cannot be combined with other generated images)
detecting, using the computer system, a second entity based on the second subset of the set of images
determining, using the computer system, a second set of attributes associated with the second entity based on the second subset of the set of images; (Moshfeghi see paragraphs 0092-0095 using for example ultrasound to create an image that can or cannot be combined with other generated images)
Moshfeghi does not distinctly disclose: using the convolution neural network
However, Hinton teaches: using the convolution neural network (Hinton see paragraph 0029 0066 a convolution neural network analyzing image)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include convolution neural networks as taught by Hinton for the predictable result of more efficiently processing image data
Regarding claim 4, Moshfeghi as modified further teaches: wherein determining the matching entity confidence factor comprises using a first image comprising a portion of the first entity and a second image comprising a portion of the second entity as inputs for a capsule neural network. (Hinton see paragraphs 0023 0029 0066 input images depicting objects and scenes from different view points into capsule neural network determining weighted values)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include convolution neural networks as taught by Hinton for the predictable result of more efficiently processing image data
Regarding claim 6, Moshfeghi as modified further teaches: determining a boundary of the first entity with respect to a first location, wherein the first entity is detected to be at the first location at a first measurement time, and wherein the boundary is determined based on the first location and a time difference between the first measurement time and a second measurement time; and(Moshfeghi see paragraph 0032 0043 0068 images determining region of accuracy such that images come from multiple cameras and regions determined using a circle with radius around a given center based on location at T1 and T0)
wherein determining the matching entity confidence factor comprises determining the matching entity confidence factor based on a distance between the boundary and a second location, wherein the second entity is determined to be at the second location at a second measurement time, wherein the second measurement time is after the first measurement time. (Moshfeghi see paragraph 0068 0070 previous location T0 dicates previous location and T1 is constrained to a circle with a radius and outside that radius any matching point is ruled out)
Regarding claim 7, Moshfeghi as modified further teaches: determining a boundary of the first entity with respect to a first location, wherein the first entity is detected to be at the first location at a first measurement time, and wherein the boundary is determined based on the first location and a time difference between the first measurement time and a second measurement time; and
determining that a field of view of the second camera is within the boundary. (Moshfeghi see paragraph 0032 0043 0068 images determining region of accuracy such that images come from multiple cameras and regions determined using a circle with radius around a given center based on location at T1 and T0)
Claim(s) 3 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Hinton et al. US2020/0285934 in view of Ziaie et al. US2015/0196231
Regarding claim 3, Moshfeghi does not teach: the first set of attributes comprises a first gait attribute associated with the first entity, wherein determining the first set of attributes comprises determining the first gait attribute based on the first subset of the set of images, and wherein the first gait attribute comprises at least one of a movement speed, postural sway, stride frequency, gait symmetry, gait dynamic range, or gait characteristic curve; and
the second set of attributes comprises a second gait attribute associated with the second entity, wherein determining the second set of attributes comprises determining the second gait attribute based on the second subset of the set of images, wherein the second gait attribute comprises a same attribute type as the first gait attribute
However, Ziaie teaches: the first set of attributes comprises a first gait attribute associated with the first entity, wherein determining the first set of attributes comprises determining the first gait attribute based on the first subset of the set of images, and wherein the first gait attribute comprises at least one of a movement speed, postural sway, stride frequency, gait symmetry, gait dynamic range, or gait characteristic curve; and
the second set of attributes comprises a second gait attribute associated with the second entity, wherein determining the second set of attributes comprises determining the second gait attribute based on the second subset of the set of images, wherein the second gait attribute comprises a same attribute type as the first gait attribute. (Ziaie see paragraph 0008 0035 0040 0045 capturing calibration images and subsequent time varying images to analyze gait of the individual including step speed, stride length, sensor data based on waist rotation where calibrated and subsequent images read on first and second subset of images)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include analyzing gait attributes as taught by Ziaie for the predictable result of more efficiently processing image data
Claim(s) 5 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Hinton et al. US2020/0285934 in view of Georgakis et al. US2020/0334519
Regarding claim 5, Moshfeghi does not teach: wherein determining the matching entity confidence factor comprises using a siamese neural network to determine the matching entity confidence factor, wherein the siamese neural network is trained using a first image comprising a portion of the first entity, and wherein using the siamese neural network comprises using a second image comprising a portion of the second entity as inputs.
However, Georgakis teaches: wherein determining the matching entity confidence factor comprises using a siamese neural network to determine the matching entity confidence factor, wherein the siamese neural network is trained using a first image comprising a portion of the first entity, and wherein using the siamese neural network comprises using a second image comprising a portion of the second entity as inputs. (Georgakis see paragraphs 0031 0033 siamese neural network receives input pair of depth images containing features of interest in the image outputting box of values determining a positive or negative label based on proximity and calculating similarity between two inputs)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include a siamese neural network as taught by Georgakis for the predictable result of more efficiently processing image data
Claim(s) 8 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Blundell et al. US2020/0327359
Regarding claim 8, Moshfeghi does not teach: wherein detecting and localizing the first entity comprises determining an attention weight, wherein the attention weight is based on the detection of an attribute associated with the first entity, and wherein the attention weight is used by a neural network to determine the set of confidence factors.
However, Blundell teaches: wherein detecting and localizing the first entity comprises determining an attention weight, wherein the attention weight is based on the detection of an attribute associated with the first entity, and wherein the attention weight is used by a neural network to determine the set of confidence factors. (Blundell see paragraph 0006 determining attention weight for comparison by applying a neural network and generating a score representing likelihood of correct label)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include a attention weights as taught by Blundell for the predictable result of more efficiently processing and comparing data
Claim(s) 9 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Dalal et al. US2020/0026257
Regarding claim 9, Moshfeghi does not teach: segmenting a first image of the set of images into a set of grid cells;
for each respective grid cell of the set of grid cells, determining a bounding box associated with the respective grid cell using a convolution operation; and
detecting the entity based on the bounding box
However, Dalal teaches: segmenting a first image of the set of images into a set of grid cells;
for each respective grid cell of the set of grid cells, determining a bounding box associated with the respective grid cell using a convolution operation; and
detecting the entity based on the bounding box. (Dalal see paragraph 0033-0035 splitting input image into nxn grid of cells with a convolution neural network analyzing image and predicting bounding boxes)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include convolution neural network as taught by Dalal for the predictable result of more efficiently processing image data
Claim(s) 10 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Song et al. US2019/0050981
Regarding claim 10, Moshfeghi as modified further teaches: determining a set visual features in a first image of the set of images and a set of visual feature positions associated with the set of visual features; (Moshfeghi see paragraphs 0031 facial recognition or color or height or clothing detection in camera coverage)
Moshfeghi does not teach: generating a set of bounding boxes for each of the set of features, wherein each of the set of bounding boxes encloses one of the set of feature positions using a convolutional operation;
for each respective bounding box in the set of bounding boxes, determine a respective class score based on a portion of the first image bounded by the respective bounding box using a convolution operation, wherein the respective class score is associated with a first object type in a set of object types, and wherein the respective class score is indicative of a likelihood that the respective bounding box is bounding an object of the object type; and
detecting the first entity based on the set of bounding boxes and a set of class scores comprising the respective class scores.
However, Song teaches: generating a set of bounding boxes for each of the set of features, wherein each of the set of bounding boxes encloses one of the set of feature positions using a convolutional operation;
for each respective bounding box in the set of bounding boxes, determine a respective class score based on a portion of the first image bounded by the respective bounding box using a convolution operation, wherein the respective class score is associated with a first object type in a set of object types, and wherein the respective class score is indicative of a likelihood that the respective bounding box is bounding an object of the object type; and
detecting the first entity based on the set of bounding boxes and a set of class scores comprising the respective class scores. (Song see paragraph 0027 0030 0032 in convolution neural network a grid cell is used to indicate presence of object in image using bonding boxes and scores in those bounding boxes such that score represents presence of object in box and each grid cell represents a patch of a 3d image)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include convolution neural network using bounding boxes as taught by Song for the predictable result of more efficiently processing image data
Claim(s) 11 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Farnham US2019/0158982
Regarding claim 11, Moshfeghi does not teach: wherein determining the sequence of locations of the first entity through the monitored environment the operations further comprises applying a Kalman filter to determine the sequence of locations based on the second set of locations.
However, Farnham teaches: wherein determining the sequence of locations of the first entity through the monitored environment the operations further comprises applying a Kalman filter to determine the sequence of locations based on the second set of locations.
(Farnham see paragraphs 0045-0055 determining sequence of locations of entity by applying Kalman filter using angle of arrival of a signal from a mobile device to track position of device)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include a Kalman filter as taught by Farnham for the predictable result of more efficiently determining a sequence of data and locations
Claim(s) 14 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Ng US2013/0253877
Regarding claim 11, Moshfeghi does not teach: wherein the plurality of sensors comprises a temperature sensor, and wherein the operations further comprise:
determining an entity temperature of the first entity using the temperature sensor, wherein the entity temperature is measured at a first time of measurement; and
determining an entity location associated with the entity temperature at the first time of measurement, wherein the second set of locations comprises the entity location.
However, Ng teaches: wherein the plurality of sensors comprises a temperature sensor, and wherein the operations further comprise:
determining an entity temperature of the first entity using the temperature sensor, wherein the entity temperature is measured at a first time of measurement; and
determining an entity location associated with the entity temperature at the first time of measurement, wherein the second set of locations comprises the entity location. (Ng see paragraph 0005 0006 determining location information by sensing temperature of an object or person at a first measurement and determining a relationship to whether person or object has moved for second measurement period)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include a temperature data as taught by Ng for the predictable result of more efficiently determining a sequence of data and locations
Claim(s) 15 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Fales US2014/0100793
Regarding claim 15, Moshfeghi does not teach: wherein the plurality of sensors comprises a chemical sensor, and wherein the operations further comprise:
determining a volatile chemical signature of the first entity using the chemical sensor, wherein the volatile chemical signature is measured at a first time of measurement; and
determining an entity location associated with the volatile chemical signature at the first time of measurement, wherein the second set of locations comprises the entity location
However, Fales teaches: wherein the plurality of sensors comprises a chemical sensor, and wherein the operations further comprise:
determining a volatile chemical signature of the first entity using the chemical sensor, wherein the volatile chemical signature is measured at a first time of measurement; and
determining an entity location associated with the volatile chemical signature at the first time of measurement, wherein the second set of locations comprises the entity location. (Fales see paragraph 0026 determining location of mobile device whether it is general or specific location based on chemical signature)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include a chemical signatures as taught by Fales for the predictable result of more efficiently determining a sequence of data and locations
Claim(s) 18 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Feifel et al. US2020/0066157
Regarding claim 18, Moshfeghi teaches: determining, using the computer system, whether a location in the sequence of locations is in a restricted area of the monitored environment; and (Moshfeghi see paragraphs 0075 pairing device and human to monitor area within a mall)
Moshfeghi does not teach: in response to a determination that the location is in the restricted area of the monitored environment, display a warning to a graphical display device.
However, Feifel teaches: in response to a determination that the location is in the restricted area of the monitored environment, display a warning to a graphical display device. (Feifel see paragraph 0037 outputting warning signal on display device around relevant danger area)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include a warning signal as taught by Feifel for the predictable result of more efficiently performing actions upon determining location
Claim(s) 19 are/is rejected under 35 U.S.C. 103 as being unpatentable over Moshfeghi US2017/0053419 in view of Kerning et al. US2017/0148241
Regarding claim 19, Moshfeghi does not teach: determining, using the computer system, whether a location in the sequence of locations is outside a permitted area of the monitored environment; and
in response to a determination that the location is outside the permitted area of the monitored environment, display a warning to a graphical display device.
However, Kerning teaches: determining, using the computer system, whether a location in the sequence of locations is outside a permitted area of the monitored environment; and
in response to a determination that the location is outside the permitted area of the monitored environment, display a warning to a graphical display device. (Kerning see fig 5. Paragraph 0029 0096 0219 on screen map with pre-defined restricted zones, figure shows a crime data alert based on location with a 10 mile radius shown on map)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a multi-camera based positioning as taught by Moshfeghi to include alerts on a map as taught by Kerning for the predictable result of more efficiently performing actions upon determining location
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALLEN S LIN whose telephone number is (571)270-0612. The examiner can normally be reached on M-F 9-5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kavita Stanley can be reached on (571)272-8352. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALLEN S LIN/Primary Examiner, Art Unit 2153
Prosecution Insights