DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/28/2025 has been entered.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 12 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The following claims 1 and 12 elements were not described in the specification “by a neural network from each frame of each image as captured by each camera” and “each frame of each image”;
“extract metadata associated with a person and metadata associated with an object as positioned at each area of interest of the monitored location from the plurality of images captured of the person and the object by each corresponding camera to generate by a neural network from each frame of each image as captured by each camera a corresponding feature vector of the person for each frame of each image and a corresponding feature vector of the object for each frame of each image as the person and the object are positioned in each area of interest as captured in each frame of each image by each camera, wherein each corresponding feature vector of the person is indicative as to an identification of the person as positioned in each area of interest and each corresponding feature vector of the object is indicative as to an identification of the object as positioned in each area of interest;”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The following claims 1 and 12 elements are vague and indefinite “by a neural network from each frame of each image as captured by each camera” and “each frame of each image”;
“extract metadata associated with a person and metadata associated with an object as positioned at each area of interest of the monitored location from the plurality of images captured of the person and the object by each corresponding camera to generate by a neural network from each frame of each image as captured by each camera a corresponding feature vector of the person for each frame of each image and a corresponding feature vector of the object for each frame of each image as the person and the object are positioned in each area of interest as captured in each frame of each image by each camera, wherein each corresponding feature vector of the person is indicative as to an identification of the person as positioned in each area of interest and each corresponding feature vector of the object is indicative as to an identification of the object as positioned in each area of interest;”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 4, 12, 14, 15 are rejected under 35 U.S.C. 103 as being unpatentable over Woolard et al. (US 2024/0212027) in view of Toni et al. (US 2023/0088414).
Regarding claim 1, Woolard teaches a visual analytics system for tracking and identifying people and objects within a monitored location based on metadata associated with each person and metadata associated with each object as provided by a plurality of images captured by a plurality of cameras positioned at the monitored location, the visual analytics system comprising:
an edge server positioned at the monitored location and communicatively coupled to the plurality of cameras over an internal communications network at the monitored location, the edge server comprising a processor executing a plurality of instructions stored in memory (see figure 36B, para. 0195-0197, Woolard discusses cameras connected in a communication network), wherein the plurality of instructions cause the processor of the edge server to:
extract metadata associated with a person and metadata associated with an object as positioned at each area of interest of the monitored location from the plurality of images captured of the person and the object by each corresponding camera to generate by a neural network from each frame of each image as captured by each camera a corresponding feature vector of the person for each frame of each image and a corresponding feature vector of the object for each frame of each image as the person and the object are positioned in each area of interest as captured in each frame of each image by each camera, wherein each corresponding feature vector of the person is indicative as to an identification of the person as positioned in each area of interest and each corresponding feature vector of the object is indicative as to an identification of the object as positioned in each area of interest (see claim 5, para. 0588, Woolard discusses metadata associated with objects; see para. 0108, 0346, 0496-0497, Woolard discusses joint and vectors based on object features generated by neural networks);
analyze whether the feature vector of the person as captured of the person from each frame by each camera as generated by the neural network matches each subsequent feature vector of the person as captured from each subsequent frame by each camera of the person within each area of interest as generated by the neural network and whether the feature vector of the object captured of the person from each frame by each camera as generated by the neural network matches each subsequent feature vector of the object as captured from each subsequent frame by each camera of the object as generated by the neural network within each area of interest to detect a presence of the person and a presence of the object within each area of interest (see para. 0108, 0346, Woolard discusses neural network generating vectors; see para. 0583, Woolard discusses subject match with re-identification feature vectors for the first subject, and assigning the first tracking identifier to the second subject and tracking the second subject as the first subject in the second area of real space);
identify the person and the object when the feature vector as generated by the neural network associated with the person and the feature vector as generated by the neural network associated with the object as positioned in each area of interest as captured of the person and the object by each frame by each camera matches each subsequent feature vector as generated by the neural network of the person and each subsequent feature vector as generated by the neural network of the object as captured of the person and the object by each subsequent frame by each camera as the person and the object are positioned in each area of interest (see para. 0108, 0346, Woolard discusses neural network generating vectors; see para. 0243, 0583, Woolard discusses subject match with re-identification feature vectors for the first subject, and assigning the first tracking identifier to the second subject and tracking the second subject as the first subject in the second area of real space), wherein the person and the object are re-identified in each subsequent frame by each camera based on the corresponding subsequent feature vector generated by the neural network from each subsequent frame thereby tracking the person and the object as captured by each frame and each subsequent frame captured by each camera of each area of interest (see para. 0182, 0191, 0204, Woolard discusses subject re-identification engine using multiple camera nodes, extracting feature vectors for subjects, and assigning tracking identifiers, and tracking matched second subject as the first subject in other spaces).
Toni teaches determine that the person identified within each area of interest is interacting with the object identified within each area of interest as the person is identified in each frame and each subsequent frame captured by each camera of each area of interest and the object is identified in each frame and each subsequent frame captured by each camera of each area of interest based on re-identifying each feature vector of the person and each feature vector of the object as generated by the neural network from each frame and each subsequent frame captured by each camera of the person and the object in each area of interest (see para. 0079, Toni discusses tracking subjects are linked with items picked up on the store); and
transmit the metadata associated with the person and the metadata associated with the object as extracted from the plurality of images captured of the person and the object when the person is identified as interacting with the object in each area of interest to a remote processing server using an external communications network to train the neural network to re-identify the object (see para. 0079, Toni discusses tracking subjects are linked with items picked up on the store; see claim 18, Toni discusses trained machine learning model to produce respective third and fourth reidentification feature vectors).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard with Toni to derive at the invention of claim 1. The result would have been expected, routine, and predictable in order to perform person and object tracking.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Woolard in this manner in order to improve person and object tracking by analyzing extracted feature vectors using a neural network from a plurality of cameras, thereby properly viewing and tracking objects Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Woolard, while the teaching of Toni continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of acquiring multiple image data from different cameras to properly track objects across a large area. The Woolard and Toni systems perform object detection and tracking, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 3, Woolard teaches wherein the one or more artificial intelligence models comprise a machine learning model (see figure 36B, figure 40, Woolard discusses for each image data captured by different cameras, implementing a convolutional neural network to extract metadata of detected persons).
The same motivation of claim 1 is applied to claim 3. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard with Toni to derive at the invention of claim 3. The result would have been expected, routine, and predictable in order to perform person and object tracking.
Regarding claim 4, Woolard teaches wherein the machine learning model is a neural network (see figure 36B, figure 40, Woolard discusses for each image data captured by different cameras, implementing a convolutional neural network to extract metadata of detected persons).
The same motivation of claim 1 is applied to claim 4. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard with Toni to derive at the invention of claim 4. The result would have been expected, routine, and predictable in order to perform person and object tracking.
Claim 12 is rejected as applied to claim 1 as pertaining to a corresponding method.
Claim 14 is rejected as applied to claim 3 as pertaining to a corresponding method.
Claim 15 is rejected as applied to claim 4 as pertaining to a corresponding method.
Claims 2, 7, 9-11, 13, 18, 20-22 are rejected under 35 U.S.C. 103 as being unpatentable over Woolard et al. (US 2024/0212027) in view of Toni et al. (US 2023/0088414) in view of Othman et al. (US 11,024,043).
Regarding claim 2, Woolard and Toni do not expressly disclose wherein to infer that the person moved from the first area of interest to the second area of interest as a function of first metadata generated in response to detection of the presence of the person within the first area of interest and second metadata generated in response to detection of the presence of the person within the second area of interest comprises to analyze the first metadata and the second metadata with the one or more artificial intelligence models.
However, Othman teaches wherein to infer that the person moved from the first area of interest to the second area of interest as a function of first metadata generated in response to detection of the presence of the person within the first area of interest and second metadata generated in response to detection of the presence of the person within the second area of interest comprises to analyze the first metadata and the second metadata with the one or more artificial intelligence models (see figure 3A, col. 6 line 29-col. 7 line 24, Othman discusses for each image data captured by different cameras, implementing a convolutional neural network to extract metadata of detected persons).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard and Toni with Othman to derive at the invention of claim 2. The result would have been expected, routine, and predictable in order to perform person and object tracking.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Woolard and Toni in this manner in order to improve person and object tracking by analyzing multiple metadata indicative of individual features captured from a plurality of cameras, thereby properly viewing each region and preventing uncaptured regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Woolard and Toni, while the teaching of Othman continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of acquiring multiple image data from different cameras to properly track objects across a large area. The Woolard, Toni, and Othman systems perform object detection and tracking, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 7, Woolard and Toni do not expressly disclose wherein the remote processing server comprises a processor executing a plurality of instructions stored in memory, wherein the plurality of instructions cause the processor of the remote processing server to: receive the first metadata, the second metadata, and third metadata generated in response to detection of the presence within the second area of interest from the edge server; analyze the first metadata, the second metadata, and the third metadata with the one or more artificial intelligence models to predict a future event at the monitored location; generate an alert indicative of the predicted future the event at the monitored location. However, Othman teaches wherein the remote processing server comprises a processor executing a plurality of instructions stored in memory, wherein the plurality of instructions cause the processor of the remote processing server to: receive the first metadata, the second metadata, and third metadata generated in response to detection of the presence within the second area of interest from the edge server (see col. 13 lines 4-50, Othman discusses receiving each person’s feature vector data);
analyze the first metadata, the second metadata, and the third metadata with the one or more artificial intelligence models to predict a future event at the monitored location (see col. 13 lines 4-50, Othman discusses the trained neural network used to predict motion and generate action recommendations); and
generate an alert indicative of the predicted future the event at the monitored location (see col. 5 lines 61-67, Othman discusses that once a detection has been identified, the visual processing unit performs the motion prediction by determining motion data for each detection comprising position, velocity, and acceleration. For example, the position, velocity, and acceleration of the detection may be measured relative to x and y coordinates corresponding to the pixels which constitute each video frame; see col. 13 lines 4-50, Othman discusses the trained neural network used to predict motion and generate action recommendations).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard and Toni with Othman to derive at the invention of claim 7. The result would have been expected, routine, and predictable in order to perform person and object tracking.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Woolard and Toni in this manner in order to improve person and object tracking by analyzing multiple metadata indicative of individual features captured from a plurality of cameras, thereby properly viewing each region and preventing uncaptured regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Woolard and Toni, while the teaching of Othman continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of acquiring multiple image data from different cameras to properly track objects across a large area. The Woolard, Toni, and Othman systems perform object detection and tracking, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 9, Woolard and Toni do not expressly disclose wherein the first plurality of images extracted from the first video stream is less than a total number of images making up the first video stream. However, Othman teaches wherein the first plurality of images extracted from the first video stream is less than a total number of images making up the first video stream (see col. 6 lines 47-48, Othman discusses extracting images that contain the person being tracked, therefore less than the total number of images making up a video stream).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard and Toni with Othman to derive at the invention of claim 9. The result would have been expected, routine, and predictable in order to perform person and object tracking.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Woolard and Toni in this manner in order to improve person and object tracking by analyzing multiple image data captured from a plurality of cameras, thereby properly viewing each region and preventing uncaptured regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Woolard and Toni, while the teaching of Othman continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of acquiring multiple image data from different cameras to properly track objects across a large area. The Woolard, Toni, and Othman systems perform object detection and tracking, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 10, Othman teaches wherein the plurality of instructions further cause the processor of the edge server to: store the first video stream in a data storage device of the edge server (see figure 1C, Othman discusses storage memory);
determine, based on the analysis of each image of the first plurality of images, that additional images are required to detect the presence of the person within the first area of interest (see figure 1C, Othman discusses multiple cameras to detect objects across different regions);
retrieve a portion of the first video stream from the data storage device, the portion of the first video stream is associated with the first plurality of images (see figure 3A, figure 3B, Othman discusses data portions of the captured video streams);
extract a third plurality of images from the retrieved portion of the first video stream, wherein the third plurality of images extracted from the portion of the first video stream comprises more extracted images than the first plurality of images (see figure 3A, figure 3B, Othman discusses data portions of the captured video streams); and
analyze each image of the third plurality of images with the one or more artificial intelligence models to detect the presence of the person within the first area of interest (see figure 4A, Othman discusses detecting the presence of person features using a convolutional neural network).
The same motivation of claim 9 is applied to claim 10. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard and Toni with Othman to derive at the invention of claim 10. The result would have been expected, routine, and predictable in order to perform person and object tracking.
Regarding claim 11, Woolard and Toni do not expressly disclose further comprising a global configuration server communicatively coupled to the edge server with the external network and remotely positioned from the monitored location, wherein the global configuration server comprises a processor executing a plurality of instructions stored in memory, wherein the plurality of instructions cause the processor of the global configuration server to transmit the one or more artificial intelligence models to the edge server.
However, Othman teaches further comprising a global configuration server communicatively coupled to the edge server with the external network and remotely positioned from the monitored location, wherein the global configuration server comprises a processor executing a plurality of instructions stored in memory, wherein the plurality of instructions cause the processor of the global configuration server to transmit the one or more artificial intelligence models to the edge server (see col. 5 lines 1-9, Othman discusses transmitting data to an external processing unit; see col. 15 lines 1-22, Othman discusses performing object detection and tracking using an internal or remote computer and server network).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard and Toni with Othman to derive at the invention of claim 11. The result would have been expected, routine, and predictable in order to perform person and object tracking.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Woolard and Toni in this manner in order to improve person and object tracking by analyzing multiple image data captured from a plurality of cameras, thereby properly viewing and tracking objects across different regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Woolard and Toni, while the teaching of Othman continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of acquiring multiple image data from different cameras to properly track objects across a large area. The Woolard, Toni, and Othman systems perform object detection and tracking, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 13 is rejected as applied to claim 2 as pertaining to a corresponding method.
Claim 18 is rejected as applied to claim 7 as pertaining to a corresponding method.
Claim 20 is rejected as applied to claim 9 as pertaining to a corresponding method.
Claim 21 is rejected as applied to claim 10 as pertaining to a corresponding method.
Claim 22 is rejected as applied to claim 11 as pertaining to a corresponding method.
Claims 8, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Woolard et al. (US 2024/0212027) in view of Toni et al. (US 2023/0088414) in view of Mahbub et al. (US 11,381,743).
Regarding claim 8, Woolard and Toni do not expressly disclose wherein the plurality of instructions further cause the processor of the edge server to adaptively increase or decrease a number of the first plurality of images extracted from the first video stream as a function of the analysis of the each image of the first plurality of images with the one or more artificial intelligence models. However, Mahbub teaches wherein the plurality of instructions further cause the processor of the edge server to adaptively increase or decrease a number of the first plurality of images extracted from the first video stream as a function of the analysis of the each image of the first plurality of images with the one or more artificial intelligence models (see col. 8 line 52- col. 9 line 10, Mahbub discusses increasing or decreasing the number of images captured by cameras based on the required power consumption or bandwidth of the network).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Woolard and Toni with Mahbub to derive at the invention of claim 8. The result would have been expected, routine, and predictable in order to perform person and object tracking.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Woolard and Toni in this manner in order to improve person and object tracking by analyzing multiple overlapping image data captured from a plurality of cameras, thereby properly viewing each region and preventing uncaptured regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Woolard and Toni, while the teaching of Mahbub continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of acquiring multiple image data from different cameras to properly track objects across a large area. The Woolard, Toni, and Mahbub systems perform object detection and tracking, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 19 is rejected as applied to claim 8 as pertaining to a corresponding method.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY A CESE whose telephone number is (571). The examiner can normally be reached on Monday – Friday, 9am – 4pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kenny A Cese/
Primary Examiner, Art Unit 2663