Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
Examiner is interpreting the common dimensions in claim 5 as bitmaps with the same dimensions size.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-5, 10, 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang (“Classifying NBA Offensive Plays Using Neural Networks.”), and further in view of Hammad (US 20210042499 A1).
Regarding claims 1, 10 and 11, Wang discloses detecting an occurrence of an event indicated by a set of data records, the event being associated with an event type, the method comprising ("page 2: 2.1. Pictorial representation of player location sequences SportVU data contains each player’s coordinates on the court and the 3D position of the ball at 25 frames per second. A long with the coordinates it provides a unique player identifier. However, (x,y) coordinates are not sensible representations for our model. We are trying to learn a mapping between the input sequence and the type of play. "):
receiving a plurality of sets of training data records, each training data record in the plurality of sets having associated a (page 4, sec 2.4: In order to disambiguate player position given any line-up, we trained an autoencoder neural network [3] based on a player’s shooting tendencies to map each player to a 10-dimensional embedding space where neighbors in this space have similar tendencies (e.g., range, movements prior to the shot, etc.). We then resolve a player’s position based on his coordinates and neighbors in the embedding space (see Figure 4).);
generating a training bitmap to represent each set of training data records in the plurality of sets ("examiner interprets Fig. 1 as a bitmap representing players’ coordinates on a basketball court
page 4, sec 2.4: In order to disambiguate player position given any line-up, we trained an autoencoder neural network [3] based on a player’s shooting tendencies to map each player to a 10-dimensional embedding space where neighbors in this space have similar tendencies (e.g., range, movements prior to the shot, etc.). We then resolve a player’s position based on his coordinates and neighbors in the embedding space (see Figure 4).
sec. 2.5. Incorporating sequence prediction
With a limited number of examples of each play in the training set, it is possible for a model to perform well on play classification without acquiring an understanding of player movements. "),
the training bitmap defining a representation of a (fig. 1: The rightmost is colored after we disambiguate on court position. Looking at the position-ordered images, both instances the PG initiated from halfcourt while the same positions occupy similar regions on the court, but in the raw SportVU data we see a random ordering of player positions.),
and the training bitmap including identifications of each training data record in the set mapped into the
("page 2, 2.1. Pictorial representation of player location sequences.
We are trying to learn a mapping between the input sequence and the type of play. (10, 10) is twice of the value of (5,5) in the number system, but that does not mean (10,10) contains 2 times the signal for a certain playcall than that of (5,5). We therefore adopt a pictorial representation of the player and ball positions, where each position is a small circle in the image. To simplify our input representation further, we combined player position sequences to form a single image containing player ‘foot-prints’ throughout the play (see Figure 1)."); and
training an image classifier based on each training bitmap such that the trained image classifier is operable to classify an input bitmap as indicating an event of the event type (page 8, sec 4: Using variants of neural networks, we demonstrated the possibility to automate play classification with promising results. With a top-1 classification accuracy of above 50%, simple NN gave us confidence that decent models can work well in this problem. With more understanding towards the data representation, and corresponding changes to a more advanced RNN model, we were able to achieve 66% top-1 accuracy, and 80% top-3 accuracy on unseen examples.). Wang implicitly teaches geospatial (examiner interprets fig. 1 as a Homography and Video Projection as geospatial imagery: For games where sensors are not used, analysts apply "homography" to 2D broadcast footage, converting raw video into a top-down, spatially accurate "minimap" of player positions.) Wang does not explicitly disclose geospatial.
However, in a similar field of endeavor of land encroachment detection, Hammad teaches receiving a plurality of sets of training data records, each training data record in the plurality of sets having associated a geospatial indication and relating to an occurrence of an event of the event type ([0013] The program instructions comprise the steps of receiving satellite image data containing an image of a geospatial area, extracting features from the satellite image data of an object in the image, classifying the object in the image based on the extracted features, comparing the extracted features of the object to previously extracted features for the geospatial area, determining delta features between the extracted features of the object and the previously extracted features for the geospatial area, determining existence of an alarm event in the geospatial area based on the delta features, and transmitting an alarm event message to a communicating device. The program instructions can comprise the further steps of receiving a training dataset for the geospatial area and supplying the training dataset to a deep learning neural network to train the deep learning neural network to determine the existence of the alarm event in the geospatial area. The program instructions can comprise the further step of receiving the previously extracted features for the geospatial area from a storage containing geographic information system data.).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the method of Wang’s offensive play classification using a players coordinates with the known technique of geospatial imagery as taught by Hammad to be able to yield the predictable results of determining the offensive plays of each player by determining their coordinates and positions through geospatial imagery.
Regarding claim 2, Wang discloses receiving an input bitmap ("page 2: A key question for any system is the representation of the input. One option is to utilize features extracted from the SportVU data that are useful for play recognition. These features could encode regular occurrences at different locations on the court (e.g., point guards tend to stand at the key while centers are in the paint), and also frequent patterns such as screens, cuts, drives, and pick-n-rolls. These features are difficult to define and extract. Neural networks, on the other hand, do not depend on hand-coded, engineered representations of input in terms of features, but instead allow these features to be learned from the data. In this paper, neural network (NN) models take as input the pictorial representation of SportVU data as described above.
page 3, sec 2.3: At each step the input is the current pictorial representation of the player position. We also include some fainter representation of player positions from the previous frames, visually similar to the shadow of a player (See Figure 2). "); and
processing the input bitmap by the trained image classifier to determine if the input bitmap indicates an event of the event type ("abstract: We apply techniques from machine learning to directly process SportVU tracking data, specifically variants of neural networks. Our system can label as many sequences with the correct playcall given roughly double the data a human expert needs with high precision, but takes only a fraction of the time.").
Regarding claim 3, Wang discloses wherein the input bitmap is generated to represent a set of input data records each having associated a (page 2, : To simplify our input representation further, we combined player position sequences to form a single image containing player ‘foot-prints’ throughout the play (see Figure 1).).
Wang implicitly teaches geospatial (examiner interprets fig. 1 as a Homography and Video Projection as geospatial imagery: For games where sensors are not used, analysts apply "homography" to 2D broadcast footage, converting raw video into a top-down, spatially accurate "minimap" of player positions.)
Wang does not explicitly disclose but Hammad teaches geospatial ([0032] The image data can include image data that varies over time, such as, for example, a plurality of images of the geospatial target area TA (shown in FIG. 1) that are captured over a period of time.).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the method of Wang’s offensive play classification using a players coordinates with the known technique of geospatial imagery as taught by Hammad to be able to yield the predictable results of determining the offensive plays of each player by determining their coordinates and positions through geospatial imagery.
Regarding claim 4, Wang discloses locations and regions (page 2: A key question for any system is the representation of the input. One option is to utilize features extracted from the SportVU data that are useful for play recognition. These features could encode regular occurrences at different locations on the court (e.g., point guards tend to stand at the key while centers are in the paint), and also frequent patterns such as screens, cuts, drives, and pick-n-rolls.).
Wang implicitly teaches geospatial (examiner interprets fig. 1 as a Homography and Video Projection as geospatial imagery: For games where sensors are not used, analysts apply "homography" to 2D broadcast footage, converting raw video into a top-down, spatially accurate "minimap" of player positions.)
Wang does not explicitly disclose but Hammad teaches geospatial ([0032] The image data can include image data that varies over time, such as, for example, a plurality of images of the geospatial target area TA (shown in FIG. 1) that are captured over a period of time.
[0035] The object data can include object metadata, which can include geospatial coordinates for each of the one or more objects in the geospatial target area TA, time when the image(s) of the one or more objects were captured, and identification and location of the aerial image source 5 or the image capture device (not shown) that captured the image(s) of the object(s).).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the method of Wang’s offensive play classification using a players coordinates with the known technique of geospatial imagery as taught by Hammad to be able to yield the predictable results of determining the offensive plays of each player by determining their coordinates and positions through geospatial imagery.
Regarding claim 5, Wang discloses wherein the training bitmaps have common dimensions (page 4, sec 2.4: In order to disambiguate player position given any line-up, we trained an autoencoder neural network [3] based on a player’s shooting tendencies to map each player to a 10-dimensional embedding space where neighbors in this space have similar tendencies (e.g., range, movements prior to the shot, etc.). We then resolve a player’s position based on his coordinates and neighbors in the embedding space (see Figure 4).).
Claim(s) 6-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang (“Classifying NBA Offensive Plays Using Neural Networks.”), in view of Hammad (US 20210042499 A1) and further in view of Parameswaran (US 20230072641 A1).
Regarding claim 6, Wang discloses a training bitmap (examiner interprets Fig. 1 as bitmaps representing players’ coordinates on a basketball court having, the bitmaps the same dimensions). Wang and Hammad do not disclose scaling; or cropping to adapt the
In a similar field of endeavor of intrusion monitoring, Parameswaran teaches scaling; or cropping to adapt the training bitmap to the common dimensions ("[0052] Referring FIG. 1, system for training a machine learning module of an edge device is shown. Image or images are captured using any suitable camera, video stream, file server or any suitable medium which provides a constant stream of images 110. Images may be in the grayscale or color format. Image may be resized into the desired size by maintaining the aspect ratio. Image may be preprocessed. Obtained image is passed through a change or motion detection module 120.
[0100] These elements will also be found in a conventional desktop or workstation computer as well as other computers suitable for executing computer programs implementing the methods described herein, which may be used in conjunction with any digital print engine or marking engine, display monitor, or other raster output device capable of producing color or gray scale pixels on paper, film, display screen, or other output medium.").
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the method of Wang’s offensive play classification using a players coordinates with the known technique of cropping/scaling an image, taught by Parameswaran, in order to yield predictable results of focusing on an area of an image to identify whether a certain offensive play was made.
Regarding claim 7, Wang discloses a training bitmap and input bitmap (examiner interprets Fig. 1 as input bitmaps representing players’ coordinates on a basketball court having, the bitmaps the same dimensions.
page 2: A key question for any system is the representation of the input. One option is to utilize features extracted from the SportVU data that are useful for play recognition. These features could encode regular occurrences at different locations on the court (e.g., point guards tend to stand at the key while centers are in the paint), and also frequent patterns such as screens, cuts, drives, and pick-n-rolls. These features are difficult to define and extract. Neural networks, on the other hand, do not depend on hand-coded, engineered representations of input in terms of features, but instead allow these features to be learned from the data. In this paper, neural network (NN) models take as input the pictorial representation of SportVU data as described above. Examiner interprets this as training images). Wang implicitly discloses have the common dimensions (examiner interprets Fig. 1 as input bitmaps representing players’ coordinates on a basketball court having, the bitmaps the same dimensions.). Wang and Hammad do not explicitly disclose have the common dimensions.
In a similar field of endeavor of intrusion monitoring, Parameswaran teaches have the common dimensions ("[0052] Referring FIG. 1, system for training a machine learning module of an edge device is shown. Image or images are captured using any suitable camera, video stream, file server or any suitable medium which provides a constant stream of images 110. Images may be in the grayscale or color format. Image may be resized into the desired size by maintaining the aspect ratio. Image may be preprocessed. Obtained image is passed through a change or motion detection module 120.
[0100] These elements will also be found in a conventional desktop or workstation computer as well as other computers suitable for executing computer programs implementing the methods described herein, which may be used in conjunction with any digital print engine or marking engine, display monitor, or other raster output device capable of producing color or gray scale pixels on paper, film, display screen, or other output medium.").
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the method of Wang’s offensive play classification using a players coordinates with the known technique of cropping/scaling an image, taught by Parameswaran, in order to yield predictable results of focusing on an area of an image to identify whether a certain offensive play was made.
Regarding claim 8, Wang discloses an input bitmap (examiner interprets Fig. 1 as input bitmaps representing players’ coordinates on a basketball court having, the bitmaps the same dimensions.).
In a similar field of endeavor of intrusion monitoring, Parameswaran teaches scaling; or cropping to adapt the input bitmap to the common dimensions ("[0052] Referring FIG. 1, system for training a machine learning module of an edge device is shown. Image or images are captured using any suitable camera, video stream, file server or any suitable medium which provides a constant stream of images 110. Images may be in the grayscale or color format. Image may be resized into the desired size by maintaining the aspect ratio. Image may be preprocessed. Obtained image is passed through a change or motion detection module 120.
[0100] These elements will also be found in a conventional desktop or workstation computer as well as other computers suitable for executing computer programs implementing the methods described herein, which may be used in conjunction with any digital print engine or marking engine, display monitor, or other raster output device capable of producing color or gray scale pixels on paper, film, display screen, or other output medium.").
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the method of Wang’s offensive play classification using a players coordinates with the known technique of cropping/scaling an image, taught by Parameswaran, in order to yield predictable results of focusing on an area of an image to identify whether a certain offensive play was made.
Regarding claim 9, Wang and Hammad do not disclose but Parameswaran teaches wherein the event type is a security event and the training data records are records of occurrences occurring at a location ("[0007] For example, the camera could be monitoring intrusion of people in a restricted area, and the model may trigger an event or alarm whenever a person enters the area. The cameras may be operational in sites with limited or no connectivity with the outside world. In such cases it may be necessary to run machine learning models on edge devices which are computing units having limited compute capability and memory.
[0084] Building on the architecture of the system as disclosed above, one aspect of the invention is a method having the processes as follows: capturing an image, detecting an event in said image, on the condition the image contains an event, identifying the event, on the condition that no event occurs, performing validation and training, and storing the result of either identifying, validating, and training.
claim 2: an image localizer module configured to receive the image from the event module and determine the specific location of change and identifying the object in the image;").
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the method of Wang’s offensive play classification using a players coordinates with the known technique of security event classification in an image, as taught by Parameswaran, in order to yield predictable results of determining a security classification of an environment by locating a person’s position to classify whether that person is authorized to be in a certain space.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20150070506 A1 to claim 1: [0050] In another embodiment of the invention the edge analytics module 106 is configured for video analytics algorithm. The edge analytics module 106 generates the cumulative metadata in xml format. The cumulative metadata comprises of event id, event type, video url, UTC timestamp of event occurrence, start time of the event, frame number, duration, location of the event, camera id, count of suspicious objects, object list and severity.
US 20070182818 A1 to claim 1: [0064] Adjacencies can also be determined based on historical data, either real, simulated, or both. In one embodiment, user activity is observed and measured, for example, determining which camera views or RFID station ranges an object is likely to travel to based on prior object movements. In another embodiment, camera images are directly analyzed to determine adjacencies based on scene activity. In this case, the scene activity is choreographed or constrained in some way, using, for example, training data.
US 20220335806 A1 to claim 1: [0038] A digital image may be a raster image, otherwise referred to as a bitmap digital image. A bitmap digital image may be a raster of pixels where each pixel has a property of color.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED A NASHER whose telephone number is (571)272-1885. The examiner can normally be reached Mon - Fri 0800 - 1700.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED A NASHER/Examiner, Art Unit 2675
/ANDREW M MOYER/Supervisory Patent Examiner, Art Unit 2675