DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 30, and therefore claims 31–38 which depend therefrom, are objected to because of the following informalities: line 5 recites “the memory” but intends, as best understood by the Examiner, to refer back to the earlier recited “non-transitory computer readable medium,” and thus should instead recite “the non-transitory computer readable medium.” Appropriate correction is required.
Claim 29 is objected to because of the following informalities: this claims recites “The method of claim 28, further including: a second tracking decoder ..” however a method claim cannot “further include” structural elements, and therefore this should be rewritten instead in a wherein clause, for example, “wherein the diffusion model further includes a second tracking decoder …” Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 25, 34, and 40, therefore claims 26–29 which depend from claim 25, and claims 35–38 which depend from claim 34 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 25, 34 and 40, and claims depending therefrom, claims 25, 34 and 40 recite “a stacking of each player’s events” thus assuming antecedent basis for “players” however, no antecedent basis exists, and it is unclear and indefinite whether it is intended to have “player’s” be “the one or more agents,” (broadly) or if these claims intend to further limit that the “the one or more agents” are “players.”
Regarding claims 29 and 38, these claims recite “trajectories” which does not use a definite article “the” in an effort to refer back to antecedent basis, specifically the earlier recited “trajectory sequences” of claims 21 and 30. Therefore, it is unclear and indefinite whether the claimed “trajectories” of claims 29 and 38 intend to refer back to the earlier recited “trajectory sequences” or if the “trajectories” are a new limitation altogether.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 21-24, 30–33 and 39 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1–3, 5, 10–12, 14 and 19 of copending Application No. 18/401,006 (herein “‘006 application”) in view of Gu et al., "Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 17092-17101, doi: 10.1109/CVPR52688.2022.01660 (herein “Gu”). This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the ‘006 application recite most of the limitations of the present application with correspondence to the claims is set forth below:
Regarding claims 21, 30 and 39, claims 1, 10 and 19, of the ‘006 application correspond as follows, with deficiencies of claims 1, 10 and 19 of the ‘006 application noted in curly brackets {}:
Claims 21, 30 and 39 of the present application
Claims 1, 10 and 19 of the ‘006 application
[Claim 21 only: A computer implemented method for tracking one or more individuals during a sporting event, the method comprising:]
[claim 1 only: A computer implemented method for tracking one or more individuals during a sporting event, the method comprising:]
[Claim 30 only: A system for tracking one or more individuals during a sporting event, the system comprising:]
[claim 10 only: A system for tracking one or more individuals during a sporting event, the system comprising:]
[Claim 39 only: A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:]
[Claim 19 only: A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:]
receiving, as an input, sporting event data, the sporting event data including geospatial data and labeled event data based on the sporting event;
receiving, as an input, geospatial data of a sporting event;
receiving, as an input, labeled event data based on sports broadcast footage of the sporting event;
performing multi-object tracking of one or more agents based on the received geospatial data to determine one or more vectors;
performing multi-object tracking of one or more agents of the geospatial data to determine one or more vectors;
inputting the labeled event data and the one or more vectors into a diffusion model
inputting the labeled event data and one or more vectors into a diffusion model;
determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
and determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
Claims 1, 10 and 19 of the ‘006 application does not, however Gu teaches the diffusion model incorporating a transformer based-neural networks and including: an encoder; and a tracking decoder (Gu page 17094, fig. 2, section 3.1, temporal-social encoder maps a history path and social interaction into a state embedding, where different pedestrians are labeled by number as shown, where section 3.2 teaches a decoder takes yk trajectories, and decodes (tracking decoder)).
Therefore, taking the teachings of claims 1, 10 and 19 of the ‘006 application and Gu together as a whole, it would have been obvious to a “PHOSITA” before the effective filing date of the claimed invention to have modified the claims 1, 10 and 19 of the ‘006 application with the encoder and decoder as disclosed by Gu at least because doing so would allow for predicting trajectories with a flexible indeterminacy that is capable of adapting to dynamic environment. See Gu page 17093, left column.
Regarding claims 22 and 31, claims 2 and 11 of the ‘006 application recite identical limitations.
Regarding claims 23 and 32, claims 3 and 12 of the ‘006 application recite identical limitations.
Regarding claims 24 and 33, claims 5 and 14 of the ‘006 application recite the same limitations.
Claims 21, 30 and 39 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 7 and 13 of copending Application No. 19/314,642 (herein “‘642 application”) in view of Salzmann et al., “Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data,” arXiv:2001.03093v5 [cs.RO], January 13, 2021, https://doi.org/10.48550/arXiv.2001.03093 (herein “Salzmann”) and in view of Gu.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the ‘642 application recite most of the limitations of the present application with correspondence to the claims is set forth below:
Regarding claims 21, 30 and 39, claims 1, 7 and 13 of the ‘642 application correspond as follows, with deficiencies of claims 1, 7 and 13 of the ‘642 application noted in curly brackets {}:
Claims 21, 30 and 39 of the present application
Claims 1, 7 and 13 of the ‘642 application
[claim 21 only: A computer implemented method for tracking one or more individuals during a sporting event, the method comprising:]
[Claim 1 only: A computer implemented method for tracking one or more individuals during a sporting event, the method comprising:]
[claim 30 only: A system for tracking one or more individuals during a sporting event, the system comprising:]
[Claim 7 only: A system for tracking one or more individuals during a sporting event, the system comprising:]
[Claim 39 only: A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:]
[Claim 13 only: A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:]
receiving, as an input, sporting event data, the sporting event data including {geospatial} data and labeled event data based on the sporting event;
receiving, as an input, broadcast tracking data of a sporting event and labeled event data of the sporting event;
performing multi-object tracking of one or more agents based on the received {geospatial} data to determine one or more vectors;
performing multi-object tracking of one or more agents of the broadcast tracking data to determine one or more vectors;
inputting the labeled event data and one or more vectors into a diffusion model;
inputting the labeled event data and one or more vectors into a diffusion model;
and determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
determining, using the diffusion model, one or more trajectory sequences for the one or more agents;
Claims 1, 7 and 13 of the ‘642 application, while reciting “broadcast tracking data of a sporting event” does not recite the data specifically as being “geospatial data” however, Salzmann teaches “geospatial data” on page 7, LIDAR data incorporated into the event/agent representation vectors.
Further, claims 1, 7 and 13 of the ‘642 application do not, however Gu teaches the diffusion model incorporating a transformer based-neural networks and including: an encoder; and a tracking decoder (Gu page 17094, fig. 2, section 3.1, temporal-social encoder maps a history path and social interaction into a state embedding, where different pedestrians are labeled by number as shown, where section 3.2 teaches a decoder takes yk trajectories, and decodes (tracking decoder)).
Therefore, taking the teachings of claims 1, 7 and 13 of the ‘642 application and Salzmann together as a whole, it would have been obvious to a “PHOSITA” before the effective filing date of the claimed invention to have modified claims 1, 7 and 13 of the ‘642 application to have the geographic data teachings cited above as disclosed by Salzmann at least because doing so would allow for modeling agents with different perception ranges. See Salzmann, bottom of page 5.
Therefore, taking the teachings of claims 1, 7 and 13 of the ‘642 application and Gu together as a whole, it would have been obvious to a “PHOSITA” before the effective filing date of the claimed invention to have modified the teachings of claims 1, 7 and 13 of the ‘642 application with the encoder and decoder as disclosed by Gu at least because doing so would allow for predicting trajectories with a flexible indeterminacy that is capable of adapting to dynamic environment. See Gu page 17093, left column.
Claims 21–23, 30–32 and 39 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1–2, 4, 11–12, 14 and 18 of copending Application No. 19/309,109 (herein “‘109 application”) in view of Salzmann et al., “Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data,” arXiv:2001.03093v5 [cs.RO], January 13, 2021, https://doi.org/10.48550/arXiv.2001.03093 (herein “Salzmann”) and in view of Gu.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the ‘109 application recite most of the limitations of the present application with correspondence to the claims is set forth below:
Regarding claims 21, 30 and 39, claims 1, 11 and 18 of the ‘109 application correspond as follows, with deficiencies of claims 1, 11 and 18 of the ‘109 application noted in curly brackets {}:
Claims 21, 30 and 39 of the present application
Claims 1, 11 and 18 of the ‘109 application
[claim 21 only: A computer implemented method for tracking one or more individuals during a sporting event, the method comprising:]
[Claim 1 only: A method for generating trajectories for one or more players during a sporting event, the method comprising:]
[claim 30 only: A system for tracking one or more individuals during a sporting event, the system comprising:]
[Claim 11 only: A system for generating trajectories for one or more players during a sporting event, the system comprising:]
[Claim 39 only: A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:]
[Claim 18 only: A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:]
receiving, as an input, sporting event data, the sporting event data including {geospatial} data and {labeled} event data based on the sporting event;
receiving, as an input, broadcast footage of a sporting event; receiving event data of the sporting event;
performing multi-object tracking of one or more agents based on the received {geospatial} data to determine one or more vectors;
performing multi-object tracking of one or more agents of the broadcast tracking data to determine one or more vectors;
inputting the labeled event data and one or more vectors into a diffusion model;
inputting the one or more vectors and event data into a multimodal model, determining, by the multimodal model, a tensor, inputting … the tensor into a diffusion model
wherein the diffusion model includes a decoder;
the diffusion model incorporating a {transformer based-neural networks} and including:{an encoder;} and a tracking decoder;
wherein the diffusion model includes a decoder;
and determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
generating, by the diffusion model, one or more trajectories for the one or more players in the sporting event.
Claims 1, 11 and 18 of the ‘109 application, while reciting “broadcast tracking data of a sporting event” does not recite the data specifically as being “geospatial data” or that the event data is “labeled” however, Salzmann teaches “geospatial data” (Salzmann page 7, LIDAR data incorporated into the event/agent representation vectors), and labeled event data (Salzmann page 5, section 4, fig. 2, agents are classified (car, bus, pedestrian), thus labeled and are nodes in a spatiotemporal graph, along with edges that represent interactions (events)).
Further, claims 1, 11 and 18 of the ‘109 application do not, however Gu teaches the diffusion model incorporating a transformer based-neural networks and including: an encoder; (Gu page 17094, fig. 2, section 3.1, MID framework including a transformer neural network and a temporal-social encoder mapping a history path and social interaction into a state embedding, where different pedestrians are labeled by number as shown).
Therefore, taking the teachings of claims 1, 11 and 18 of the ‘109 application and Salzmann together as a whole, it would have been obvious to a “PHOSITA” before the effective filing date of the claimed invention to have modified claims 1, 11 and 18 of the ‘109 application to have the geographic data teachings cited above as disclosed by Salzmann at least because doing so would allow for modeling agents with different perception ranges. See Salzmann, bottom of page 5.
Therefore, taking the teachings of claims 1, 11 and 18 of the ‘109 application and Gu together as a whole, it would have been obvious to a “PHOSITA” before the effective filing date of the claimed invention to have modified the teachings of claims 1, 11 and 18 of the ‘109 application with the encoder as disclosed by Gu at least because doing so would allow for predicting trajectories with a flexible indeterminacy that is capable of adapting to dynamic environment. See Gu page 17093, left column.
Regarding claims 22 and 31, claims 4 and 14 of the ‘109 application recite the same limitations.
Regarding claims 23 and 32, claims 2 and 12 of the ‘109 application recite the same limitations.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 21–23, 30–32, and 39 are rejected under 35 U.S.C. 103 as being unpatentable over Salzmann et al., “Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data,” arXiv:2001.03093v5 [cs.RO], January 13, 2021, https://doi.org/10.48550/arXiv.2001.03093 (herein “Salzmann” – an earlier version of this reference was cited by Applicant in the IDS filed 9/18/2024), in view of Gu et al., "Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 17092-17101, doi: 10.1109/CVPR52688.2022.01660 (herein “Gu”) in view of Zhu et al., "Event Tactic Analysis Based on Broadcast Sports Video," in IEEE Transactions on Multimedia, vol. 11, no. 1, pp. 49-67, Jan. 2009, doi: 10.1109/TMM.2008.2008918 (herein “Zhu”).
Regarding claims 21, 30 and 39, with claim 21 as exemplary, substantive differences
between the claims noted in curly brackets {}, and deficiencies of Salzmann noted in square brackets [], Salzmann teaches {A computer implemented method for tracking one or more individuals during a [sporting] event, the method comprising: - claim 1 / A system for tracking one or more individuals during a [sporting] event, the system comprising: a non-transitory computer readable medium configured to store processor-readable instructions; and a processor operatively connected to the memory, and configured to execute the instructions to perform operations comprising: - claim 7 / A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising: - claim 13}(Salzmann abstract, pages 5 and 9, Trajectron++ model for predicting trajectories based on tracking pedestrians in a street scene, is implemented in PyTorch on a computer running Ubuntu (instructions), the computer containing a CPU and GPUs, where CPUs and GPUs are understood to have computer readable memory for executing processes)
receiving, as an input, [sporting] event data, the [sporting] event data including geospatial data (Salzmann page 7, first full paragraph, page 2, additional information of LIDAR data (geospatial data) is included (received input) into the Trajectron++ model framework and encoded as a vector to be added to the backbone of representation vectors ex, the LIDAR data being of the environment in which the dynamics (events) of the agents is taking place), and labeled event data based on the [sporting] event (Salzmann page 5, section 4, fig. 2, agents are classified (car, bus, pedestrian), thus labeled and are nodes in a created (receiving) spatiotemporal graph, along with edges that represent interactions (events));
performing multi-object tracking of one or more agents based on the received geospatial data to determine one or more vectors (Salzmann page 6, Modeling Agent History and Encoding Agent Interactions sections, each agent (multi-object) has its current state and its history encoded (tracking), the agents being “of the received geospatial data” since the agents are tracked in the environment the LIDAR data represents, and encodings are made from the agent interactions to produce a single node representation vector ex (vector));
inputting the labeled event data and one or more vectors into a [diffusion] model (Salzmann page 7, Producing Dynamically-Feasible Trajectories section, the backbone representation ex is fed (inputting) into a decoder (model));
[the diffusion model incorporating a transformer based-neural networks and including: an encoder; and a tracking decoder;] and
determining, using the [diffusion] model, one or more trajectory sequences for the one or more agents (Salzmann page 7, Producing Dynamically-Feasible Trajectories section, the decoder produces trajectories in position space).
While Salzmann teaches receiving data tracking pedestrians in a street scene, nonetheless, Salzmann does not explicitly teach “sporting event.”
Further, Salzmann does not teach that its model is a “diffusion model” or that the diffusion model is “incorporating a transformer based-neural networks and including: an encoder; and a tracking decoder.”
Gu teaches a diffusion model (Gu page 17094, section 3.2, trajectories are determined by way of a diffusion process), incorporating a transformer based-neural networks and including: an encoder; and a tracking decoder” (Gu page 17094, fig. 2, sections 3.1 and 3.2, MID framework including a temporal-social encoder mapping a history path and social interaction into a state embedding, and a transformer based decoder that decodes trajectories (tracking)).
Zhu teaches “sporting event” (Zhu fig. 1, page 52, section III, attack event extraction performed on input broadcast soccer video of a soccer game (sporting event)).
Therefore, taking the teachings of Salzmann and Gu together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the model of Salzmann to include diffusion and thus be a diffusion model as disclosed by Gu at least because doing so would allow for predicting trajectories with a flexible indeterminacy that is capable of adapting to dynamic environment. See Gu page 17093, left column.
Further, taking the teachings of Salzmann as modified by Gu and Zhu together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the pedestrian event data and LIDAR data of Salzmann to be sporting event data of the sporting event as disclosed by Zhu at least because doing so would allow for discovering tactic patterns amongst professional athletes, in order to improve team performance during a game. See Zhu abstract, section I.
Regarding claims 22 and 31, with claim 22 as exemplary, Salzmann does not explicitly teach, but Zhu teaches wherein the labeled event data includes a sequential stream of one or more major events throughout a sport event, the one or more major events including at least one of a pass, shot, tackle, foul, turnover, penalty, goal, score, or substitution from the sporting event (Zhu pages 52–53, section III A, fig. 2, web-casting text describing the event that has happened in a game with a timestamp and brief description such as “goal” or “headed in” for a scoring event, also called a “shot”), and wherein the geospatial data includes one or more of a sports broadcast footage, in-venue footage, global positioning system (GPS) data, near field communication (NFC) data, or radio-frequency identification (RFID) data (Zhu page 53, section B, broadcast soccer videos are analyzed and input into the analysis model).
Further, taking the teachings of Salzmann as modified by Gu and Zhu together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the pedestrian event data and LIDAR data of Salzmann to be major events in a sports game like a goal or score or shot as disclosed by Zhu at least because doing so would allow for discovering tactic patterns amongst professional athletes, in order to improve team performance during a game. See Zhu abstract, section I.
Regarding claims 23 and 32, with claim 23 as exemplary, Salzmann teaches wherein the one or more vector includes at least one of an agent two dimensional coordinates on a sporting event's field, an agent position, an agent team, an indicator indicating the agent is a ball, or player visibility information (given that the claim only requires “at least one,” Salzmann teaches on page 5, the spatiotemporal graph which is converted to the representation vector ex includes 2D world positions of agents (an agent position)).
Claims 24 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Salzmann in view of Gu in view of Zhu, as set forth above regarding claims 21 and 30 from which claims 24 and 33 respectively depend, further in view of Alcorn et al., "baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotemporal Modeling," arXiv:2102.03291v3, September 28, 2021, https://doi.org/10.48550/arXiv.2102.03291 (herein “Alcorn”).
Regarding claims 24 and 33, with claim 24 as exemplary, Salzmann teaches wherein determining, using the diffusion model, one or more trajectory sequences for the one or more agents, as set forth above in the rejection rationale for claims 21 and 30, but does not explicitly teach, where Alcorn teaches further including: applying spatiotemporal axial attention to the labeled event data and the one or more vectors (Alcorn page 2, fig. 1, transformer model applies self-attention, given the locations of the players (spatio) through time (temporal) using a self-attention mask tensor (multiple dimensional/axial) on multi-entity (one or more vectors) sequential data (event data)).
Therefore, taking the teachings of Salzmann as modified by Gu and Zhu and Alcorn together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified to have the model of Salzmann to apply spatiotemporal axial attention teachings cited above as disclosed by Alcorn at least because doing so would allow for capturing idiosyncratic qualities of players, thus providing a more detailed analysis of game play for a particular sport. See Alcorn, bottom of page 2.
Allowable Subject Matter
Claims 25, 34 and 40, and therefore claims 26–29, and 35–38 which depend from claims 25 and 34 respectively, are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and that any claim objections and the obviousness non-statutory double patenting issues given above are addressed.
Specifically, claims 25, 34 and 40 recite “tokenizing the labeled event data using a linear projection; applying sinusoidal positional embeddings to specify temporal occurrences of the event data,; processing the event data with stacked encoders; and outputting event embeddings” which is not taught or suggested in Salzmann, Gu, Zhu or Alcorn. Further, additionally cited, but not used in any of the above rejections, Strudel et al., US PgPub No. US 2024/0119261, while setting forth using a diffusion model that uses a sequence of discrete tokens, and applies a linear projection to convert tokens, does not teach that the linear projection is used for the tokenizing, much less tokenizing the labeled event data, as claimed. Further, none of Salzmann, Gu, Zhu, Alcorn, or Strudel, or any of the other cited art of record, whether considered alone or in an obvious combination to a PHOSITA teaches or suggests “applying sinusoidal positional embeddings to specify temporal occurrences of the event data,” and all other limitations from claims 25, 34 and 40, and thus, these claims, and any other claims which depend therefrom, are allowable over the cited art of record.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Darwish et al., “STE: Spatio-Temporal Encoder for Action Spotting in Soccer Videos,” Proceedings of the 5th International ACM Workshop on Multimedia Content Analysis in Sports (MMSports’22), October 10, 2022, Lisboa, Portugal, ACM, 6 pages, directed towards extracting events from video of sports games.
Tjondronegoro et al., "Multi-modal summarization of key events and top players in sports tournament videos," 2011 IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, USA, 2011, pp. 471-478, doi: 10.1109/WACV.2011.5711541, directed towards a multi-modal analysis framework to automatically detect, annotate and visualize sports summaries in matches and tournaments.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/
Primary Examiner, Art Unit 2671