DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are pending under this Office action.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 11-16 are rejected under §35 U.S.C. 101 as not falling within one of the four statutory categories of invention because the claimed invention is directed to computer program per se. See MPEP 2106(1). A claim directed toward a non-transitory computer readable medium having the program encoded thereon establishes a sufficient functional relationship between the program and a computer so as to remove it from the realm of “program per se”. MPEP 2111.05(111). Hence, adding the limitation of “non-transitory” before “computer-readable storage medium” for claims 11-16 would resolve this issue.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7-15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Long, etc. (US 20170157512 A1) in view of Chiu (US 20060248209 A1), further in view of Shear, etc. (US 20160034305 A1) and Mahan, etc. (US 20100262780 A1).
Regarding claim 1, Long teaches that a method for pre-storing resources for artificial reality (XR) effects, the method (See Long: Fig. 34, and [0154], “Finally, FIG. 34 is a schematic diagram 3400 of a computing device in which the present invention may be utilized, according to some embodiments of the present invention. A computing device comprises a hardware processor 3402 for executing program code, an operating system 3414, an application software 3416, which may implement the various embodiments of the present invention described herein, a physical memory 3404, at least one user device 3406, at least one output device 3408, a mass storage device 3410, and a network 3412. The network 3412 comprises a wired or wireless network to communicate to remote servers and databases via data networks such as the Internet. The program code utilized by the computing device may be provided on a non-transitory physical storage medium, such as a local hard-disk, a hard-disk in the cloud, or any other physical storage medium (not shown)”) comprising:
predictively pre-storing one or more resources for one or more XR effects (See Long: Fig. 2, and [0134], “In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”; [0105], “Highlight and special effect servers 270 may utilize machine learning and/or machine vision algorithms to auto-detect particular game events, scenes or moments of interest, and generate highlight videos, with or without highlight effects such as slow-motion or close-up with fly-by camera angles. Such highlight videos may be broadcasted or streamed on its own, may be spliced back into a live stream, or may be made available to tournament and league operators to incorporate into existing broadcast to game video streaming and sharing platforms, such as Twitch and YouTube”; and [0091], “FIG. 2 is an architectural overview of a game video live cast and replay framework 200, according to one embodiment of the present invention. In this embodiment, a tournament server 210 provides game feed, program feed, and optionally stadium feed to a SLIVER server 240, which in turn provides media content, either as live streams to a SLIVER livestream content distribution network (CDN) 280, or as video-on-demand to a SLIVER content database 290”. Note that the VOD has special effect added into the real play games as predicted in the critical event moments in the games) by:
generating a prediction of improved performance resulting from pre- storing the one or more resources (See Long: Fig. 2, and [0134], “In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”; and [0106], “In the present disclosure, a highlight of a game play refers to media clips or recordings that feature or focus on one or more particular periods of time, or moments, during a game play, often extending over auto-determined or user-identified gaming events that are either exciting, memorable, or of special interest to viewers. A gaming event or moment is generally associated with a timestamp and/or a location with a game map of the source computer game”. Note that the timestamp and location of the critical events in the games may be the prediction of improved performance, where the stored special effects will be applied to modify the recorded games, and when the games are playback to the user, the modified games with special effects added at the predicted timestamps and locations are of improved performance);
determining that one or more conditions, for storing the one or more resources in a cache (See Long: Fig. 34, and [0156], “The processor may represent one or more processors (e.g., microprocessors), and the memory may represent random access memory (RAM) devices comprising a main storage of the hardware, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or back-up memories (e.g., programmable or flash memories), read-only memories, etc. In addition, the memory may be considered to include memory storage physically located elsewhere in the hardware, e.g. any cache memory in the processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device”. Note that the program, game recording, and special effects may be stored in the cache), was satisfied by making a comparison (See Long: Fig. 2, and [0096], “In some embodiments, such visual, audio, and metadata cues may be combined, correlated or cross-compared to generate highlight metadata, which may provide more accurate gaming moment identification results than using individual cues. In yet some embodiments, highlight metadata may be equivalent to extracted visual, audio, and/or metadata cues”) of a) a resource score, computed for the one or more resources and based on statistics of XR effect use (See Long: Figs. 3-5. And [0128], “In yet some embodiments, one or more viewers can specify important game actions, and direct the system to highlight certain game actions or game moments. In some embodiments, one or more viewers can vote highlight videos up or down or score different aspects of highlight videos, thus providing feedback to machine learning algorithms driving the highlight generation module”), with b) a cache threshold; and
in response to the generating the prediction and to the determining that the one or more conditions was satisfied, adding, to the cache, the one or more resources (See Long: Fig. 4, and [0120], “In the particular embodiment shown in FIG. 4, tournament server 410 comprises a game client 420, a SLIVER SDK 425, and a game server 430. Game client 420 may be configured into a spectator mode to provide functionalities similar to those performed by spectator interface 225, caster 230, and live stadium server 235 shown in FIG. 2. SLIVER SDK 245 is integrated into the tournament server 410 to interface with SLIVER server 440. A Software Development Kit (SDK) is a set of software development tools or programming packages for creating applications for a specific platform. An SDK may be compiled as part of the developed application to provide dedicated interfaces and functionalities. Alternatively, an SDK may be an individually compiled module, incorporable into an existing game on a user device as a plug-in, add-on, or extension in order to add specific features to the game without accessing its source code. In this embodiment, SLIVER SDK 425 may be used to insert virtual cameras or virtual camera arrays into the source computer game, while also collaborating with SLIVER server 440 through SLIVER standard interface 450 to control such virtual cameras for game video and game highlight video generation. For example, SLIVER SDK 425 may receive camera control data, time control data, or API handles for game or tournament configuration and other functions necessary for capturing recordings of the game play. SLIVER SDK 425 may also provide rendered textures and game event callbacks to SLIVER server 440”; and Figs. 9A-D, and [0138], “FIGS. 9B, 9C, and 9D are exemplary screenshots 952, 954, and 956 of a bullet-time highlight replay of the game play in FIG. 9A, captured at locations 922, 924, and 926, as a virtual camera pans-in or scans around the group of players shown, in a clockwise direction, along virtual camera trajectory or path 920. The virtual camera points radially inward along the trajectory. In screenshot 952 shown in FIG. 9B, player B1 941 not visible in screenshot 952 in FIG. 9A becomes clearly visible; in screenshot 954 shown in FIG. 9C, player B1 941 is still partially visible through the player information bar displayed on top of the player avatar. Thus, the SLIVER bullet-time effect shown in this example allows the viewer to see players and possibly other parts of the game world that would have otherwise been omitted or obstructed in a conventional screencast game video of a regular gameplay”. Note that special features or effects, such as the “bullet-time effect”, are added into the recorded games at the determined timestamps and locations, which may be mapped to the claimed limitation of “add the one or more resources”); and
rendering the one or more XR effects (See Long: Figs. 23-24, and [0145], “FIGS. 23 and 24 are exemplary screenshots 2300 and 2400 of a time freeze highlight of the gaming moment shown in FIG. 15, respectively, according to one embodiment of the present invention. The time freeze highlight effect is captured or applied at the killing moment of player D1, right before or just-in-time for the kill. The game video is paused in FIG. 23, zooms into player D1 briefly with some transition effects in FIG. 24, for example, for half a second, then zooms out again to resume the game video”),
wherein the rendering the one or more XR effects uses at least some of the pre-stored one or more resources (See Long: Fig. 7, and [0134], “A gaming moment may be a particular period of time extending over an auto-determined or user-identified gaming event that are either exciting, memorable, or of special interest to viewers during a game play. Thus, the gaming moment may start before the gaming event of interest takes place. Such a “premonition” of the gaming event for game moment capture may be enabled by the use of buffered data, where a SLIVER server may record and store game data including game metadata and video recordings for any duration, from a few milliseconds to an entire game play, for “post-processing” actions where such post-processing may occur with very little delay relative to the live game play. In some embodiments, the SLIVER server may utilize inherent transmission delays in a game broadcast to process game data, thus providing highlight videos in perceived real-time, or just-in-time. In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”. Note that use the buffer data for the special effects may be use the pre-stored resources); and
wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre-stored one or more resources is not available in the cache.
However, Long fails to explicitly disclose that determining that one or more conditions, was satisfied by making a comparison of a) a resource score, computed for the one or more resources and based on statistics of XR effect use, with b) a cache threshold; and wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre-stored one or more resources is not available in the cache.
However, Chiu teaches that determining that one or more conditions (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data”; Note that match statistics may be determining one or more conditions), was satisfied by making a comparison of a) a resource score, computed for the one or more resources (See Chiu: Fig. 7, and [0125], “In this embodiment, users 701 1 and 701 n are downloading the same podcast but the advertisement or advertisements that user 1 hears while listening to the podcast are East Coast relevant and the advertisement or advertisements that user n hears while listening to the podcast are West Coast relevant. In this way, advertisers may more closely target their advertising to those consumers of a podcastor's contents who might best fit the advertiser's profile of a best target audience”. Note that the best fit AD may be the resource score) and based on statistics of XR effect use (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data”, Note that the statistics of AD placements and frequency may be the XR effect statistics), with b) a cache (See Chiu: Fig. 7, and [0120], “Server 702 has connection to a mass repository 703 adapted to store actual podcast multimedia files including concatenated advertisement files. Web storage 703 may be a local media storage system with respect to server 702 for convenience, or it may be a remote storage system or even a plurality of accessible storage systems. Within storage system 703 there is illustrated 2 separate versions of a same podcast each version containing different advertisements. A podcast A (P-Cast A) is illustrated in a hierarchy with the 2 versions associated under the title. For example, P-Cast A-east coast (EC) is available and P-Cast A-WC is available. The version differs by the advertisements, which in this case, are determined appropriate for service in part by region in which a user accessing the podcast resides”; [0119], “Host server 702 has appropriate Input/output (I/O) ports for communication as a web server. Server 702 publishes at least one if not several RSS feeds 705, which may reference commercialized podcast multimedia content. Server 702 has a server cache for caching multimedia content for download or streaming access by consumers who subscribe to particular podcast feeds. In one embodiment, users may also customize content by subscribing to several feeds that the server aggregates into one RSS feed channel. When users 701 (1-n) go online using their devices running RSS, any new commercialized content that is available is published to their devices via RSS and is ready to access”. Note that the podcast with matched advertisements are selected and cached for users based on user’s location, and this may mapped to the claimed limitation of determining the cached content based on XR effect score) threshold.
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Long to have determining that one or more conditions, was satisfied by making a comparison of a) a resource score, computed for the one or more resources and based on statistics of XR effect use, with b) a cache as taught by Chiu in order to enable matching of sponsors such as advertisers with publishers such as podcastors and to concatenate or multiplex the podcasts with suitable advertisements (See Chiu: Fig. 1, and [0012], “What is clearly needed in the art is a network and interface to enable matching of sponsors (advertisers) with (publishers) podcastors and to concatenate or multiplex the podcasts with suitable advertisements”). Long teaches a method and system that may generate and provide the highlighted game videos for users to replay by detecting the critical events in the recorded games and adding special effects on the critical events to generate the highlighted video games stored in memory or cache for users; while Chiu teaches a system and method that may deliver efficiently audio and video advertising podcasts to user by associating advisers to the podcast content in some specific contexts or at some timestamps and allowing the users to pre-download and cache the content in local memory. Therefore, it is obvious to one of ordinary skill in the art to modify Long by Chiu to use various criterions to cache the special effects for the video game contents based on contexts and cache capacity. The motivation to modify Long by Chiu is “Use of known technique to improve similar devices (methods, or products) in the same way”.
However, Long, modified by Chiu, fails to explicitly disclose that b) a cache threshold; and wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre-stored one or more resources is not available in the cache.
However, Shear teaches that determining that b) a cache threshold (See Shear: Figs. 1-2, and [5185], “Optimization for purpose—resource alignment, arrangement and/or parameterizations may be managed so as to optimize to purpose expression (e.g. CPE), for example, discovering resources for purpose from boundless resource arrays. For example, such processes can identify resource parameters in suitable for proper or optimal user purpose satisfaction, such as inadequate video resolution, streaming bitrate, cache storage capacity, cost, and/or the like”; and [01590], “To reduce storage requirements, some embodiments may limit the number of cached values of internal class system expressions in contexts to a bounded number, per expression, per context, and/or overall. Such bounded caches may manage eviction of values using techniques analogous to well-known techniques used in virtual memory systems, for example, Least Recently Used (LRU) and/or First In, First Out (FIFO). If an evicted value is needed again, it may be re-computed in the same way it was originally. The re-evaluation may be less costly than the original evaluation, because it may be able to use other values that are still in the cache”; Note that the cache capacity and bounded cache may be the cache threshold).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Long to have b) a cache threshold as taught by Shear in order to ensure facilitating user purpose in the computing architecture in an effective manner (See Shear: Fig. 2, and [5193], “In some embodiments, each of these Coherence process, specifications, Resolutions and Operations operate in an iterative manner and may include feedback loops. In one example implementation, for any given instanced Coherence process set. There is also PERCos Platform Coherence Management Services which provides access to previous Coherence implementations, specifications and operations in, for example, the form of specifications, templates and/or persisted operational sessions, such that similar specifications and/or operations sets may be made available in an efficient and effective manner in pursuit of purpose”). Long teaches a method and system that may generate and provide the highlighted game videos for users to replay by detecting the critical events in the recorded games and adding special effects on the critical events to generate the highlighted video games stored in memory or cache for users; while Shear teaches a system and method that may organize the computing resources to suit the people’s purpose of computing needs such as caching the purposes of the user based on the purposes and the cache capacity (threshold). Therefore, it is obvious to one of ordinary skill in the art to modify Long by Shear to cache the special effects (user purposes) for the video game contents based on contexts and cache threshold. The motivation to modify Long by Shear is “Use of known technique to improve similar devices (methods, or products) in the same way”.
However, Long, modified by Chiu and Shear, fails to explicitly disclose that wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre-stored one or more resources is not available in the cache.
However, Mahan teaches that determining that wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre-stored one or more resources is not available in the cache (See Mahan: Fig. 1, and [0054], “In these aspects, shared DOM 38 may allow for faster rendering of the requested instance of the page 34, as web engine 14 does not need to reconstruct an entire DOM structure for a new instance of a page corresponding to a DOM already stored. Instead, web engine 14 can reuse static DOM portion 42, and only needs to perform the processing related to the one or more dynamic DOM portion(s) 48 corresponding to the requested instance of the page 34”; and [0031], “According to one or more aspects, apparatus and methods of rendering a page provide a web engine or other components operable to create a shared DOM that may be used by two or more instances of the page. The shared DOM includes a static DOM portion that is common to the different instances of a page, and one or more dynamic DOM portions that are unique to a respective one or more instances of the page. As such, the described aspects improve efficiency in page rendering by reusing the static DOM portion in rendering a new instance of a page that corresponds to a stored or cached DOM, which can be based on a previously processed, different instance of the same page, thereby avoiding having to create an entirely new DOM for each instance of the page”; Note that the locally cached contents may be rendered faster, which is very common practice in graphic rendering, i.e., cached visual effects will be rendered faster than non-cached content, as disclosed in this prior art).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Long to have wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre-stored one or more resources is not available in the cache as taught by Mahan in order to reuse the static portion of DOM while reducing the consumption of communication resources (See Mahan: Fig. 1, and [0033], “In other words, in one aspect, when a unique instance of a page (for example, an "itemdetail" page) has never been fetched or cached on a computer device, and that instance of the page is requested, the described aspects provide a behavior that results in fetching the page data (for example, including the hypertext markup language (html) or extensible html (xhtml), cascading sheet style (css), and javascript (js)) and creating a static portion of a document object model (DOM) from the xhtml and css. This static portion of the DOM is stored in cache and can be reused. For that page, the js is then executed, resulting in one or more data requests (for example, an XMLHttpRequest (XHR)). The initial js and the handling of each of the one or more responses results in the creating of one or more dynamic portions of the DOM for that instance of the page. Then, when a second instance of the page is requested, the static DOM portion can be reused, and the js is executed for the new query string of the request corresponding to the second instance of the page, resulting in one or more new data requests and the creating of one or more dynamic portions of the DOM for the new instance of the page. As a result, the shared DOM includes at least one static portion and one or more dynamic portions, enabling the DOM to define a shared DOM for use in rendering different instances of the page”). Long teaches a method and system that may generate and provide the highlighted game videos for users to replay by detecting the critical events in the recorded games and adding special effects on the critical events to generate the highlighted video games stored in memory or cache for users; while Mahan teaches a system and method that may cache the un-changed portions and reuse it to make the next pages rendering faster. Therefore, it is obvious to one of ordinary skill in the art to modify Long by Mahan to use the cached resources to make the rendering faster. The motivation to modify Long by Mahan is “Use of known technique to improve similar devices (methods, or products) in the same way”.
Regarding claim 2, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long teaches that the method of claim 1, wherein at least some of the statistics are identified as being specific to a context corresponding to a current situation, the context identifying one or more of: user characteristics, a location, identified video content the one or more XR effects are to be applied to, or any combination thereof (See Long: Figs. 3-4, and [0113], “Moreover, historical data including statistics from multiple game plays may be correlated and/or analyzed, possibly together with live game play data, to determined virtual camera locations for gaming moment capturing and highlight video generation. For example, virtual cameras may be inserted around areas where a competing player is most likely to get killed, where a visually-spectacular event is most likely to happen, or along a path or lane that is most critical for winning a game. In some embodiments, such historical data may be player-specific, team-specific, or be classified or selected based on predetermined or configurable conditions”).
Regarding claim 3, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long and Chiu teach that the method of claim 1, wherein at least some of the statistics are identified as being specific to a current user, the statistics signifying:
a frequency of XR effect use by the current user (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data. In one embodiment, data managed may also include performance statistics related to traffic accessing commercialized podcasts. Performance statistics related to advertisement placement may also be provided”); and/or
in what context the current user selects particular XR effects (See Long: Figs. 3-4, and [0113], “As discussed, computer vision and other machine learning algorithms may be applied to the game environment, previously recorded game plays, or live game plays for highlight metadata generation, possibly based on extracted visual, audio, and/or metadata cues, in different embodiments of the present invention. Examples of such algorithms include, but are not limited to edge detection, feature extraction, segmentation, object recognition, pose estimation, motion analysis, liner and non-liner transforms in time, spatial, or frequency domains, hypothesis testing, decision trees, neural networks including convolutional neural networks, vector quantization, and many others. Moreover, historical data including statistics from multiple game plays may be correlated and/or analyzed, possibly together with live game play data, to determined virtual camera locations for gaming moment capturing and highlight video generation. For example, virtual cameras may be inserted around areas where a competing player is most likely to get killed, where a visually-spectacular event is most likely to happen, or along a path or lane that is most critical for winning a game. In some embodiments, such historical data may be player-specific, team-specific, or be classified or selected based on predetermined or configurable conditions”; and [0018], “In some embodiments, the determination of the one or more highlight virtual camera trajectories is based on a type of a desired highlight effect, where the desired highlight effect is selected from the group consisting of a spatial scaling, a temporal scaling, a visual special effect, and an augmentation with game metadata”).
Regarding claim 4, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long teaches that the e method of claim 1, wherein at least some of the XR statistics are identified as being specific to a current area, signifying a frequency of XR effect use for the current area (See Long: Figs. 9A-D, and [0113], “] As discussed, computer vision and other machine learning algorithms may be applied to the game environment, previously recorded game plays, or live game plays for highlight metadata generation, possibly based on extracted visual, audio, and/or metadata cues, in different embodiments of the present invention. Examples of such algorithms include, but are not limited to edge detection, feature extraction, segmentation, object recognition, pose estimation, motion analysis, liner and non-liner transforms in time, spatial, or frequency domains, hypothesis testing, decision trees, neural networks including convolutional neural networks, vector quantization, and many others. Moreover, historical data including statistics from multiple game plays may be correlated and/or analyzed, possibly together with live game play data, to determined virtual camera locations for gaming moment capturing and highlight video generation. For example, virtual cameras may be inserted around areas where a competing player is most likely to get killed, where a visually-spectacular event is most likely to happen, or along a path or lane that is most critical for winning a game. In some embodiments, such historical data may be player-specific, team-specific, or be classified or selected based on predetermined or configurable conditions”).
Regarding claim 5, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long teaches that the method of claim 1, wherein the generating the prediction is based on processing by a machine learning model that was trained to predict which XR effects a user will select (See Long: Fig. 3, and [0116], “Mapping and auto-syncing algorithms 364 may analyze the program feed in real-time, apply computer vision or other machine learning algorithms to determine the location of the current or the first point-of-view player, and identify one or more best or optimal locations of virtual cameras or virtual camera arrays to capture actions in and around this player. Similarly, in some embodiments, mapping and auto-syncing algorithms 364 may be configured to determine the location of players not shown on screen, and identify optimal virtual camera locations in and around such off-screen players”).
Regarding claim 7, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long teaches that the method of claim 1, wherein the cache is in RAM or flash storage (See Long: Fig. 34, and [0156], “The present invention may be implemented in hardware and/or in software. Many components of the system, for example, network interfaces etc., have not been shown, so as not to obscure the present invention. However, one of ordinary skill in the art would appreciate that the system necessarily includes these components. A computing device is a hardware that includes at least one processor coupled to a memory. The processor may represent one or more processors (e.g., microprocessors), and the memory may represent random access memory (RAM) devices comprising a main storage of the hardware, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or back-up memories (e.g., programmable or flash memories), read-only memories, etc. In addition, the memory may be considered to include memory storage physically located elsewhere in the hardware, e.g. any cache memory in the processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device”).
Regarding claim 8, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long teaches that the method of claim 1, wherein the one or more resources comprise an audio graph, a gesture recognition system, a face tracking system, a movement or object target tracking system, music services, location services, or any combination thereof (See Long: Fig. 2, and [0096], “Through the game feed from spectator interface 225, SLIVER server 240 may have full access to recordings and data of the game play, including player interface data as shown to individual players, spectator interface data as broadcasted, as well as other game play data compilations, in the form or one or more merged video screencast streams, or in multiple data streams that can be analyzed separately. For example, SLIVER server 240 may examine a video stream to identify or extract visual and/or audio cues that indicate an exciting moment has occurred in the game play. Exemplary visual cues include sudden, successive changes in color that indicates the occurrence of explosions, placement of multiple players within very close proximity which may indicate an intense battle scene, and the like. Exemplary audio cues include explosion sounds, changes in tone in player commentaries, and the like. SLIVER server 240 may further examine game metadata to extract metadata cues for critical game moment identification. For example, death of a player is often considered an important or critical event, and the killing moment is subsequently highlighted or replayed. In some embodiments, such visual, audio, and metadata cues may be combined, correlated or cross-compared to generate highlight metadata, which may provide more accurate gaming moment identification results than using individual cues. In yet some embodiments, highlight metadata may be equivalent to extracted visual, audio, and/or metadata cues”. Note that audio may be the music).
Regarding claim 9, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long teaches that the method of claim 1, wherein the one or more resources comprise a video segmentation system that identifies portions of video content including one or more of a background, pre-defined objects, parts of users, or any combination thereof (See Long: Figs. 2-3, and [0106], “In the present disclosure, a highlight of a game play refers to media clips or recordings that feature or focus on one or more particular periods of time, or moments, during a game play, often extending over auto-determined or user-identified gaming events that are either exciting, memorable, or of special interest to viewers. A gaming event or moment is generally associated with a timestamp and/or a location with a game map of the source computer game. A highlight video may comprise screencasts captured using pre-existing virtual cameras within the game world, game play captured by inserted virtual cameras from viewing perspectives different from those shown during an initial broadcast, highlight effects, augmentations, or game video segments generated using any other suitable video processing techniques that make the highlight video attractive to spectators and the like. Exemplary highlight effects include spatial scaling, temporal scaling, visual special effects, augmentations, or any other processing techniques that make the highlight video different from a static screencast of the original game play. Examples of spatial scaling includes zoom-in, zoom-out, preview close-up, and the like. Examples of temporal scaling includes time freeze, time-lapse, slow-motion, and the like. Examples of visual special effects include bullet-time, glitch effect, exposure effect, noir effect, morphing, stitching, optical effects, and the like. Augmentations or annotations may be performed to supplement the highlight video with game metadata or other available game information. For example, annotations may be provided by the SLIVER system, by broadcasters, or even spectators; augmentation may also be provided by marking the location and field-of-view of an active player, and overlaying game statistics on a video. In different embodiments, augmentations or annotations may be provided in audio and/or visual forms”; and [0107], “FIG. 3 is a schematic diagram showing the overall architecture 300 of game and system servers for game video generation and processing, according to another embodiment of the present invention. In this embodiment, a tournament server 310 provides game data such as metadata and rendered scenes to SLIVER server 340, and in turn receives game control information as well as API handles for game or tournament configuration and other functions necessary for running a game play and capturing recordings of the game play. Such game control data may include virtual camera control signals and other game capture configuration parameters”. Note that the scene may be the game backgrounds).
Regarding claim 10, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long teaches that the method of claim 1, wherein the one or more resources comprise machine learning models, 3D modeling systems, 2D to 3D conversion systems, or any combination thereof (See Long: Figs. 3-4, and [0111], “In some embodiments, map data and logic analyzer 354 may collaborate with other components such as mapping and autosyncing algorithms 364, capture module 362 and/or highlights module 372 to conduct auto-detection of one or more optimal virtual camera locations even before a game play is initiated, by analyzing map or game data as provided by game developers or tournament operators, or by leveraging computer vision as well as other machine learning algorithms applied to previously recorded game plays or the game environment itself. Such identified virtual camera locations may be considered “pre-determined” relative to the game play, and the identified optional game virtual camera locations may or may not coincide with virtual camera locations within the game map as originally provided by the game developers. In some embodiments, pre-determined locations may refer to user-identified or user configured locations. In yet some embodiments, locations may be considered “pre-determined” relative to particular gaming moments; in other words, any virtual camera inserted, activated, or configured before or when a gaming event or moment takes place, even after initiation of the overall game play, may be considered “pre-determined.” In some embodiments, each predetermined location may be a static location within a game map, a tracking location associated with a game player, a tracking location associated with a game object, a dynamic location that may be controlled by a game broadcaster, or a dynamic location that may be controlled by a spectator. In some embodiments, virtual cameras may be inserted into the source computer game at the identified optional virtual camera locations using an SDK such as SLIVER SDK 425 shown in FIG. 4 or a game connector module, such as game connector client 350 and game connector server 352”).
Regarding claim 11, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long, Chiu, Shear, and Mahan teach that a computer-readable storage medium storing instructions, for pre-storing resources for artificial reality (XR) effects, the instructions, when executed by a computing system, cause the computing system (See Long: Fig. 34, and [0154], “Finally, FIG. 34 is a schematic diagram 3400 of a computing device in which the present invention may be utilized, according to some embodiments of the present invention. A computing device comprises a hardware processor 3402 for executing program code, an operating system 3414, an application software 3416, which may implement the various embodiments of the present invention described herein, a physical memory 3404, at least one user device 3406, at least one output device 3408, a mass storage device 3410, and a network 3412. The network 3412 comprises a wired or wireless network to communicate to remote servers and databases via data networks such as the Internet. The program code utilized by the computing device may be provided on a non-transitory physical storage medium, such as a local hard-disk, a hard-disk in the cloud, or any other physical storage medium (not shown)”) to:
predictively pre-store one or more resources for one or more XR effects (See Long: Fig. 2, and [0134], “In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”; [0105], “Highlight and special effect servers 270 may utilize machine learning and/or machine vision algorithms to auto-detect particular game events, scenes or moments of interest, and generate highlight videos, with or without highlight effects such as slow-motion or close-up with fly-by camera angles. Such highlight videos may be broadcasted or streamed on its own, may be spliced back into a live stream, or may be made available to tournament and league operators to incorporate into existing broadcast to game video streaming and sharing platforms, such as Twitch and YouTube”; and [0091], “FIG. 2 is an architectural overview of a game video live cast and replay framework 200, according to one embodiment of the present invention. In this embodiment, a tournament server 210 provides game feed, program feed, and optionally stadium feed to a SLIVER server 240, which in turn provides media content, either as live streams to a SLIVER livestream content distribution network (CDN) 280, or as video-on-demand to a SLIVER content database 290”. Note that the VOD has special effect added into the real play games as predicted in the critical event moments in the games) by:
generating a prediction of improved performance resulting from pre- storing the one or more resources (See Long: Fig. 2, and [0134], “In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”; and [0106], “In the present disclosure, a highlight of a game play refers to media clips or recordings that feature or focus on one or more particular periods of time, or moments, during a game play, often extending over auto-determined or user-identified gaming events that are either exciting, memorable, or of special interest to viewers. A gaming event or moment is generally associated with a timestamp and/or a location with a game map of the source computer game”. Note that the timestamp and location of the critical events in the games may be the prediction of improved performance, where the stored special effects will be applied to modify the recorded games, and when the games are playback to the user, the modified games with special effects added at the predicted timestamps and locations are of improved performance);
determining that one or more conditions (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data”; Note that match statistics may be determining one or more conditions), was satisfied by making a comparison of a) a resource score, computed for the one or more resources (See Chiu: Fig. 7, and [0125], “In this embodiment, users 701 1 and 701 n are downloading the same podcast but the advertisement or advertisements that user 1 hears while listening to the podcast are East Coast relevant and the advertisement or advertisements that user n hears while listening to the podcast are West Coast relevant. In this way, advertisers may more closely target their advertising to those consumers of a podcastor's contents who might best fit the advertiser's profile of a best target audience”. Note that the best fit AD may be the resource score) and based on statistics of XR effect use (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data”, Note that the statistics of AD placements and frequency may be the XR effect statistics), with b) a cache (See Chiu: Fig. 7, and [0120], “Server 702 has connection to a mass repository 703 adapted to store actual podcast multimedia files including concatenated advertisement files. Web storage 703 may be a local media storage system with respect to server 702 for convenience, or it may be a remote storage system or even a plurality of accessible storage systems. Within storage system 703 there is illustrated 2 separate versions of a same podcast each version containing different advertisements. A podcast A (P-Cast A) is illustrated in a hierarchy with the 2 versions associated under the title. For example, P-Cast A-east coast (EC) is available and P-Cast A-WC is available. The version differs by the advertisements, which in this case, are determined appropriate for service in part by region in which a user accessing the podcast resides”; [0119], “Host server 702 has appropriate Input/output (I/O) ports for communication as a web server. Server 702 publishes at least one if not several RSS feeds 705, which may reference commercialized podcast multimedia content. Server 702 has a server cache for caching multimedia content for download or streaming access by consumers who subscribe to particular podcast feeds. In one embodiment, users may also customize content by subscribing to several feeds that the server aggregates into one RSS feed channel. When users 701 (1-n) go online using their devices running RSS, any new commercialized content that is available is published to their devices via RSS and is ready to access”. Note that the podcast with matched advertisements are selected and cached for users based on user’s location, and this may mapped to the claimed limitation of determining the cached content based on XR effect score), for storing the one or more resources in a cache (See Long: Fig. 34, and [0156], “The processor may represent one or more processors (e.g., microprocessors), and the memory may represent random access memory (RAM) devices comprising a main storage of the hardware, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or back-up memories (e.g., programmable or flash memories), read-only memories, etc. In addition, the memory may be considered to include memory storage physically located elsewhere in the hardware, e.g. any cache memory in the processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device”. Note that the program, game recording, and special effects may be stored in the cache), was satisfied by making a comparison (See Long: Fig. 2, and [0096], “In some embodiments, such visual, audio, and metadata cues may be combined, correlated or cross-compared to generate highlight metadata, which may provide more accurate gaming moment identification results than using individual cues. In yet some embodiments, highlight metadata may be equivalent to extracted visual, audio, and/or metadata cues”) of a) a resource score, computed for the one or more resources and based on statistics of XR effect use (See Long: Figs. 3-5. And [0128], “In yet some embodiments, one or more viewers can specify important game actions, and direct the system to highlight certain game actions or game moments. In some embodiments, one or more viewers can vote highlight videos up or down or score different aspects of highlight videos, thus providing feedback to machine learning algorithms driving the highlight generation module”), with b) a cache threshold (See Shear: Figs. 1-2, and [5185], “Optimization for purpose—resource alignment, arrangement and/or parameterizations may be managed so as to optimize to purpose expression (e.g. CPE), for example, discovering resources for purpose from boundless resource arrays. For example, such processes can identify resource parameters in suitable for proper or optimal user purpose satisfaction, such as inadequate video resolution, streaming bitrate, cache storage capacity, cost, and/or the like”; and [01590], “To reduce storage requirements, some embodiments may limit the number of cached values of internal class system expressions in contexts to a bounded number, per expression, per context, and/or overall. Such bounded caches may manage eviction of values using techniques analogous to well-known techniques used in virtual memory systems, for example, Least Recently Used (LRU) and/or First In, First Out (FIFO). If an evicted value is needed again, it may be re-computed in the same way it was originally. The re-evaluation may be less costly than the original evaluation, because it may be able to use other values that are still in the cache”; Note that the cache capacity and bounded cache may be the cache threshold); and
in response to the generating the prediction and to the determining that the one or more conditions was satisfied, adding, to the cache, the one or more resources (See Long: Fig. 4, and [0120], “In the particular embodiment shown in FIG. 4, tournament server 410 comprises a game client 420, a SLIVER SDK 425, and a game server 430. Game client 420 may be configured into a spectator mode to provide functionalities similar to those performed by spectator interface 225, caster 230, and live stadium server 235 shown in FIG. 2. SLIVER SDK 245 is integrated into the tournament server 410 to interface with SLIVER server 440. A Software Development Kit (SDK) is a set of software development tools or programming packages for creating applications for a specific platform. An SDK may be compiled as part of the developed application to provide dedicated interfaces and functionalities. Alternatively, an SDK may be an individually compiled module, incorporable into an existing game on a user device as a plug-in, add-on, or extension in order to add specific features to the game without accessing its source code. In this embodiment, SLIVER SDK 425 may be used to insert virtual cameras or virtual camera arrays into the source computer game, while also collaborating with SLIVER server 440 through SLIVER standard interface 450 to control such virtual cameras for game video and game highlight video generation. For example, SLIVER SDK 425 may receive camera control data, time control data, or API handles for game or tournament configuration and other functions necessary for capturing recordings of the game play. SLIVER SDK 425 may also provide rendered textures and game event callbacks to SLIVER server 440”; and Figs. 9A-D, and [0138], “FIGS. 9B, 9C, and 9D are exemplary screenshots 952, 954, and 956 of a bullet-time highlight replay of the game play in FIG. 9A, captured at locations 922, 924, and 926, as a virtual camera pans-in or scans around the group of players shown, in a clockwise direction, along virtual camera trajectory or path 920. The virtual camera points radially inward along the trajectory. In screenshot 952 shown in FIG. 9B, player B1 941 not visible in screenshot 952 in FIG. 9A becomes clearly visible; in screenshot 954 shown in FIG. 9C, player B1 941 is still partially visible through the player information bar displayed on top of the player avatar. Thus, the SLIVER bullet-time effect shown in this example allows the viewer to see players and possibly other parts of the game world that would have otherwise been omitted or obstructed in a conventional screencast game video of a regular gameplay”. Note that special features or effects, such as the “bullet-time effect”, are added into the recorded games at the determined timestamps and locations, which may be mapped to the claimed limitation of “add the one or more resources”); and
render the one or more XR effects (See Long: Figs. 23-24, and [0145], “FIGS. 23 and 24 are exemplary screenshots 2300 and 2400 of a time freeze highlight of the gaming moment shown in FIG. 15, respectively, according to one embodiment of the present invention. The time freeze highlight effect is captured or applied at the killing moment of player D1, right before or just-in-time for the kill. The game video is paused in FIG. 23, zooms into player D1 briefly with some transition effects in FIG. 24, for example, for half a second, then zooms out again to resume the game video”),
wherein the rendering the one or more XR effects uses at least some of the pre-stored one or more resources (See Long: Fig. 7, and [0134], “A gaming moment may be a particular period of time extending over an auto-determined or user-identified gaming event that are either exciting, memorable, or of special interest to viewers during a game play. Thus, the gaming moment may start before the gaming event of interest takes place. Such a “premonition” of the gaming event for game moment capture may be enabled by the use of buffered data, where a SLIVER server may record and store game data including game metadata and video recordings for any duration, from a few milliseconds to an entire game play, for “post-processing” actions where such post-processing may occur with very little delay relative to the live game play. In some embodiments, the SLIVER server may utilize inherent transmission delays in a game broadcast to process game data, thus providing highlight videos in perceived real-time, or just-in-time. In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”. Note that use the buffer data for the special effects may be use the pre-stored resources); and
wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre-stored one or more resources is not available in the cache (See Mahan: Fig. 1, and [0054], “In these aspects, shared DOM 38 may allow for faster rendering of the requested instance of the page 34, as web engine 14 does not need to reconstruct an entire DOM structure for a new instance of a page corresponding to a DOM already stored. Instead, web engine 14 can reuse static DOM portion 42, and only needs to perform the processing related to the one or more dynamic DOM portion(s) 48 corresponding to the requested instance of the page 34”; and [0031], “According to one or more aspects, apparatus and methods of rendering a page provide a web engine or other components operable to create a shared DOM that may be used by two or more instances of the page. The shared DOM includes a static DOM portion that is common to the different instances of a page, and one or more dynamic DOM portions that are unique to a respective one or more instances of the page. As such, the described aspects improve efficiency in page rendering by reusing the static DOM portion in rendering a new instance of a page that corresponds to a stored or cached DOM, which can be based on a previously processed, different instance of the same page, thereby avoiding having to create an entirely new DOM for each instance of the page”; Note that the locally cached contents may be rendered faster, which is very common practice in graphic rendering, i.e., cached visual effects will be rendered faster than non-cached content, as disclosed in this prior art).
Regarding claim 12, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 11 as outlined above. Further, Long teaches that the computer-readable storage medium of claim 11, wherein at least some of the statistics are identified as being specific to a context corresponding to a current situation, the context identifying one or more of: user characteristics, a location, identified video content the one or more XR effects are to be applied to, or any combination thereof (See Long: Figs. 3-4, and [0113], “Moreover, historical data including statistics from multiple game plays may be correlated and/or analyzed, possibly together with live game play data, to determined virtual camera locations for gaming moment capturing and highlight video generation. For example, virtual cameras may be inserted around areas where a competing player is most likely to get killed, where a visually-spectacular event is most likely to happen, or along a path or lane that is most critical for winning a game. In some embodiments, such historical data may be player-specific, team-specific, or be classified or selected based on predetermined or configurable conditions”).
Regarding claim 13, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 11 as outlined above. Further, Long and Chiu teach that the computer-readable storage medium of claim 11, wherein at least some of the statistics are identified as being specific to a current user, the statistics signifying:
a frequency of XR effect use by the current user (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data. In one embodiment, data managed may also include performance statistics related to traffic accessing commercialized podcasts. Performance statistics related to advertisement placement may also be provided”); and/or
in what context the current user selects particular XR effects (See Long: Figs. 3-4, and [0113], “As discussed, computer vision and other machine learning algorithms may be applied to the game environment, previously recorded game plays, or live game plays for highlight metadata generation, possibly based on extracted visual, audio, and/or metadata cues, in different embodiments of the present invention. Examples of such algorithms include, but are not limited to edge detection, feature extraction, segmentation, object recognition, pose estimation, motion analysis, liner and non-liner transforms in time, spatial, or frequency domains, hypothesis testing, decision trees, neural networks including convolutional neural networks, vector quantization, and many others. Moreover, historical data including statistics from multiple game plays may be correlated and/or analyzed, possibly together with live game play data, to determined virtual camera locations for gaming moment capturing and highlight video generation. For example, virtual cameras may be inserted around areas where a competing player is most likely to get killed, where a visually-spectacular event is most likely to happen, or along a path or lane that is most critical for winning a game. In some embodiments, such historical data may be player-specific, team-specific, or be classified or selected based on predetermined or configurable conditions”; and [0018], “In some embodiments, the determination of the one or more highlight virtual camera trajectories is based on a type of a desired highlight effect, where the desired highlight effect is selected from the group consisting of a spatial scaling, a temporal scaling, a visual special effect, and an augmentation with game metadata”).
Regarding claim 14, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 11 as outlined above. Further, Long teaches that the computer-readable storage medium of claim 11, wherein at least some of the XR statistics are identified as being specific to a current area, signifying a frequency of XR effect use for the current area (See Long: Figs. 9A-D, and [0113], “] As discussed, computer vision and other machine learning algorithms may be applied to the game environment, previously recorded game plays, or live game plays for highlight metadata generation, possibly based on extracted visual, audio, and/or metadata cues, in different embodiments of the present invention. Examples of such algorithms include, but are not limited to edge detection, feature extraction, segmentation, object recognition, pose estimation, motion analysis, liner and non-liner transforms in time, spatial, or frequency domains, hypothesis testing, decision trees, neural networks including convolutional neural networks, vector quantization, and many others. Moreover, historical data including statistics from multiple game plays may be correlated and/or analyzed, possibly together with live game play data, to determined virtual camera locations for gaming moment capturing and highlight video generation. For example, virtual cameras may be inserted around areas where a competing player is most likely to get killed, where a visually-spectacular event is most likely to happen, or along a path or lane that is most critical for winning a game. In some embodiments, such historical data may be player-specific, team-specific, or be classified or selected based on predetermined or configurable conditions”).
Regarding claim 15, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 11 as outlined above. Further, Long teaches that the e computer-readable storage medium of claim 11, wherein the one or more resources comprise machine learning models, 3D modeling systems, 2D to 3D conversion systems, or any combination thereof (See Long: Figs. 3-4, and [0111], “In some embodiments, map data and logic analyzer 354 may collaborate with other components such as mapping and autosyncing algorithms 364, capture module 362 and/or highlights module 372 to conduct auto-detection of one or more optimal virtual camera locations even before a game play is initiated, by analyzing map or game data as provided by game developers or tournament operators, or by leveraging computer vision as well as other machine learning algorithms applied to previously recorded game plays or the game environment itself. Such identified virtual camera locations may be considered “pre-determined” relative to the game play, and the identified optional game virtual camera locations may or may not coincide with virtual camera locations within the game map as originally provided by the game developers. In some embodiments, pre-determined locations may refer to user-identified or user configured locations. In yet some embodiments, locations may be considered “pre-determined” relative to particular gaming moments; in other words, any virtual camera inserted, activated, or configured before or when a gaming event or moment takes place, even after initiation of the overall game play, may be considered “pre-determined.” In some embodiments, each predetermined location may be a static location within a game map, a tracking location associated with a game player, a tracking location associated with a game object, a dynamic location that may be controlled by a game broadcaster, or a dynamic location that may be controlled by a spectator. In some embodiments, virtual cameras may be inserted into the source computer game at the identified optional virtual camera locations using an SDK such as SLIVER SDK 425 shown in FIG. 4 or a game connector module, such as game connector client 350 and game connector server 352”).
Regarding claim 17, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. Further, Long, Chiu, Shear, and Mahan teach that a computing system for pre-storing resources for artificial reality (XR) effects, the computing system (See Long: Fig. 34, and [0154], “Finally, FIG. 34 is a schematic diagram 3400 of a computing device in which the present invention may be utilized, according to some embodiments of the present invention. A computing device comprises a hardware processor 3402 for executing program code, an operating system 3414, an application software 3416, which may implement the various embodiments of the present invention described herein, a physical memory 3404, at least one user device 3406, at least one output device 3408, a mass storage device 3410, and a network 3412. The network 3412 comprises a wired or wireless network to communicate to remote servers and databases via data networks such as the Internet. The program code utilized by the computing device may be provided on a non-transitory physical storage medium, such as a local hard-disk, a hard-disk in the cloud, or any other physical storage medium (not shown)”) comprising:
one or more processors (See Long: Fig. 34, and [0154], “Finally, FIG. 34 is a schematic diagram 3400 of a computing device in which the present invention may be utilized, according to some embodiments of the present invention. A computing device comprises a hardware processor 3402 for executing program code, an operating system 3414, an application software 3416, which may implement the various embodiments of the present invention described herein, a physical memory 3404, at least one user device 3406, at least one output device 3408, a mass storage device 3410, and a network 3412. The network 3412 comprises a wired or wireless network to communicate to remote servers and databases via data networks such as the Internet. The program code utilized by the computing device may be provided on a non-transitory physical storage medium, such as a local hard-disk, a hard-disk in the cloud, or any other physical storage medium (not shown)”); and
one or more memories storing instructions that, when executed by the one or more
processors, cause the computing system (See Long: Fig. 34, and [0154], “Finally, FIG. 34 is a schematic diagram 3400 of a computing device in which the present invention may be utilized, according to some embodiments of the present invention. A computing device comprises a hardware processor 3402 for executing program code, an operating system 3414, an application software 3416, which may implement the various embodiments of the present invention described herein, a physical memory 3404, at least one user device 3406, at least one output device 3408, a mass storage device 3410, and a network 3412. The network 3412 comprises a wired or wireless network to communicate to remote servers and databases via data networks such as the Internet. The program code utilized by the computing device may be provided on a non-transitory physical storage medium, such as a local hard-disk, a hard-disk in the cloud, or any other physical storage medium (not shown)”) to:
predictively pre-store one or more resources for one or more XR effects (See Long: Fig. 2, and [0134], “In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”; [0105], “Highlight and special effect servers 270 may utilize machine learning and/or machine vision algorithms to auto-detect particular game events, scenes or moments of interest, and generate highlight videos, with or without highlight effects such as slow-motion or close-up with fly-by camera angles. Such highlight videos may be broadcasted or streamed on its own, may be spliced back into a live stream, or may be made available to tournament and league operators to incorporate into existing broadcast to game video streaming and sharing platforms, such as Twitch and YouTube”; and [0091], “FIG. 2 is an architectural overview of a game video live cast and replay framework 200, according to one embodiment of the present invention. In this embodiment, a tournament server 210 provides game feed, program feed, and optionally stadium feed to a SLIVER server 240, which in turn provides media content, either as live streams to a SLIVER livestream content distribution network (CDN) 280, or as video-on-demand to a SLIVER content database 290”. Note that the VOD has special effect added into the real play games as predicted in the critical event moments in the games) by:
generating a prediction of improved performance resulting from pre- storing the one or more resources (See Long: Fig. 2, and [0134], “In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”; and [0106], “In the present disclosure, a highlight of a game play refers to media clips or recordings that feature or focus on one or more particular periods of time, or moments, during a game play, often extending over auto-determined or user-identified gaming events that are either exciting, memorable, or of special interest to viewers. A gaming event or moment is generally associated with a timestamp and/or a location with a game map of the source computer game”. Note that the timestamp and location of the critical events in the games may be the prediction of improved performance, where the stored special effects will be applied to modify the recorded games, and when the games are playback to the user, the modified games with special effects added at the predicted timestamps and locations are of improved performance);
determining that one or more conditions (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data”; Note that match statistics may be determining one or more conditions), was satisfied by making a comparison of a) a resource score, computed for the one or more resources (See Chiu: Fig. 7, and [0125], “In this embodiment, users 701 1 and 701 n are downloading the same podcast but the advertisement or advertisements that user 1 hears while listening to the podcast are East Coast relevant and the advertisement or advertisements that user n hears while listening to the podcast are West Coast relevant. In this way, advertisers may more closely target their advertising to those consumers of a podcastor's contents who might best fit the advertiser's profile of a best target audience”. Note that the best fit AD may be the resource score) and based on statistics of XR effect use (See Chiu: Figs. 1-2, and [0069], “PAS 200 has a data interface 203 to system internal, or to a locally managed advertiser/publisher database similar to database 113 described with reference to FIG. 1. Interface 203 enables the application to manage, input, and retrieve data for use in other tasks performed by the software. In one embodiment, the data managed may include account data for publishers and advertisers, advertisement matching statistics and history, advertisement placement statistics and history, frequency of use statistics, publisher compensation schedules and history, advertiser payment schedule and history and general billing histories and data”, Note that the statistics of AD placements and frequency may be the XR effect statistics), with b) a cache (See Chiu: Fig. 7, and [0120], “Server 702 has connection to a mass repository 703 adapted to store actual podcast multimedia files including concatenated advertisement files. Web storage 703 may be a local media storage system with respect to server 702 for convenience, or it may be a remote storage system or even a plurality of accessible storage systems. Within storage system 703 there is illustrated 2 separate versions of a same podcast each version containing different advertisements. A podcast A (P-Cast A) is illustrated in a hierarchy with the 2 versions associated under the title. For example, P-Cast A-east coast (EC) is available and P-Cast A-WC is available. The version differs by the advertisements, which in this case, are determined appropriate for service in part by region in which a user accessing the podcast resides”; [0119], “Host server 702 has appropriate Input/output (I/O) ports for communication as a web server. Server 702 publishes at least one if not several RSS feeds 705, which may reference commercialized podcast multimedia content. Server 702 has a server cache for caching multimedia content for download or streaming access by consumers who subscribe to particular podcast feeds. In one embodiment, users may also customize content by subscribing to several feeds that the server aggregates into one RSS feed channel. When users 701 (1-n) go online using their devices running RSS, any new commercialized content that is available is published to their devices via RSS and is ready to access”. Note that the podcast with matched advertisements are selected and cached for users based on user’s location, and this may mapped to the claimed limitation of determining the cached content based on XR effect score), for storing the one or more resources in a cache (See Long: Fig. 34, and [0156], “The processor may represent one or more processors (e.g., microprocessors), and the memory may represent random access memory (RAM) devices comprising a main storage of the hardware, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or back-up memories (e.g., programmable or flash memories), read-only memories, etc. In addition, the memory may be considered to include memory storage physically located elsewhere in the hardware, e.g. any cache memory in the processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device”. Note that the program, game recording, and special effects may be stored in the cache), was satisfied by making a comparison (See Long: Fig. 2, and [0096], “In some embodiments, such visual, audio, and metadata cues may be combined, correlated or cross-compared to generate highlight metadata, which may provide more accurate gaming moment identification results than using individual cues. In yet some embodiments, highlight metadata may be equivalent to extracted visual, audio, and/or metadata cues”) of a) a resource score, computed for the one or more resources and based on statistics of XR effect use (See Long: Figs. 3-5. And [0128], “In yet some embodiments, one or more viewers can specify important game actions, and direct the system to highlight certain game actions or game moments. In some embodiments, one or more viewers can vote highlight videos up or down or score different aspects of highlight videos, thus providing feedback to machine learning algorithms driving the highlight generation module”), with b) a cache threshold (See Shear: Figs. 1-2, and [5185], “Optimization for purpose—resource alignment, arrangement and/or parameterizations may be managed so as to optimize to purpose expression (e.g. CPE), for example, discovering resources for purpose from boundless resource arrays. For example, such processes can identify resource parameters in suitable for proper or optimal user purpose satisfaction, such as inadequate video resolution, streaming bitrate, cache storage capacity, cost, and/or the like”; and [01590], “To reduce storage requirements, some embodiments may limit the number of cached values of internal class system expressions in contexts to a bounded number, per expression, per context, and/or overall. Such bounded caches may manage eviction of values using techniques analogous to well-known techniques used in virtual memory systems, for example, Least Recently Used (LRU) and/or First In, First Out (FIFO). If an evicted value is needed again, it may be re-computed in the same way it was originally. The re-evaluation may be less costly than the original evaluation, because it may be able to use other values that are still in the cache”; Note that the cache capacity and bounded cache may be the cache threshold); and
in response to the generating the prediction and to the determining that the one or more conditions was satisfied, adding, to the cache, the one or more resources (See Long: Fig. 4, and [0120], “In the particular embodiment shown in FIG. 4, tournament server 410 comprises a game client 420, a SLIVER SDK 425, and a game server 430. Game client 420 may be configured into a spectator mode to provide functionalities similar to those performed by spectator interface 225, caster 230, and live stadium server 235 shown in FIG. 2. SLIVER SDK 245 is integrated into the tournament server 410 to interface with SLIVER server 440. A Software Development Kit (SDK) is a set of software development tools or programming packages for creating applications for a specific platform. An SDK may be compiled as part of the developed application to provide dedicated interfaces and functionalities. Alternatively, an SDK may be an individually compiled module, incorporable into an existing game on a user device as a plug-in, add-on, or extension in order to add specific features to the game without accessing its source code. In this embodiment, SLIVER SDK 425 may be used to insert virtual cameras or virtual camera arrays into the source computer game, while also collaborating with SLIVER server 440 through SLIVER standard interface 450 to control such virtual cameras for game video and game highlight video generation. For example, SLIVER SDK 425 may receive camera control data, time control data, or API handles for game or tournament configuration and other functions necessary for capturing recordings of the game play. SLIVER SDK 425 may also provide rendered textures and game event callbacks to SLIVER server 440”; and Figs. 9A-D, and [0138], “FIGS. 9B, 9C, and 9D are exemplary screenshots 952, 954, and 956 of a bullet-time highlight replay of the game play in FIG. 9A, captured at locations 922, 924, and 926, as a virtual camera pans-in or scans around the group of players shown, in a clockwise direction, along virtual camera trajectory or path 920. The virtual camera points radially inward along the trajectory. In screenshot 952 shown in FIG. 9B, player B1 941 not visible in screenshot 952 in FIG. 9A becomes clearly visible; in screenshot 954 shown in FIG. 9C, player B1 941 is still partially visible through the player information bar displayed on top of the player avatar. Thus, the SLIVER bullet-time effect shown in this example allows the viewer to see players and possibly other parts of the game world that would have otherwise been omitted or obstructed in a conventional screencast game video of a regular gameplay”. Note that special features or effects, such as the “bullet-time effect”, are added into the recorded games at the determined timestamps and locations, which may be mapped to the claimed limitation of “add the one or more resources”); and
render the one or more XR effects (See Long: Figs. 23-24, and [0145], “FIGS. 23 and 24 are exemplary screenshots 2300 and 2400 of a time freeze highlight of the gaming moment shown in FIG. 15, respectively, according to one embodiment of the present invention. The time freeze highlight effect is captured or applied at the killing moment of player D1, right before or just-in-time for the kill. The game video is paused in FIG. 23, zooms into player D1 briefly with some transition effects in FIG. 24, for example, for half a second, then zooms out again to resume the game video”),
wherein the rendering the one or more XR effects uses at least some of the pre-stored one or more resources (See Long: Fig. 7, and [0134], “A gaming moment may be a particular period of time extending over an auto-determined or user-identified gaming event that are either exciting, memorable, or of special interest to viewers during a game play. Thus, the gaming moment may start before the gaming event of interest takes place. Such a “premonition” of the gaming event for game moment capture may be enabled by the use of buffered data, where a SLIVER server may record and store game data including game metadata and video recordings for any duration, from a few milliseconds to an entire game play, for “post-processing” actions where such post-processing may occur with very little delay relative to the live game play. In some embodiments, the SLIVER server may utilize inherent transmission delays in a game broadcast to process game data, thus providing highlight videos in perceived real-time, or just-in-time. In addition, machine learning algorithms may be employed to see if particular triggering actions are expected, where thresholds and auto-signals may be used to predict is something might happen soon with high probability. Such machine learning algorithms may be trained using historical game play data and viewer feedbacks”. Note that use the buffer data for the special effects may be use the pre-stored resources); and
wherein the rendering using the at least some of the pre-stored one or more resources is performed more quickly than rendering the one or more XR effects when the at least some of the pre- stored one or more resources is not available in the cache (See Mahan: Fig. 1, and [0054], “In these aspects, shared DOM 38 may allow for faster rendering of the requested instance of the page 34, as web engine 14 does not need to reconstruct an entire DOM structure for a new instance of a page corresponding to a DOM already stored. Instead, web engine 14 can reuse static DOM portion 42, and only needs to perform the processing related to the one or more dynamic DOM portion(s) 48 corresponding to the requested instance of the page 34”; and [0031], “According to one or more aspects, apparatus and methods of rendering a page provide a web engine or other components operable to create a shared DOM that may be used by two or more instances of the page. The shared DOM includes a static DOM portion that is common to the different instances of a page, and one or more dynamic DOM portions that are unique to a respective one or more instances of the page. As such, the described aspects improve efficiency in page rendering by reusing the static DOM portion in rendering a new instance of a page that corresponds to a stored or cached DOM, which can be based on a previously processed, different instance of the same page, thereby avoiding having to create an entirely new DOM for each instance of the page”; Note that the locally cached contents may be rendered faster, which is very common practice in graphic rendering, i.e., cached visual effects will be rendered faster than non-cached content, as disclosed in this prior art).
Regarding claim 18, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 17 as outlined above. Further, Long teaches that the computing system of claim 17, wherein the one or more resources comprise an audio graph, a gesture recognition system, a face tracking system, a movement or object target tracking system, music services, location services, or any combination thereof (See Long: Fig. 2, and [0096], “Through the game feed from spectator interface 225, SLIVER server 240 may have full access to recordings and data of the game play, including player interface data as shown to individual players, spectator interface data as broadcasted, as well as other game play data compilations, in the form or one or more merged video screencast streams, or in multiple data streams that can be analyzed separately. For example, SLIVER server 240 may examine a video stream to identify or extract visual and/or audio cues that indicate an exciting moment has occurred in the game play. Exemplary visual cues include sudden, successive changes in color that indicates the occurrence of explosions, placement of multiple players within very close proximity which may indicate an intense battle scene, and the like. Exemplary audio cues include explosion sounds, changes in tone in player commentaries, and the like. SLIVER server 240 may further examine game metadata to extract metadata cues for critical game moment identification. For example, death of a player is often considered an important or critical event, and the killing moment is subsequently highlighted or replayed. In some embodiments, such visual, audio, and metadata cues may be combined, correlated or cross-compared to generate highlight metadata, which may provide more accurate gaming moment identification results than using individual cues. In yet some embodiments, highlight metadata may be equivalent to extracted visual, audio, and/or metadata cues”. Note that audio may be the music).
Regarding claim 19, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 17 as outlined above. Further, Long teaches that the computing system of claim 17, wherein the one or more resources comprise a video segmentation system that identifies portions of video content including one or more of a background, pre-defined objects, parts of users, or any combination thereof (See Long: Figs. 2-3, and [0106], “In the present disclosure, a highlight of a game play refers to media clips or recordings that feature or focus on one or more particular periods of time, or moments, during a game play, often extending over auto-determined or user-identified gaming events that are either exciting, memorable, or of special interest to viewers. A gaming event or moment is generally associated with a timestamp and/or a location with a game map of the source computer game. A highlight video may comprise screencasts captured using pre-existing virtual cameras within the game world, game play captured by inserted virtual cameras from viewing perspectives different from those shown during an initial broadcast, highlight effects, augmentations, or game video segments generated using any other suitable video processing techniques that make the highlight video attractive to spectators and the like. Exemplary highlight effects include spatial scaling, temporal scaling, visual special effects, augmentations, or any other processing techniques that make the highlight video different from a static screencast of the original game play. Examples of spatial scaling includes zoom-in, zoom-out, preview close-up, and the like. Examples of temporal scaling includes time freeze, time-lapse, slow-motion, and the like. Examples of visual special effects include bullet-time, glitch effect, exposure effect, noir effect, morphing, stitching, optical effects, and the like. Augmentations or annotations may be performed to supplement the highlight video with game metadata or other available game information. For example, annotations may be provided by the SLIVER system, by broadcasters, or even spectators; augmentation may also be provided by marking the location and field-of-view of an active player, and overlaying game statistics on a video. In different embodiments, augmentations or annotations may be provided in audio and/or visual forms”; and [0107], “FIG. 3 is a schematic diagram showing the overall architecture 300 of game and system servers for game video generation and processing, according to another embodiment of the present invention. In this embodiment, a tournament server 310 provides game data such as metadata and rendered scenes to SLIVER server 340, and in turn receives game control information as well as API handles for game or tournament configuration and other functions necessary for running a game play and capturing recordings of the game play. Such game control data may include virtual camera control signals and other game capture configuration parameters”. Note that the scene may be the game backgrounds).
Regarding claim 20, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 17 as outlined above. Further, Long teaches that the computing system of claim 17, wherein the generating the prediction is based on processing by a machine learning model that was trained to predict which XR effects a user will select (See Long: Fig. 3, and [0116], “Mapping and auto-syncing algorithms 364 may analyze the program feed in real-time, apply computer vision or other machine learning algorithms to determine the location of the current or the first point-of-view player, and identify one or more best or optimal locations of virtual cameras or virtual camera arrays to capture actions in and around this player. Similarly, in some embodiments, mapping and auto-syncing algorithms 364 may be configured to determine the location of players not shown on screen, and identify optimal virtual camera locations in and around such off-screen players”).
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Long, etc. (US 20170157512 A1) in view of Chiu (US 20060248209 A1), further in view of Shear, etc. (US 20160034305 A1), Mahan, etc. (US 20100262780 A1), and Spradlin, etc. (US 20120324197 A1).
Regarding claim 6, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 1 as outlined above. However, Long, modified by Chiu, Shear, and Mahan fails to explicitly disclose that the method of claim 1 further comprising: determining that at least one resource, of the one or more resources, has been freed; determining, based on a resource schedule, that the freed at least one resource should be unloaded; and in response, unloading the freed at least one resource.
However, Spradlin teaches that the method of claim 1 further comprising:
determining that at least one resource, of the one or more resources, has been freed (See Spradlin: Fig. 3, and [0064], “Continuing in decision block 350, if the application detects that the application is done with the allocated memory, then the application completes, else the application continues at block 360. Before completing the application may invoke the allocation interface to deallocate (or free) the previously allocated memory. If the application did not use the memory, and the host did not actually allocate the memory, then this action may simply clean up the host's stored entry related to the allocation and return control to the application”);
determining, based on a resource schedule, that the freed at least one resource should be unloaded (See Spradlin: Fig. 3, and [0066], “Continuing in block 370, the application requests direct access to the memory allocation from the host. If the host has already allocated the memory, then the host returns a pointer to the application at which the memory can be accessed. If the host has not allocated the memory or has deallocated and reused the memory, then the host allocates the memory in response to the application request, invokes the received fill function to populate the memory contents, and then returns a pointer or other means of accessing the memory to the application”. Note that “deallocating the memory may be to unload it); and
in response, unloading the freed at least one resource (See Spradlin: Fig. 3, and [0025], “Utilizing static analysis of a binary, runtime analysis of the binary's behavior, and by instrumenting the binary, it is possible to gather additional information about any of the binary's given memory allocations and the usage of those allocations. This information may then be used to drive more intelligent behavior around the loading/unloading and location of the allocation within physical memory. The memory management system provides a means to automatically annotate application memory allocations with metadata describing the potential or actual usage of the allocation itself. This analysis can be automatically executed, either statically on the binary or dynamically during runtime, without requiring any developer interaction or re-authoring of existing applications. Once performed, the analysis may be cached by the system so the host operating system knows how to treat the application in the future. The analysis may also be published for discovery by other clients, not just cached locally. In addition, it is possible for the information to be exposed to the user for optional editing, allowing an administrator or user to tailor the application's metadata and how the application host will deal with the application's memory allocations”).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Long to have the method of claim 1 further comprising: determining that at least one resource, of the one or more resources, has been freed; determining, based on a resource schedule, that the freed at least one resource should be unloaded; and in response, unloading the freed at least one resource as taught by Spradlin in order to allow the applications to use computing resources more efficiently, by preventing interfering of application with the another application accidentally or intentionally (See Spradlin: Fig. 1, and [0001], “In some cases, virtual memory allows the operating system to restrict each application to accessing a particular portion of memory, to prevent one application from interfering with the operation of another application by accidentally or intentionally modifying the other application's memory”). Long teaches a method and system that may generate and provide the highlighted game videos for users to replay by detecting the critical events in the recorded games and adding special effects on the critical events to generate the highlighted video games stored in memory or cache for users; while Spradlin teaches a system and method that may manage the memory and cache memory by freeing or deallocating the memory/cache when it is no longer needed, and unloading the content stored in the cache/memory to free up the memory for other usage. Therefore, it is obvious to one of ordinary skill in the art to modify Long by Spradlin to free and unload the memory when it is no longer needed. The motivation to modify Long by Spradlin is “Use of known technique to improve similar devices (methods, or products) in the same way”.
Regarding claim 16, Long, Chiu, Shear, and Mahan teach all the features with respect to claim 11 as outlined above. Further, Spradlin teaches that the computer-readable storage medium of claim 11, wherein the instructions, when executed, further cause the computing system to:
determine that at least one resource, of the one or more resources, has been freed (See Spradlin: Fig. 3, and [0064], “Continuing in decision block 350, if the application detects that the application is done with the allocated memory, then the application completes, else the application continues at block 360. Before completing the application may invoke the allocation interface to deallocate (or free) the previously allocated memory. If the application did not use the memory, and the host did not actually allocate the memory, then this action may simply clean up the host's stored entry related to the allocation and return control to the application”);
determine, based on a resource schedule, that the freed at least one resource should be unloaded (See Spradlin: Fig. 3, and [0066], “Continuing in block 370, the application requests direct access to the memory allocation from the host. If the host has already allocated the memory, then the host returns a pointer to the application at which the memory can be accessed. If the host has not allocated the memory or has deallocated and reused the memory, then the host allocates the memory in response to the application request, invokes the received fill function to populate the memory contents, and then returns a pointer or other means of accessing the memory to the application”. Note that “deallocating the memory may be to unload it); and
in response, unload the freed at least one resource (See Spradlin: Fig. 3, and [0025], “Utilizing static analysis of a binary, runtime analysis of the binary's behavior, and by instrumenting the binary, it is possible to gather additional information about any of the binary's given memory allocations and the usage of those allocations. This information may then be used to drive more intelligent behavior around the loading/unloading and location of the allocation within physical memory. The memory management system provides a means to automatically annotate application memory allocations with metadata describing the potential or actual usage of the allocation itself. This analysis can be automatically executed, either statically on the binary or dynamically during runtime, without requiring any developer interaction or re-authoring of existing applications. Once performed, the analysis may be cached by the system so the host operating system knows how to treat the application in the future. The analysis may also be published for discovery by other clients, not just cached locally. In addition, it is possible for the information to be exposed to the user for optional editing, allowing an administrator or user to tailor the application's metadata and how the application host will deal with the application's memory allocations”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GORDON G LIU whose telephone number is (571)270-0382. The examiner can normally be reached Monday - Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona E Faulk can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GORDON G LIU/Primary Examiner, Art Unit 2618