DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 5, 14, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hultquist et al. (U.S. Pub. No. 20130321456) in view of Dharmapurikar et al. (U.S. Pub. No. 20210245047), and Lee et al. (U.S. Pub. No. 20250328298).
Regarding claim 1, Hultquist discloses a first computing system (para 78, “Attention is now directed toward embodiments of portable devices with touch-sensitive displays. FIG. 7 is a block diagram illustrating portable multifunction device 1000 with touch-sensitive displays 1012 in accordance with some embodiments.”; also, para 38, “Various embodiments of client devices (e.g., client devices 102a-102c) may be implemented on different device types. Examples of a portable-multifunction device include the devices illustrated in FIGS. 6 through 9, such as multifunction device 1200 and multifunction device 1400.”), comprising: a display device (para 86, “Touch-sensitive display 1012 provides an input interface and an output interface between the device and a user. Display controller 1056 receives and/or sends electrical signals from/to touch screen 1012. Touch screen 1012 displays visual output to the user. The visual output may include graphics, text, icons, video, and any combination thereof (collectively termed "graphics")”; one or more memories configured to store instructions (para 80, “Memory 1002 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices.”; also, para 81, “Peripherals interface 1018 can be used to couple input and output peripherals of the device to CPU 1020 and memory 1002. The one or more processors 1020 run or execute various software programs and/or sets of instructions stored in memory 1002 to perform various functions for device 1000 and to process data.”); and one or more processors configured to execute the instructions to perform operations, the operations comprising (para 78, “Device 1000 may include memory 1002 (which may include one or more computer readable storage mediums), memory controller 1022, one or more processing units (CPU's) 1020, peripherals interface 1018, RF circuitry 1008, audio circuitry 1010, speaker 1011, microphone 1013, input/output (I/O) subsystem 1006, other input or control devices 1016, and external port 1024.”; also, para 81, “Peripherals interface 1018 can be used to couple input and output peripherals of the device to CPU 1020 and memory 1002. The one or more processors 1020 run or execute various software programs and/or sets of instructions stored in memory 1002 to perform various functions for device 1000 and to process data.”): a plurality of visual elements associated with an interactive three-dimensional (3D) scene of a geographic location (para 39, “A client device may render a map in two-dimensional or three-dimensional views. Some embodiments of a client device may display a rendered map and allow a user, system, or device providing input to manipulate a virtual camera in the map, changing the map display according to the virtual camera's position, orientation, and field-of-view.”; also, para 63, “In some embodiments the raster graphics data may be a background layer, and vector graphics may describe one or more objects overlaying the background layer. For example, a device may receive hybrid map data where the raster data describes an urban area, with houses, buildings, and other map objects. The vector graphics data may over lay roads, railroads, and other transportation paths as vector objects on top of the background layer”), to generate a plurality of rendered visual elements (para 61, “Once the hybrid map data is obtained, as indicated at 310, a client device may render a map view based upon the obtained hybrid map data for display on a display device, such as a touch-sensitive display 1012 described with regard to FIG. 9. A client device may render the raster data using raster graphics techniques 320.”; also, para 63, “Various embodiments may then combine the rendered raster graphics data with the rendered vector graphics data to create the map view 340.”); interactive 3D scene of the geographic location (para 59, “Hybrid map data may be sent in multiple sizes or file formats and may represent two-dimensional and three-dimensional map data. “; also, para 39, “A client device may render a map in two-dimensional or three-dimensional views. Some embodiments of a client device may display a rendered map and allow a user, system, or device providing input to manipulate a virtual camera in the map, changing the map display according to the virtual camera's position, orientation, and field-of-view.“) information associated with the interactive 3D scene of the geographic location to the first computing system (para 59, “Various embodiments send the hybrid map data to a client device for rendering 230. Some embodiments may utilize the communication channels of a map service operating environment, such as described in FIG. 1. For example, hybrid map data may be sent over a wide area network, such as the Internet 100 in FIG. 1, to client devices, such as client devices 102 through wireless and wired access points.”; also, para 59, “Hybrid map data may be sent in multiple sizes or file formats and may represent two-dimensional and three-dimensional map data.”; also, para 60, “A client device may obtain from a server hybrid map data containing raster graphics data and vector graphics data.”); transfer of the information associated with the interactive 3D scene of the geographic location (para 60, “A client device may obtain from a server hybrid map data containing raster graphics data and vector graphics data.”; also, para 61, “Once the hybrid map data is obtained, as indicated at 310, a client device may render a map view based upon the obtained hybrid map data for display on a display device, such as a touch-sensitive display 1012 described with regard to FIG. 9. A client device may render the raster data using raster graphics techniques 320.”); and providing, for presentation via the display device, the interactive 3D scene of the geographic location, based on the plurality of rendered visual elements (para 64, “Various embodiments may then display the rendered map view 350 on a display device, such as a touch-sensitive display 1012 described with regard to FIG. 9. The displayed map view may be a two-dimensional or three-dimensional map view of a map. It may comprise various features, such as roads, parks, lakes, paths, in various modes, such as navigation mode, night mode, pedestrian mode, current weather mode, or terrain mode.”). Hultquist does not disclose selecting, based on whether one or more rendering criteria are satisfied, at least one of the first computing system or a second computing system to perform one or more rendering operations; wherein satisfying the one or more rendering criteria comprises a size of information associated with the interactive 3D scene of the geographic location exceeding a threshold level; based on satisfying the one or more rendering criteria, selecting the second computing system to perform the one or more rendering operations; receiving, by the first computing system, a stream of the plurality of rendered visual elements from the second computing system; transitioning the one or more rendering operations from the second computing system to the first computing system.
However, in a similar field of endeavor, Dharmapurikar discloses selecting, based on whether one or more rendering criteria are satisfied (para 2, “FIG. 17A illustrates a method 1700 to determine whether the cloud gaming system is to execute game logic locally on a client or remotely on a server.”; also, 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”), at least one of the first computing system or a second computing system to perform one or more rendering operations (para 200, “FIG. 15 illustrates a cloud gaming system 1500 in which graphics processing operations can be performed using cloud, edge, or client-based compute resources.”; also, para 200, “In one embodiment the system includes a cloud-based compute, GPU, and storage system 1502 that is coupled, via a wide area network (WAN), such as the Internet, to edge GPU servers 1504 and one or more end clients (e.g., high-performance client 1520, streaming client 1530) within a client endpoint 1510, such as a home network of a user of the cloud gaming system 1500.”; also, para 203, “The cloud gaming system 1500 can enable at least a portion of the graphics processing activities for the game to be executed by one or more local GPUs 1526 on the high-performance client 1520 when a cloud-based game is played on the high-performance client 1520.”); selecting the second computing system to perform the one or more rendering operations (para 209, “When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network.”; also, para 216, “The cloud gaming client can then perform operations 1708 to launch the game on the server and stream output to the client.”); receiving, by the first computing system (para 200, “The streaming client 1530 can include a web browser or a streaming application that includes a web client engine 1532 that is used to connect with a server of the cloud-based compute, GPU, and storage system 1502 or an edge network GPU server 1504 and receive a stream of game application frames from those servers.”), a stream of the plurality of rendered visual elements from the second computing system (para 28, “Described herein is cloud gaming system in which graphics processing operations can be performed using cloud, edge, or client-based compute resources. If a client network environment includes a client having sufficient graphics processing resources to remotely execute a game, the game server stack can be downloaded by the client and the game server can be executed locally on the client. During the download of the game server stack to the client, the game can be executed by a remote server and rendered output can be streamed to the client.”; also, para 200, “The streaming client 1530 can include a web browser or a streaming application that includes a web client engine 1532 that is used to connect with a server of the cloud-based compute, GPU, and storage system 1502 or an edge network GPU server 1504 and receive a stream of game application frames from those servers.”) while transferring (para 218, “In one embodiment, when the high-performance client 1520 is capable of local execution of a cloud game, the game can initially be remotely executed and streamed to the client while the game logic and assets are downloaded to the client in the background. One or more GPU servers 1504 can initially be used to execute a selected game remotely.”; also, para 223, “While the game data is being downloaded, gameplay can begin immediately by streaming a remotely executed instance of the game until the game is ready for local execution (block 2005).”); transitioning the one or more rendering operations from the second computing system to the first computing system (para 209, “When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network.”; also, para 223, “Once the game is ready for local execution, gameplay can transition to local execution while preserving saved game state that was generated during remote execution (block 2006).”; also, para 203, “The cloud gaming system 1500 can enable at least a portion of the graphics processing activities for the game to be executed by one or more local GPUs 1526 on the high-performance client 1520 when a cloud-based game is played on the high-performance client 1520.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hultquist's invention of a client computing device having a display device, memory storing instructions and processing units that execute them, which renders visual elements of an interactive two-dimensional or three-dimensional map view of a geographic location from obtained hybrid map data and displays that map view on the display device, with the features of Dharmapurikar's invention of a system in which graphics processing operations are performed using either the compute resources of a graphics processing unit server or the compute resources of a client, of determining whether that work is to be executed locally on the client or remotely on the server according to whether stated capabilities exceed a threshold, of the server performing the graphics processing and transmitting the rendered output to the client when remote execution is enabled, of streaming that remotely rendered output to the client while the underlying data is downloaded to the client in the background, and of transitioning to local execution on the client's own graphics processors once the client is ready for it. The combination would have been obvious because Hultquist already recognizes that the client device that must render its map view has finite rendering and communication capability, and Hultquist already adapts its behavior to that capability by selecting a complexity threshold value from client capability and link bandwidth information, but Hultquist leaves the client to render every map feature itself and therefore has no answer when the client cannot render the map view acceptably or when the map data has not yet arrived. Dharmapurikar answers exactly that problem in the same client and server graphics setting, by naming a graphics processing unit server and a client as the two systems that may carry the graphics processing operations, by testing the client against a threshold to choose between them, and by keeping a usable image in front of the user through a stream of server-rendered output while the data travels to the client. A person of ordinary skill applying Dharmapurikar's remote and local execution determination to Hultquist's map client would have obtained the predictable result that the map view is rendered by whichever of the two systems the criteria select, that the user continues to see the geographic scene while the hybrid map data is still being transferred, and that the rendering moves to the client once Hultquist's client has obtained that data.
Lee discloses wherein satisfying the one or more rendering criteria comprises a size of information (para 332, “For example, the cloud policy manager 400 may consider a data size of the first signage-related information when determining the processing location of the first signage-related information [S141].”; also, para 335, “For example, when the type of data for the first signage-related information is 3D, the data size may be larger than when the type is 2D.”; also, para 337, “When the data size of the first signage-related information is greater than or equal to a predetermined value, the cloud policy manager 400 may determine the processing location of the first signage-related information as the cloud server 200 [S142].”; also, para 338, “When the data size of the first signage-related information is less than the predetermined value, the cloud policy manager 400 may determine the processing location of the first signage-related information as the vehicle 100 [S143].”) exceeding a threshold level; based on satisfying the one or more rendering criteria (para 337, “When the data size of the first signage-related information is greater than or equal to a predetermined value, the cloud policy manager 400 may determine the processing location of the first signage-related information as the cloud server 200 [S142].”; also, para 338, “When the data size of the first signage-related information is less than the predetermined value, the cloud policy manager 400 may determine the processing location of the first signage-related information as the vehicle 100 [S143].”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar, in which a client device selects between itself and a server to render an interactive three-dimensional scene of a geographic location and receives a stream of server-rendered output until the scene data has finished transferring, with the features of Lee's invention of comparing the data size of the information to a predetermined value and assigning the work to the cloud server when the data size is at or above that value and to the local device when it is below. The combination would have been obvious because the combined system already conditions the choice between the two systems on a threshold comparison, and its remaining exposure is the case Dharmapurikar's capability test does not reach, namely a capable client facing a body of scene information so large that waiting for it to arrive would leave the user with nothing to look at. Lee addresses that case with the measurement that matters to it, expressly noting that the data size is larger for three-dimensional content than for two-dimensional content, and expressly explaining that assigning the larger body of information to the cloud server relieves the local device of it. A person of ordinary skill would have used the data size of the geographic scene's information as one of the combined system's rendering criteria, with the predictable result that the server is selected to render whenever that quantity exceeds the threshold, which is precisely the circumstance in which the client's own transfer of that information will take longest.
Regarding claim 2, Hultquist as modified by Dharmapurikar and Lee discloses the first computing system of claim 1, wherein Dharmapurikar further discloses the second computing system has a higher processing power than the first computing system (para 200, “Games that require a high level of graphics processing capability can be played on a streaming client 1530, such as a thin client with limited graphics processing capabilities relative to a high-performance computing device.”; also, para 202, “The GPU server 1504 can include a set of high-performance GPUs 1508 that can be used to execute a game server stack 1506.”), when the one or more rendering criteria are satisfied (para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”; also, para 222, “The method 2000 additionally includes operations to determine whether a locally available client is capable of local execution. A locally available client may be considered capable of local execution if the client has a graphics processor having sufficient processing capability for the selected game and sufficient available storage for the game. If it is determined that a locally available client is capable of local execution (block 2003), the method 2000 can proceed to operations to download the game at the determined local client (block 2004).”), selecting the first computing system to perform the one or more rendering operations (para 203, “The cloud gaming system 1500 can enable at least a portion of the graphics processing activities for the game to be executed by one or more local GPUs 1526 on the high-performance client 1520 when a cloud-based game is played on the high-performance client 1520.”); and when the one or more rendering criteria are not satisfied (para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”; also, para 215, “When remote execution is enabled, the cloud gaming client can perform operations 1704 to cause the server to map selected client resources at the server to enable the server to access any client-based resources that will be accessed by the server.”), selecting the second computing system to perform the one or more rendering operations (para 209, “When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network.”), and receiving, from the second computing system, the plurality of rendered visual elements from the second computing system (para 200, “The streaming client 1530 can include a web browser or a streaming application that includes a web client engine 1532 that is used to connect with a server of the cloud-based compute, GPU, and storage system 1502 or an edge network GPU server 1504 and receive a stream of game application frames from those servers.”; also, para 209, “When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hultquist's invention of a client computing device that renders and displays an interactive map view of a geographic location, with the features of Dharmapurikar's invention of a thin client having limited graphics processing capability relative to a set of high-performance server graphics processors, of enabling local execution when the client capability test is met, and of performing the graphics processing at the server and transmitting the rendered output to the client when it is not. The combination would have been obvious because Hultquist expressly adapts what it sends a client to that client's rendering capability, so a person of ordinary skill working with Hultquist already had in view the fact that client devices differ in graphics capability and that some cannot render what the map service holds. Dharmapurikar supplies the corresponding hardware premise and the two-way outcome, keeping the work on the client while the client can carry it and moving it to the more capable server graphics processors when it cannot, with the predictable result that the map client renders its own map view whenever it is able and otherwise receives a rendered map view generated on the more powerful server.
Regarding claim 5, Hultquist as modified by Dharmapurikar and Lee discloses the first computing system of claim 1, wherein Dharmapurikar further discloses the one or more rendering criteria includes whether a processing power of the first computing system meets processing power criteria (paragraph 212, “As shown in FIG. 17A, a cloud gaming client can perform operations 1701 to determine client and GPU capabilities of the gaming client. The determined client capabilities can include the total amount of system memory on the client and an amount of system memory that is available to client processes or applications associated with the cloud gaming service. The determined GPU capabilities can include the amount of local and system memory available to the graphics processor, the memory bandwidth, a fill rate of the GPU, such as the texel fill rate and/or the pixel fill rate.”; also, para 222, “A locally available client may be considered capable of local execution if the client has a graphics processor having sufficient processing capability for the selected game and sufficient available storage for the game.”), in response to determining the processing power of the first computing system meets the processing power criteria (para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”; also, para 222, “The method 2000 additionally includes operations to determine whether a locally available client is capable of local execution. A locally available client may be considered capable of local execution if the client has a graphics processor having sufficient processing capability for the selected game and sufficient available storage for the game. If it is determined that a locally available client is capable of local execution (block 2003), the method 2000 can proceed to operations to download the game at the determined local client (block 2004).”), selecting the first computing system to perform the one or more rendering operations (para 203, “The cloud gaming system 1500 can enable at least a portion of the graphics processing activities for the game to be executed by one or more local GPUs 1526 on the high-performance client 1520 when a cloud-based game is played on the high-performance client 1520.”), and in response to determining the processing power of the first computing system does not meet the processing power criteria (para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”; also, para 215, “When remote execution is enabled, the cloud gaming client can perform operations 1704 to cause the server to map selected client resources at the server to enable the server to access any client-based resources that will be accessed by the server.”), selecting the second computing system to perform the one or more rendering operations (para 209, ” When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network.”; also, para 216, “The cloud gaming client can then perform operations 1708 to launch the game on the server and stream output to the client.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hultquist's invention of a client computing device that renders and displays an interactive map view of a geographic location, with the features of Dharmapurikar's invention of determining the graphics processing capabilities of the client and enabling local execution only for clients whose graphics processing capabilities exceed a threshold, with remote execution otherwise. The combination would have been obvious because Hultquist already collects and acts on client hardware information, describing that a request may carry central processing unit and graphics processing unit capabilities, but Hultquist uses that information only to pick an encoding and never to decide which machine renders. Dharmapurikar takes the same class of client hardware measurement and makes it the test for whether the client renders at all. A person of ordinary skill would have applied Dharmapurikar's capability test to the client capability information Hultquist already gathers, with the predictable result that a map client whose graphics processing capability is sufficient renders the geographic scene itself and one whose capability is insufficient has it rendered at the server.
Regarding claim 14, Hultquist discloses a computer-implemented method, comprising (para 60, “A client device, such as client device 102 in FIG. 1, may obtain from a server, such as a map service 130, hybrid map data and render the hybrid map data for display. According to various embodiments FIG. 3 outlines a method for rendering hybrid map data according to some embodiments.”): a plurality of visual elements associated with an interactive three-dimensional (3D) scene of a geographic location (para 39, “A client device may render a map in two-dimensional or three-dimensional views. Some embodiments of a client device may display a rendered map and allow a user, system, or device providing input to manipulate a virtual camera in the map, changing the map display according to the virtual camera's position, orientation, and field-of-view.”; also, para 63, “In some embodiments the raster graphics data may be a background layer, and vector graphics may describe one or more objects overlaying the background layer. For example, a device may receive hybrid map data where the raster data describes an urban area, with houses, buildings, and other map objects. The vector graphics data may over lay roads, railroads, and other transportation paths as vector objects on top of the background layer.”), to generate a plurality of rendered visual elements (para 61, “Once the hybrid map data is obtained, as indicated at 310, a client device may render a map view based upon the obtained hybrid map data for display on a display device, such as a touch-sensitive display 1012 described with regard to FIG. 9. A client device may render the raster data using raster graphics techniques 320.”; also, para 63, “Various embodiments may then combine the rendered raster graphics data with the rendered vector graphics data to create the map view 340.”), interactive 3D scene of the geographic location (para 59, “Hybrid map data may be sent in multiple sizes or file formats and may represent two-dimensional and three-dimensional map data.”; also, para 39, “A client device may render a map in two-dimensional or three-dimensional views. Some embodiments of a client device may display a rendered map and allow a user, system, or device providing input to manipulate a virtual camera in the map, changing the map display according to the virtual camera's position, orientation, and field-of-view.”) information associated with the interactive 3D scene of the geographic location to the first computing system (para 59, “Various embodiments send the hybrid map data to a client device for rendering 230. Some embodiments may utilize the communication channels of a map service operating environment, such as described in FIG. 1. For example, hybrid map data may be sent over a wide area network, such as the Internet 100 in FIG. 1, to client devices, such as client devices 102 through wireless and wired access points.”; also, para 59, “Hybrid map data may be sent in multiple sizes or file formats and may represent two-dimensional and three-dimensional map data.”; also, para 60, “A client device may obtain from a server hybrid map data containing raster graphics data and vector graphics data.”); transfer of the information associated with the interactive 3D scene of the geographic location (para 60, “A client device may obtain from a server hybrid map data containing raster graphics data and vector graphics data.”; also, para 61, “Once the hybrid map data is obtained, as indicated at 310, a client device may render a map view based upon the obtained hybrid map data for display on a display device, such as a touch-sensitive display 1012 described with regard to FIG. 9. A client device may render the raster data using raster graphics techniques 320.”); and providing, for presentation via a display device of the first computing system, the interactive 3D scene of the geographic location, based on the plurality of rendered visual elements (para 86, “Touch-sensitive display 1012 provides an input interface and an output interface between the device and a user. Display controller 1056 receives and/or sends electrical signals from/to touch screen 1012. Touch screen 1012 displays visual output to the user. The visual output may include graphics, text, icons, video, and any combination thereof (collectively termed "graphics").”; also, para 64, “Various embodiments may then display the rendered map view 350 on a display device, such as a touch-sensitive display 1012 described with regard to FIG. 9. The displayed map view may be a two-dimensional or three-dimensional map view of a map. It may comprise various features, such as roads, parks, lakes, paths, in various modes, such as navigation mode, night mode, pedestrian mode, current weather mode, or terrain mode”). Hultquist does not disclose selecting, by a first computing system comprising one or more processors, based on whether one or more rendering criteria are satisfied, at least one of the first computing system or a second computing system to perform one or more rendering operations, wherein satisfying the one or more rendering criteria comprises a size of information associated with exceeding a threshold level; based on satisfying the one or more rendering criteria, selecting the second computing system to perform the one or more rendering operations; receiving, by the first computing system, a stream of the plurality of rendered visual elements from the second computing system while transferring; transitioning the one or more rendering operations from the second computing system to the first computing system.
However, in a similar field of endeavor, Dharmapurikar discloses selecting, by a first computing system comprising one or more processors (para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”; also, para 212, “As shown in FIG. 17A, a cloud gaming client can perform operations 1701 to determine client and GPU capabilities of the gaming client. The determined client capabilities can include the total amount of system memory on the client and an amount of system memory that is available to client processes or applications associated with the cloud gaming service. The determined GPU capabilities can include the amount of local and system memory available to the graphics processor, the memory bandwidth, a fill rate of the GPU, such as the texel fill rate and/or the pixel fill rate.”), based on whether one or more rendering criteria are satisfied (para 211, “FIG. 17A illustrates a method 1700 to determine whether the cloud gaming system is to execute game logic locally on a client or remotely on a server.”; also, para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”), at least one of the first computing system or a second computing system to perform one or more rendering operations (para 200, “FIG. 15 illustrates a cloud gaming system 1500 in which graphics processing operations can be performed using cloud, edge, or client-based compute resources.”; also, para 200, “In one embodiment the system includes a cloud-based compute, GPU, and storage system 1502 that is coupled, via a wide area network (WAN), such as the Internet, to edge GPU servers 1504 and one or more end clients (e.g., high-performance client 1520, streaming client 1530) within a client endpoint 1510, such as a home network of a user of the cloud gaming system 1500.”; also, para 203, “The cloud gaming system 1500 can enable at least a portion of the graphics processing activities for the game to be executed by one or more local GPUs 1526 on the high-performance client 1520 when a cloud-based game is played on the high-performance client 1520.”), selecting the second computing system to perform the one or more rendering operations (para 209, “When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network”; also, para 216, “The cloud gaming client can then perform operations 1708 to launch the game on the server and stream output to the client.”); receiving, by the first computing system (para 200, “The streaming client 1530 can include a web browser or a streaming application that includes a web client engine 1532 that is used to connect with a server of the cloud-based compute, GPU, and storage system 1502 or an edge network GPU server 1504 and receive a stream of game application frames from those servers.”), a stream of the plurality of rendered visual elements from the second computing system (para 28, “Described herein is cloud gaming system in which graphics processing operations can be performed using cloud, edge, or client-based compute resources. If a client network environment includes a client having sufficient graphics processing resources to remotely execute a game, the game server stack can be downloaded by the client and the game server can be executed locally on the client. During the download of the game server stack to the client, the game can be executed by a remote server and rendered output can be streamed to the client.”; also, para 200, “The streaming client 1530 can include a web browser or a streaming application that includes a web client engine 1532 that is used to connect with a server of the cloud-based compute, GPU, and storage system 1502 or an edge network GPU server 1504 and receive a stream of game application frames from those servers.”) while transferring (para 218, “In one embodiment, when the high-performance client 1520 is capable of local execution of a cloud game, the game can initially be remotely executed and streamed to the client while the game logic and assets are downloaded to the client in the background. One or more GPU servers 1504 can initially be used to execute a selected game remotely.”; also, para 223, “While the game data is being downloaded, gameplay can begin immediately by streaming a remotely executed instance of the game until the game is ready for local execution (block 2005).”); transitioning the one or more rendering operations from the second computing system to the first computing system (para 209, “When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network.”; also, para 223, “Once the game is ready for local execution, gameplay can transition to local execution while preserving saved game state that was generated during remote execution (block 2006).”; also, para 203, “The cloud gaming system 1500 can enable at least a portion of the graphics processing activities for the game to be executed by one or more local GPUs 1526 on the high-performance client 1520 when a cloud-based game is played on the high-performance client 1520”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hultquist's invention of a method in which a client computing device obtains hybrid map data from a server, renders visual elements of an interactive two-dimensional or three-dimensional map view of a geographic location, and displays that map view on a display device, with the features of Dharmapurikar's invention of a system in which graphics processing operations are performed using either the compute resources of a graphics processing unit server or the compute resources of a client, of determining whether that work is to be executed locally on the client or remotely on the server according to whether stated capabilities exceed a threshold, of the server performing the graphics processing and transmitting the rendered output to the client when remote execution is enabled, of streaming that remotely rendered output to the client while the underlying data is downloaded to the client in the background, and of transitioning to local execution on the client's own graphics processors once the client is ready for it. The combination would have been obvious because Hultquist already recognizes that the client device that must render its map view has finite rendering and communication capability, and Hultquist already adapts its behavior to that capability by selecting a complexity threshold value from client capability and link bandwidth information, but Hultquist leaves the client to render every map feature itself and therefore has no answer when the client cannot render the map view acceptably or when the map data has not yet arrived. Dharmapurikar answers exactly that problem in the same client and server graphics setting, by naming a graphics processing unit server and a client as the two systems that may carry the graphics processing operations, by testing the client against a threshold to choose between them, and by keeping a usable image in front of the user through a stream of server-rendered output while the data travels to the client. A person of ordinary skill applying Dharmapurikar's remote and local execution determination to Hultquist's map client would have obtained the predictable result that the map view is rendered by whichever of the two systems the criteria select, that the user continues to see the geographic scene while the hybrid map data is still being transferred, and that the rendering moves to the client once Hultquist's client has obtained that data.
Lee discloses wherein satisfying the one or more rendering criteria comprises a size of information (para 332, “For example, the cloud policy manager 400 may consider a data size of the first signage-related information when determining the processing location of the first signage-related information [S141].”; also, para 335, “For example, when the type of data for the first signage-related information is 3D, the data size may be larger than when the type is 2D.”); exceeding a threshold level (para 337, “When the data size of the first signage-related information is greater than or equal to a predetermined value, the cloud policy manager 400 may determine the processing location of the first signage-related information as the cloud server 200 [S142].”; also, para 338, “When the data size of the first signage-related information is less than the predetermined value, the cloud policy manager 400 may determine the processing location of the first signage-related information as the vehicle 100 [S143].”); based on satisfying the one or more rendering criteria (para 337, “When the data size of the first signage-related information is greater than or equal to a predetermined value, the cloud policy manager 400 may determine the processing location of the first signage-related information as the cloud server 200 [S142].”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar, in which a method selects between a client device and a server to render an interactive three-dimensional scene of a geographic location and streams server-rendered output until the scene data has finished transferring, with the features of Lee's invention of comparing the data size of the information to a predetermined value and assigning the work to the cloud server when the data size is at or above that value and to the local device when it is below. The combination would have been obvious because the combined system already conditions the choice between the two systems on a threshold comparison, and its remaining exposure is the case Dharmapurikar's capability test does not reach, namely a capable client facing a body of scene information so large that waiting for it to arrive would leave the user with nothing to look at. Lee addresses that case with the measurement that matters to it, expressly noting that the data size is larger for three-dimensional content than for two-dimensional content, and expressly explaining that assigning the larger body of information to the cloud server relieves the local device of it. A person of ordinary skill would have used the data size of the geographic scene's information as one of the combined system's rendering criteria, with the predictable result that the server is selected to render whenever that quantity exceeds the threshold, which is precisely the circumstance in which the client's own transfer of that information will take longest.
Regarding claim 17, Hultquist as modified by Dharmapurikar and Lee discloses the computer-implemented method of claim 14, wherein Dharmapurikar further discloses the one or more rendering criteria includes whether a processing power of the first computing system meets processing power criteria, and the method comprises (para 212, “As shown in FIG. 17A, a cloud gaming client can perform operations 1701 to determine client and GPU capabilities of the gaming client. The determined client capabilities can include the total amount of system memory on the client and an amount of system memory that is available to client processes or applications associated with the cloud gaming service. The determined GPU capabilities can include the amount of local and system memory available to the graphics processor, the memory bandwidth, a fill rate of the GPU, such as the texel fill rate and/or the pixel fill rate.”; also, para 222, “A locally available client may be considered capable of local execution if the client has a graphics processor having sufficient processing capability for the selected game and sufficient available storage for the game.”): in response to determining the processing power of the first computing system meets the processing power criteria (para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”; also, para 222, “The method 2000 additionally includes operations to determine whether a locally available client is capable of local execution. A locally available client may be considered capable of local execution if the client has a graphics processor having sufficient processing capability for the selected game and sufficient available storage for the game. If it is determined that a locally available client is capable of local execution (block 2003), the method 2000 can proceed to operations to download the game at the determined local client (block 2004).”), selecting the first computing system to perform the one or more rendering operations (para 203, “The cloud gaming system 1500 can enable at least a portion of the graphics processing activities for the game to be executed by one or more local GPUs 1526 on the high-performance client 1520 when a cloud-based game is played on the high-performance client 1520.”), and in response to determining the processing power of the first computing system does not meet the processing power criteria (para 213, “The cloud gaming client can then perform operations 1703 to determine whether local or remote execution is to be used for game execution. The local or remote determination can be performed on a per-game or per-application basis. In one embodiment, local execution is enabled for clients for which at least the GPU capabilities exceed a threshold. In one embodiment, local execution is enabled for clients for which both the GPU capabilities and network capabilities exceed a threshold.”; also, para 215, “When remote execution is enabled, the cloud gaming client can perform operations 1704 to cause the server to map selected client resources at the server to enable the server to access any client-based resources that will be accessed by the server.”), selecting the second computing system to perform the one or more rendering operations (para 209,” When the game application is executed on one or more cloud servers or on an edge server, graphics processing for the game is performed on the server or servers and the rendered output is transmitted to the cloud gaming client via the network.”; also, para 216, “The cloud gaming client can then perform operations 1708 to launch the game on the server and stream output to the client.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hultquist's invention of a method in which a client computing device renders and displays an interactive map view of a geographic location, with the features of Dharmapurikar's invention of determining the graphics processing capabilities of the client and enabling local execution only for clients whose graphics processing capabilities exceed a threshold, with remote execution otherwise. The combination would have been obvious because Hultquist already gathers the client's central processing unit and graphics processing unit capabilities and uses them only to pick an encoding, while Dharmapurikar makes that same class of measurement the test for whether the client renders at all, yielding the predictable result that a sufficiently capable map client renders the geographic scene itself and an insufficiently capable one has it rendered at the server.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hultquist et al. (U.S. Pub. No. 20130321456) as modified by Dharmapurikar et al. (U.S. Pub. No. 20210245047), and Lee et al. (U.S. Pub. No. 20250328298), further in view of Kulkarni et al. (U.S. Doc. No. 10764403).
Regarding claim 6, Hultquist as modified by Dharmapurikar and Lee discloses the first computing system of claim 1, one or more rendering criteria includes whether the first computing system stores a 3D graphics library, in response to determining the first computing system stores the 3D graphics library, selecting the first computing system to perform the one or more rendering operations, and in response to determining the first computing system does not store the 3D graphics library, selecting the second computing system to perform the one or more rendering operations.
However, in a similar field of endeavor, Kulkarni discloses wherein the one or more rendering criteria includes whether the first computing system stores a 3D graphics library (col 6, “Row 317 sets forth a feature capability attribute for a feature capability, such as a capability of a client device to support use of Web Graphics Library (WebGL). WebGL is a JavaScript API that enables a web browser to provide interactive two-dimensional (2D) or three-dimensional (3D) graphics without requiring a plug-in to be installed in the web browser specifically to support graphics.”), in response to determining the first computing system stores the 3D graphics library (col 6, “A client device that does support WebGL, as in any of columns 325 through 327, may be given one of client resource ratings 304 to 306, equal to four, five, or six, respectively, as may be determined according to other criteria, as described herein.”), selecting the first computing system to perform the one or more rendering operations (col 8, “Row 515 identifies a client device as being "dev-workstation-1" having a resource rating value of five ("high"), to operate using no SSR, with no transformation.”; also, col 11, “Client devices identified by the webserver as not requiring SSR (such as being placed in a No-SSR category) can be provided untransformed content, which need not include SSR.”; also, col 7, “In columns 425, 426, and 427, for client resource ratings 304, 305, and 306, respectively, equal to four, five, and six, respectively, no transformation is specified, as higher performance client devices may be capable of satisfactorily processing and presenting content in its original form without transformation.”), and in response to determining the first computing system does not store the 3D graphics library (col 6, “A client device that does not support WebGL, as in any of columns 322 through 324, may be given one of client resource ratings 301 to 303, equal to one, two, or three, respectively, as may be determined according to other criteria, as described herein.”), selecting the second computing system to perform the one or more rendering operations (col 8, “The heading of column 523 describes the information below in column 523 as being a mode of operation for providing information to the respective client device (e.g., using server-side rendering (SSR) or not using server-side rendering (no-SSR). The heading of column 524 describes the preferred rendering scheme to be used for the respective client device.”; also, col 8, “Row 514 identifies a client device as being "x-desktop-thin-client" having a resource rating value of one ("very low"), to operate using SSR, with a full transformation to be used to accommodate it.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hultquist's invention of a client computing device that renders and displays an interactive map view of a geographic location, with the features of Kulkarni's invention of testing whether a client device supports the Web Graphics Library, of sorting client devices that do not support it into the lower client resource ratings and those that do support it into the higher ones, of operating the lower-rated client devices using server-side rendering, and of operating the higher-rated client devices without server-side rendering so that they are provided untransformed content. The combination would have been obvious because Hultquist's client device is the only machine that renders anything, and Hultquist expressly depends on that client holding an implementation of Open Graphics Library in order to render the vector half of its hybrid map data, yet Hultquist never asks whether a given client actually holds one. Hultquist collects the client's operating system, device type and driver information only to choose which encoding to send, so a client without the graphics software receives data it cannot draw. Kulkarni asks that question directly in the same client and server setting, testing for the graphics library by name and answering a negative result by moving the rendering to the server. A person of ordinary skill would have applied Kulkarni's test to the client capability information Hultquist already gathers, with the predictable result that a map client holding the graphics library renders the geographic scene itself and a map client lacking it has that scene rendered at the server.
Claim(s) 13 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hultquist et al. (U.S. Pub. No. 20130321456) as modified by Dharmapurikar et al. (U.S. Pub. No. 20210245047), and Lee et al. (U.S. Pub. No. 20250328298), further in view of Dietrich (U.S. Pub. No. 20240226729).
Regarding claim 13, Hultquist as modified by Dharmapurikar and Lee discloses the first computing system of claim 1, wherein Hultquist discloses interactive 3D scene of the geographic location (para 39, “A client device may render a map in two-dimensional or three-dimensional views. Some embodiments of a client device may display a rendered map and allow a user, system, or device providing input to manipulate a virtual camera in the map, changing the map display according to the virtual camera's position, orientation, and field-of-view.”; also, para 63, “In some embodiments the raster graphics data may be a background layer, and vector graphics may describe one or more objects overlaying the background layer. For example, a device may receive hybrid map data where the raster data describes an urban area, with houses, buildings, and other map objects. The vector graphics data may over lay roads, railroads, and other transportation paths as vector objects on top of the background layer.”) , one or more rendering criteria includes whether a particular frame includes at least some content having a complexity level greater than a threshold complexity level, in response to determining the particular frame associated with the interactive 3D scene of the geographic location includes at least some content having the complexity level greater than the threshold complexity level, selecting the first computing system to perform the one or more rendering operations for a first portion of the particular frame and selecting the second computing system to perform the one or more rendering operations for a second portion of the particular frame, and content in the first portion of the particular frame does not have the complexity level greater than the threshold complexity level and content in the second portion of the particular frame has the complexity level greater than the threshold complexity level.
However, in a similar field of endeavor, Dietrich discloses the one or more rendering criteria includes whether a particular frame (para 55, “FIG. 3 depicts a simplified relative remote rendering prioritization graph 300 for the objects within the virtual environment 200 of FIG. 2 based on their relative complexity and proximity to the virtual viewpoint of the user within that virtual environment.”) includes at least some content having a complexity level greater than a threshold complexity level (para 55, “As used herein, complexity of an object may refer to a quantity of components associated with an object (e.g., surfaces, texture maps, bump maps, lighting maps, specular components, etc.). In addition, an individual component may be associated with a relative degree of complexity based on (as non-limiting examples) a quantity of coordinate points, triangles, curves, splines, or other primitives associated with the component.”; also, para 63, “In various embodiments, determination of the set of remote rendering prioritization values may be based on one or more of criteria 450. In the depicted embodiment, criteria 450 include network conditions criterion 450-1, indicating conditions for one or more networks communicatively coupled to the processor-based server system (such as may indicate parameters related to bandwidth, latency, etc.); spatial proximity criterion 450-2, indicating a spatial proximity between a virtual viewpoint in the virtual environment and one or more components in the virtual environment; component complexity criterion 450-3, indicating a degree of complexity associated with one or more components in the virtual environment; recent motion criterion 450-4, indicating a degree of recent motion associated with one or more components in the virtual environment; predicted motion criterion 450-5, indicating a degree of predicted motion associated with one or more components of the virtual environment; and prior environment criterion 450-6, based on an analysis of identical or similar components that have been rendered wholly or in part by the processor-based server system, either in the current virtual environment or one or more prior virtual environments.”; also, para 57, “In certain embodiments, the environment rendering threshold may be determined as a threshold for assigned remote rendering prioritization values, such that objects assigned a remote rendering prioritization value that meets or exceeds the environment rendering threshold are rendered by the server, with objects assigned a remote rendering prioritization value below the environment rendering threshold being rendered by the local client device.”), in response to determining the particular frame associated with the interactive 3D scene of the geographic location includes at least some content having the complexity level greater than the threshold complexity level (para 57, “In certain embodiments, the environment rendering threshold may be determined as a threshold for assigned remote rendering prioritization values, such that objects assigned a remote rendering prioritization value that meets or exceeds the environment rendering threshold are rendered by the server, with objects assigned a remote rendering prioritization value below the environment rendering threshold being rendered by the local client device.”), selecting the first computing system to perform the one or more rendering operations for a first portion of the particular frame (para 60, “In certain embodiments, after determining the set of remote rendering prioritization values for components of the virtual environment 200, the server may generate a partial scene of the virtual environment by rendering a set of first components and transmitting the rendered set of first components to the client device, such as for compositing with a set of second components of the virtual environment that are rendered by the client device itself”) and selecting the second computing system to perform the one or more rendering operations for a second portion of the particular frame (para 60, “In certain embodiments, after determining the set of remote rendering prioritization values for components of the virtual environment 200, the server may generate a partial scene of the virtual environment by rendering a set of first components and transmitting the rendered set of first components to the client device, such as for compositing with a set of second components of the virtual environment that are rendered by the client device itself.”), and content in the first portion of the particular frame does not have the complexity level greater than the threshold complexity level (para 59, “For example, environment rendering threshold 305 represents relatively favorable network conditions, such that the remote server may render all of the enumerated objects within the virtual environment 200 other than box objects 210, each of which is associated with a low degree of complexity.”) and content in the second portion of the particular frame has the complexity level greater than the threshold complexity level (para 59, “For example, environment rendering threshold 305 represents relatively favorable network conditions, such that the remote server may render all of the enumerated objects within the virtual environment 200 other than box objects 210, each of which is associated with a low degree of complexity.”; also, para 57, “In certain embodiments, the environment rendering threshold may be determined as a threshold for assigned remote rendering prioritization values, such that objects assigned a remote rendering prioritization value that meets or exceeds the environment rendering threshold are rendered by the server, with objects assigned a remote rendering prioritization value below the environment rendering threshold being rendered by the local client device.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar and Lee, in which a client device selects between itself and a server to render an interactive three-dimensional scene of a geographic location, with the features of Dietrich's invention of measuring the complexity of an object by the quantity of its components and primitives, of assigning each object a value determined from a component complexity criterion among others, of assigning objects within one scene to the server or to the local client device according to whether that value meets or exceeds a rendering threshold, and of the server rendering a first set of components and transmitting them to the client for compositing with a second set rendered by the client itself. The combination would have been obvious because the combined system decides where to render an entire scene, which forces an all-or-nothing choice even when only part of a map frame is demanding, and Hultquist's own map frames are expressly heterogeneous, combining a raster background layer with vector objects overlaid on it. Dietrich addresses that heterogeneity directly in the same client and server rendering setting, by splitting a single frame along an object-level complexity threshold and compositing the two halves at the client. A person of ordinary skill would have applied that split to the combined system's map frames, with the predictable result that the more complex content of a frame is rendered at the server while the less complex content is rendered at the client and the two are combined for display.
Regarding claim 19, Hultquist as modified by Dharmapurikar and Lee discloses the computer-implemented method of claim 14, wherein the associated with the interactive 3D scene of the geographic location (para 39, “A client device may render a map in two-dimensional or three-dimensional views. Some embodiments of a client device may display a rendered map and allow a user, system, or device providing input to manipulate a virtual camera in the map, changing the map display according to the virtual camera's position, orientation, and field-of-view.”; also, para 63, “In some embodiments the raster graphics data may be a background layer, and vector graphics may describe one or more objects overlaying the background layer. For example, a device may receive hybrid map data where the raster data describes an urban area, with houses, buildings, and other map objects. The vector graphics data may over lay roads, railroads, and other transportation paths as vector objects on top of the background layer”) one or more rendering criteria includes whether a particular frame, includes at least some content having a complexity level greater than a threshold complexity level, and the method comprises: in response to determining the particular frame associated with the interactive 3D scene of the geographic location includes at least some content having the complexity level greater than the threshold complexity level, selecting the first computing system to perform the one or more rendering operations for a first portion of the particular frame and selecting the second computing system to perform the one or more rendering operations for a second portion of the particular frame, and content in the first portion of the particular frame does not have the complexity level greater than the threshold complexity level and content in the second portion of the particular frame has the complexity level greater than the threshold complexity level.
However, in a similar field of endeavor, Dietrich discloses one or more rendering criteria includes whether a particular frame (para 55, “FIG. 3 depicts a simplified relative remote rendering prioritization graph 300 for the objects within the virtual environment 200 of FIG. 2 based on their relative complexity and proximity to the virtual viewpoint of the user within that virtual environment.”), includes at least some content having a complexity level greater than a threshold complexity level, and the method comprises (para 55, “As used herein, complexity of an object may refer to a quantity of components associated with an object (e.g., surfaces, texture maps, bump maps, lighting maps, specular components, etc.). In addition, an individual component may be associated with a relative degree of complexity based on (as non-limiting examples) a quantity of coordinate points, triangles, curves, splines, or other primitives associated with the component.”; also, para 63, “In various embodiments, determination of the set of remote rendering prioritization values may be based on one or more of criteria 450. In the depicted embodiment, criteria 450 include network conditions criterion 450-1, indicating conditions for one or more networks communicatively coupled to the processor-based server system (such as may indicate parameters related to bandwidth, latency, etc.); spatial proximity criterion 450-2, indicating a spatial proximity between a virtual viewpoint in the virtual environment and one or more components in the virtual environment; component complexity criterion 450-3, indicating a degree of complexity associated with one or more components in the virtual environment; recent motion criterion 450-4, indicating a degree of recent motion associated with one or more components in the virtual environment; predicted motion criterion 450-5, indicating a degree of predicted motion associated with one or more components of the virtual environment; and prior environment criterion 450-6, based on an analysis of identical or similar components that have been rendered wholly or in part by the processor-based server system, either in the current virtual environment or one or more prior virtual environments.”; also, para 57, “In certain embodiments, the environment rendering threshold may be determined as a threshold for assigned remote rendering prioritization values, such that objects assigned a remote rendering prioritization value that meets or exceeds the environment rendering threshold are rendered by the server, with objects assigned a remote rendering prioritization value below the environment rendering threshold being rendered by the local client device.”): in response to determining the particular frame associated with the interactive 3D scene of the geographic location includes at least some content having the complexity level greater than the threshold complexity level (para 57, “In certain embodiments, the environment rendering threshold may be determined as a threshold for assigned remote rendering prioritization values, such that objects assigned a remote rendering prioritization value that meets or exceeds the environment rendering threshold are rendered by the server, with objects assigned a remote rendering prioritization value below the environment rendering threshold being rendered by the local client device.”), selecting the first computing system to perform the one or more rendering operations for a first portion of the particular frame (para 60, “In certain embodiments, after determining the set of remote rendering prioritization values for components of the virtual environment 200, the server may generate a partial scene of the virtual environment by rendering a set of first components and transmitting the rendered set of first components to the client device, such as for compositing with a set of second components of the virtual environment that are rendered by the client device itself.”) and selecting the second computing system to perform the one or more rendering operations for a second portion of the particular frame (para 60, “In certain embodiments, after determining the set of remote rendering prioritization values for components of the virtual environment 200, the server may generate a partial scene of the virtual environment by rendering a set of first components and transmitting the rendered set of first components to the client device, such as for compositing with a set of second components of the virtual environment that are rendered by the client device itself.”), and content in the first portion of the particular frame does not have the complexity level greater than the threshold complexity level (para 59, “For example, environment rendering threshold 305 represents relatively favorable network conditions, such that the remote server may render all of the enumerated objects within the virtual environment 200 other than box objects 210, each of which is associated with a low degree of complexity”) and content in the second portion of the particular frame has the complexity level greater than the threshold complexity level (para 59, “For example, environment rendering threshold 305 represents relatively favorable network conditions, such that the remote server may render all of the enumerated objects within the virtual environment 200 other than box objects 210, each of which is associated with a low degree of complexity”; also, para 57, “In certain embodiments, the environment rendering threshold may be determined as a threshold for assigned remote rendering prioritization values, such that objects assigned a remote rendering prioritization value that meets or exceeds the environment rendering threshold are rendered by the server, with objects assigned a remote rendering prioritization value below the environment rendering threshold being rendered by the local client device.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar and Lee, in which a method selects between a client device and a server to render an interactive three-dimensional scene of a geographic location, with the features of Dietrich's invention of measuring the complexity of an object by the quantity of its components and primitives, of assigning each object a value determined from a component complexity criterion among others, of assigning objects within one scene to the server or to the local client device according to whether that value meets or exceeds a rendering threshold, and of the server rendering a first set of components and transmitting them to the client for compositing with a second set rendered by the client itself. The combination would have been obvious because the combined system decides where to render an entire scene, which forces an all-or-nothing choice even when only part of a map frame is demanding, and Hultquist's own map frames are expressly heterogeneous, combining a raster background layer with vector objects overlaid on it. Dietrich addresses that heterogeneity directly in the same client and server rendering setting, by splitting a single frame along an object-level complexity threshold and compositing the two halves at the client. A person of ordinary skill would have applied that split to the combined system's map frames, with the predictable result that the more complex content of a frame is rendered at the server while the less complex content is rendered at the client and the two are combined for display.
Claim(s) 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hultquist et al. (U.S. Pub. No. 20130321456) as modified by Dharmapurikar et al. (U.S. Pub. No. 20210245047), and Lee et al. (U.S. Pub. No. 20250328298), further in view of Yerli (U.S. Pub. No. 20200402314) and Oduwaiye et al. (U.S. Pub. No. 20250030688).
Regarding claim 21, Hultquist as modified by Dharmapurikar and Lee discloses the first computing system of claim 1, one or more rendering criteria further comprises a network condition determined by calculating a signal-to-noise ratio (SNR) based on a ratio between a received signal power and a background noise power.
However, in a similar field of endeavor, Yerli discloses wherein the one or more rendering criteria further comprises a network condition (para 103, “In other embodiments, the transition may switch back and forth between the server and client based on the quality of service (QoS) available in the network.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar and Lee, in which a client device selects between itself and a server to render an interactive three-dimensional scene of a geographic location and receives a stream of server-rendered output from that server while the scene data is transferred to it, with the features of Yerli's invention of switching the rendering back and forth between the server and the client according to the quality of service available in the network. The combination would have been obvious because the combined system fixes the renderer from the size of the scene information and the state of the transfer alone, so a client that has taken over the rendering keeps it even after the link that carried the stream has degraded, and a client that has not yet taken it over keeps waiting on a stream the link can no longer carry. Yerli, working with the same arrangement in which a server renders first and a client renders afterward, treats the condition of the network as a live input that can move the rendering in either direction rather than as a fact fixed at the outset. A person of ordinary skill would have added the network condition to the combined system's rendering criteria, with the predictable result that the choice between the two systems answers to the condition of the link that carries the rendered output.
Oduwaiye discloses determined by calculating a signal-to-noise ratio (SNR) (para 60, “In some implementations, signal strength determiner 131 may be configured to measure or calculate a signal-to-noise ratio (SNR) as the signal strength. For example, the SNR value corresponds to a measure of the ratio between the received signal power and the background noise power. Generally, SNR is expressed in decibels and a higher SNR value means a higher signal strength.”) based on a ratio between a received signal power and a background noise power (para 60, “In some implementations, signal strength determiner 131 may be configured to measure or calculate a signal-to-noise ratio (SNR) as the signal strength. For example, the SNR value corresponds to a measure of the ratio between the received signal power and the background noise power. Generally, SNR is expressed in decibels and a higher SNR value means a higher signal strength.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar and Lee, in which the choice between the first and second computing systems turns in part on the condition of the network, with the features of Oduwaiye's invention of calculating a signal-to-noise ratio as the measure of the ratio between the received signal power and the background noise power. The combination would have been obvious because the combined system conditions the change of renderer on network quality of service but never says how that quality is to be measured, and a criterion stated only as quality of service cannot be evaluated until a quantity is chosen for it. Oduwaiye supplies a standard quantity for exactly that purpose, defining the signal-to-noise ratio in terms of received signal power and background noise power and noting that a higher value means a higher signal strength, which makes it directly usable as a decision input over the wireless link that carries map data to Hultquist's client. A person of ordinary skill would have measured the combined system's network condition that way, with the predictable result that the renderer selection responds to a computed signal-to-noise ratio rather than to an unquantified notion of network quality.
Regarding claim 22, Hultquist as modified by Dharmapurikar and Lee discloses the computer-implemented method of claim 14, one or more rendering criteria further comprises a network condition determined by calculating a signal-to- noise ratio (SNR) based on a ratio between a received signal power and a background noise power.
However, in a similar field of endeavor, Yerli discloses wherein the one or more rendering criteria further comprises a network condition (para 103, “In other embodiments, the transition may switch back and forth between the server and client based on the quality of service (QoS) available in the network.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar and Lee, in which a method selects between a client device and a server to render an interactive three-dimensional scene of a geographic location and streams server-rendered output to the client device while the scene data is transferred to it, with the features of Yerli's invention of switching the rendering back and forth between the server and the client according to the quality of service available in the network. The combination would have been obvious because the combined system fixes the renderer from the size of the scene information and the state of the transfer alone, so a client that has taken over the rendering keeps it even after the link that carried the stream has degraded, and a client that has not yet taken it over keeps waiting on a stream the link can no longer carry. Yerli, working with the same arrangement in which a server renders first and a client renders afterward, treats the condition of the network as a live input that can move the rendering in either direction rather than as a fact fixed at the outset. A person of ordinary skill would have added the network condition to the combined system's rendering criteria, with the predictable result that the choice between the two systems answers to the condition of the link that carries the rendered output.
Oduwaiye discloses determined by calculating a signal-to- noise ratio (SNR) (para 60, “In some implementations, signal strength determiner 131 may be configured to measure or calculate a signal-to-noise ratio (SNR) as the signal strength. For example, the SNR value corresponds to a measure of the ratio between the received signal power and the background noise power. Generally, SNR is expressed in decibels and a higher SNR value means a higher signal strength.”) based on a ratio between a received signal power and a background noise power (para 60, “In some implementations, signal strength determiner 131 may be configured to measure or calculate a signal-to-noise ratio (SNR) as the signal strength. For example, the SNR value corresponds to a measure of the ratio between the received signal power and the background noise power. Generally, SNR is expressed in decibels and a higher SNR value means a higher signal strength.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Hultquist in view of Dharmapurikar and Lee, in which a method conditions the choice between the first and second computing systems on the condition of the network, with the features of Oduwaiye's invention of calculating a signal-to-noise ratio as the measure of the ratio between the received signal power and the background noise power. The combination would have been obvious because the combined system conditions the change of renderer on network quality of service but never says how that quality is to be measured, and a criterion stated only as quality of service cannot be evaluated until a quantity is chosen for it. Oduwaiye supplies a standard quantity for exactly that purpose, defining the signal-to-noise ratio in terms of received signal power and background noise power and noting that a higher value means a higher signal strength, which makes it directly usable as a decision input over the wireless link that carries map data to Hultquist's client. A person of ordinary skill would have measured the combined system's network condition that way, with the predictable result that the renderer selection responds to a computed signal-to-noise ratio rather than to an unquantified notion of network quality.
Allowable Subject Matter
Claim 20 is allowed.
The following is a statement of reasons for the indication of allowable subject matter: Claim 20 is allowed because the located art does not teach or suggest the two transmissions the claim requires the first computing system to make to the second computing system on complementary branches of a single rendering criteria determination, in a first computing system that is also the system that receives the stream of rendered visual elements from the second computing system while the information is transferred to it and that has the rendering operations transitioned to it upon completion of that transfer.
On the branch where the criteria are satisfied, claim 20 requires transmitting the plurality of visual elements associated with the interactive 3D scene of the geographic location from the first computing system to the second computing system for rendering. On the branch where the criteria are not satisfied, claim 20 requires the first computing system to perform the rendering operations itself to generate a plurality of rendered visual elements and to transmit those rendered visual elements to the second computing system. Both transmissions therefore run outward from the first computing system, while the stream of rendered visual elements and the transferred information run inward to that same system.
The application places that aspect at the server rather than at the display device. Paragraph [0026] introduces this aspect as an example computing system or computing device, given as a server computing system, whose operations are to determine whether the rendering criteria are satisfied, to select a second computing system to perform the rendering operations and transmit the plurality of visual elements from the first computing system to the second computing system for rendering when they are satisfied, and to perform the rendering operations itself and transmit the resulting rendered visual elements to the second computing system when they are not. The claim thus places the criteria determination, both outbound transmissions, the receipt of the stream, and the transition of the rendering operations in a single system.
The located art divides those functions among machines holding different roles and never assembles them in one system. Art in which a device sends scene data up to a cloud renderer treats that device as one that cannot render, so it renders on no branch and returns no rendered output, and its traffic is a one-way upload of scene attributes answered by a one-way return of a finished frame, with no criteria determination choosing between the two directions. Art in which a machine renders and returns rendered frames to a server is distributed or crowd-sourced rendering, in which the rendering machine is a worker node contributing frames to a farm rather than a system that displays or consumes the scene, and in which there is no interactive three-dimensional scene of a geographic location, no stream received while information is transferred, and no transition of the rendering operations to that machine. Art that does select between two systems on rendering criteria in a mapping or scene-rendering setting sends the rendered output in one direction only, to the system that displays it, and never has the selecting system either forward the visual elements for rendering elsewhere or return its own rendered visual elements.
One of ordinary skill working from the located art had no reason to arrive at the claimed arrangement. In that art a machine forwards scene data because it lacks the capability to render, and a machine returns rendered output because it has been engaged as a renderer for another party's work. Claim 20 requires one system to do both, chosen by the same criteria determination, while remaining the system that receives the stream and assumes the rendering. Assembling that from the located art requires reassigning the roles the references give their machines and accumulating references beyond what the art itself suggests, which is a reconstruction available only with the claim in hand.
Claim 3, 7-12, 15, and 18 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant's arguments filed July 8, 2026 have been fully considered.
Applicant argues at pages 12 to 14 of the Remarks that Claydon's threshold is directed to the capabilities of a client device and that Claydon's predetermined criteria are network bandwidth, processing power of the client device and type of content, so that Claydon does not disclose or suggest that satisfying the rendering criteria comprises a size of information associated with the interactive 3D scene of the geographic location exceeding a threshold level. These arguments are persuasive with respect to Claydon. Claydon's only recitations of the word size are display size and the size of the visible slice, and the parameters that Claydon compares to a predetermined threshold are the device and link properties Applicant identifies. The rejections under 35 U.S.C. 102(a)(1) over Westerhoff and under 35 U.S.C. 103 over Westerhoff in view of Claydon, over Westerhoff in view of Jackson, and over Westerhoff in view of Claydon and van Os have therefore been withdrawn. However, upon further consideration and as necessitated by Applicant's amendment, new grounds of rejection under 35 U.S.C. 103 over Hultquist in view of Dharmapurikar and Lee, and over those references further in view of Kulkarni, Dietrich, Yerli, and Oduwaiye, are made as set forth above. No new ground is made against claim 20. The amended limitations of claim 20 place both the transmission of the visual elements for rendering and the transmission of the rendered visual elements in the first computing system, on complementary branches of the rendering criteria determination, and the located art does not reach that arrangement without an accumulation of references that the art itself does not suggest. Claim 20 is allowed for the reasons given in the Allowable Subject Matter section above.
Applicant's argument at page 14 of the Remarks that none of Westerhoff, Jackson, or Van Os cures the deficiencies of Claydon has been considered but is moot because it does not apply to the new combination of references being used in the current rejection. The size of information limitation is now met by Lee, which compares the data size of the information to a predetermined value and assigns the work to the cloud server when the data size is at or above that value and to the local device.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jai Li whose telephone number is (571)272-1170. The examiner can normally be reached Mon-Thu between 06:00-16:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571)272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAI W LI/Junior Patent Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
