DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 9-10, and 19 are objected to because of the following informalities:
The term “the streamed visualization data” lacks antecedent basis. Appropriate correction is required.
Claims 1, 11, and 20 are objected to because of the following informalities.
The phrase “executing on first specialized computational hardware” in claim 1 line 13, claim 11 line 11, and claim 20 line 15, should read “executing on a first specialized computational hardware”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Calegari et al. (US 20210203739 A1, hereinafter “Calegari”) in view of Sherman et al. (US 9413807 B1, hereinafter “Sherman”).
Regarding claim 20,
Calegari teaches:
A system comprising (Calegari: Fig. 1, ¶73, “FIG. 1 represents a diagram of a computer system for generating an aggregated data according to the invention”):
one or more compute nodes of a virtual data center (VDC) having compute, memory, and storage resources configured to execute a scientific visualization system (Calegari: ¶22, “. . . a single scientific computation job, especially for a parallel distributed memory application (the most important case), aggregates the power of several tens, hundreds or thousands of these physical servers (nodes). . .”; ¶248, “. . . a remote visualization server 80 allows user clients 2 to execute interactive 2D/3D graphics applications on remote servers instead of their system, and to open visualization sessions to control them remotely. . . the 3D rendering, is carried out on the server side on dedicated resources. . .”; ¶93, “. . . a processor . . . in the memories of the computer system or other devices for storing, transmitting or displaying the information . . .”; NOTE: The physical server(s) are the nodes that allocate hosting virtualized/containerized services. The user is able to visualize scientific jobs remotely.)
configured for interactive visualization of simulation results (Calegari: ¶248, “. . . a remote visualization server 80 allows user clients 2 to execute interactive 2D/3D graphics applications on remote servers. . .”; NOTE: The simulation results is the result from the scientific computation job described in paragraph 22. ),
the scientific visualization system configured to:
render a video stream of the simulation results from the visualization code executing on a second specialized computational hardware resource of the VDC, the second specialized computational hardware resource configured to encode and forward the video stream to the client (Calegari: ¶248, “. . . allows user clients 2 to execute interactive 2D/3D graphics applications on remote servers . . . open visualization sessions to control them remotely. Thus, all of the computing and rendering of the graphics applications, including the 3D rendering, is carried out on the server side on dedicated resources. The keyboard and mouse inputs from the user client 2 are transferred to the server which, in return, encodes the graphics scene in pixels and returns the data in the form of video stream to the user client. . .”; NOTE: The second specialized computational hardware is the processor that executes instructions that encodes the graphics scene in forwarded to the client in the form of a video stream).
However, Calegari does not determine or track client’s resources whether it is capable of rendering the scientific job computation because “the computing result files can be very large (from gigabytes to terabytes)” as described in ¶248. Calegari’s servers do all the rendering instead of the client’s system (only taking user inputs such as keyboard and mouse inputs), and just forwards the data as video stream to the user.
Therefore, Calegari fails to teach:
negotiate one of client-side or server-side rendering of visualization information for a data streaming service using one or more post-processing resources of the VDC, the negotiated rendering configured to determine resource capacity and capability of a client to handle various types of streams and data sent over the streams from the data streaming service;
in response to determining that sufficient resource capacity and capability is available at the client for a geometry data stream of the visualization information, compute visualization filters for visualization data from is visualization code executing on first specialized computational hardware resource of the VDC, the visualization filters computed by a compute hardware resource of the post-processing resources configured to forward the visualization data and the computed visualization filters over one or more geometry streams to the client;
and in response to determining that insufficient resource capacity and capability is available at the client, render a video stream of the simulation results from the visualization code executing on a second specialized computational hardware resource of the VDC, the second specialized computational hardware resource configured to encode and forward the video stream to the client.
The analogous art Sherman teaches:
negotiate one of client-side or server-side rendering of visualization information for a data streaming service using one or more post-processing resources of the VDC (Sherman: col 10 lines 13-16, and lines 39-42 “. . . the sever system 130 includes . . . a dynamic determination module 136. . . the dynamic determination module determines, between the requesting client system and the server system, which system is more efficient at producing the visual representation of the requested data. . .”; col 7 lines 34-41, “. . . When the server system is an efficient location to render the visual representations, the requested visual representations are rendered into image files and the image files are transmitted to the client system for display. When the client system is an efficient place to render the visual representations, the server system transmits the requested data set to the client system for rendering and display. . .”; NOTE: Sherman’s VDC is its server system 130. Sherman’s dynamic determination module, which is the claimed post-processing resource negotiate one of client-side or server-side by identifying which system (client or server) is more efficient for visual representation. The data stream includes image files rendered by the server-side (if server is more efficient), or rendering data is sent to the client (if client is more efficient) to be rendered in the client-side)
the negotiated rendering configured to determine resource capacity and capability of a client to handle various types of streams and data sent over the streams from the data streaming service (Sherman: col 1 lines 45-65, “. . . a method for dynamically assigning tasks for visually presenting interactive data. . . specifies one or more client device characteristics. The one or more device characteristics may include: memory capacity of the client device; current memory utilization of the client device; processor capability of the client device; current utilization of the processors of the client device; and/or identification of a web browser at the client device that issued the request. The server determines whether to render the requested visual representation at the server or at the client device based on a plurality of factors, including the one or more specified client device characteristics and a size of the designated data set. . .”);
in response to determining that sufficient resource capacity and capability is available at the client for a geometry data stream of the visualization information (NOTE: As cited above in reference to Sherman col 10 lines 13-16, and lines 39-42, col 7 lines 34-41, col 1 lines 45-65, the response of the server is to select the client’s system to receive rendering data of the visual representation based on sufficient resource capacity and capability such as processor/memory utilization, memory capacity, etc. The client then proceeds on rendering the visual representation),
compute visualization filters for visualization data from visualization code executing on first specialized computational hardware resource of the VDC (Sherman: col. 2 lines 45-50, “. . . the server also determines the requested type of operation. For example, the client system can request filtering, sorting, drilling-down, tooltip generation, and aggregation of data. . .”; NOTE: The visualization code to compute the visualization filters (filters) is inherent because the VDC (server) comprising at least one processor, which is the specialized computation hardware resource of the VDC) processes filter instructions.),
the visualization filters computed by a compute hardware resource of the post-processing resources configured to forward the visualization data and the computed visualization filters over one or more geometry streams to the client (Sherman: col 7 lines 62-67, “. . . the server could produce visual data by calculating the size, location, and color of the circles. The server system then transmits this visual data to the client system and the client system renders the actual image based on the visual data”; NOTE: The geometry stream is the stream of transmitted visual data to the client system because it calculates size, location, and shape, which are geometric features. As cited above, the server also processes visualization filters, which are also part of the visual data transmitted to the client.);
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Calegari, and Sherman, and include: negotiate one of client-side or server-side rendering of visualization information for a data streaming service using one or more post-processing resources of the VDC, the negotiated rendering configured to determine resource capacity and capability of a client to handle various types of streams and data sent over the streams from the data streaming service; in response to determining that sufficient resource capacity and capability is available at the client for a geometry data stream of the visualization information, compute visualization filters for visualization data from is visualization code executing on first specialized computational hardware resource of the VDC, the visualization filters computed by a compute hardware resource of the post-processing resources configured to forward the visualization data and the computed visualization filters over one or more geometry streams to the client.
The reason for doing so is because “dynamically allocating work allows the server system to scale to very large data sets, rendering the visual representation at the server when it is impractical to transfer all the raw data over a network to a client. Furthermore, dynamic allocation of rendering tasks allows the visualization server to serve client devices with low bandwidth connections, old or out of date browsers, or limited processing power. Also, by dynamically allocating rendering tasks, the server system eliminates the need for browser plugins” (Sherman: col 6 lines 4-12).
However, Sherman also fails to teach: in response to determining that insufficient resource capacity and capability is available at the client, render a video stream of the simulation results from the visualization code executing on a second specialized computational hardware resource of the VDC, the second specialized computational hardware resource configured to encode and forward the video stream to the client (NOTE: Sherman’s response to determining that insufficient resource and capability is available at the client is to render the images at the server as described in Sherman: col 6 lines 4-12. However, Sherman’s visual data is only static images. On the other hand, Calegari’s system always render at the server and encodes the visual data and forwarded to the client as a form of video stream.).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Calegari, and Sherman and apply Sherman’s dynamic allocation of rendering tasks for visualization to Calegari’s and include: in response to determining that insufficient resource capacity and capability is available at the client (NOTE: As taught by Sherman as discussed above), render a video stream of the simulation results from the visualization code executing on a second specialized computational hardware resource of the VDC, the second specialized computational hardware resource configured to encode and forward the video stream to the client (NOTE: As taught by Calegari as discussed above).
The reason for doing so is to “efficiently provide visual representations of complex data services over a network, some implementations dynamically determine where to perform the calculations and rendering for those visual representations. By dynamically determining where to render data, the server provides a more engaging and responsive user experience. When the data set is small, doing work locally removes the latency of round-trips to the server” (Sherman: col 5 lines 63-67 to col 6 lines 1-3).
Regarding method claim 11, method claim 11 is drawn to the method corresponding to the configuration of using same as claimed in system of claim 20. Therefore, method claim 1 corresponds to the configuration in system of claim 20, and is rejected for the same reasons of obviousness as used above.
Regarding CRM claim 1, CRM claim 1 is drawn to the method corresponding to the configuration of using same as claimed in system of claim 20. Therefore, CRM claim 1 corresponds to the configuration in system of claim 20, and is rejected for the same reasons of obviousness as used above.
Regarding claim 2, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Calegari further teaches:
wherein the second specialized computational hardware resource is configured to operate as a post-process server of the scientific visualization system (NOTE: The processor in the server that does the data analysis (¶32) and filter computations (¶26) is a post-process server of the scientific visualization system because there are prior processes that happens before the data analysis and filter computation and transmitting video stream to the client)
to perform data analysis (Calegari: ¶32, “. . . Determining redundant data. . .”; )
and filter computations (Calegari: ¶26, “. . . a method allows eliminating the data redundancy. . .”)
to render the video stream forwarded to the client (Calegari: ¶248, “. . . the server which, in return, encodes the graphics scene in pixels and returns the data in the form of video stream to the user client. . .”).
Regarding claim 3, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Calegari further teaches:
wherein the first and second specialized computational hardware resources are graphics processing units (Calegari: ¶95, “. . . The hardware circuit can be an integrated circuit. Examples of a processor comprise, . . . a central processing unit, a graphic processor. . .¶84, “The data . . . correspond to input computing files accessible and processable by several intensive computing solutions, computing results accessible and processable by several intensive computing solutions, . . . resource use . . . GPU, etc.). . .”; ¶22, “. . . a microservice corresponds by definition to a service that can “fit” in a virtual machine or a container. . . allocate . . . physical resources . . .; ¶86, “. . . “microservice” corresponds to an application chain generally including a plurality of applications capable of executing one or several tasks. . .”; NOTE: The virtualized microservices that executes several tasks such as encoding, and rendering have separate allocated resources including a GPU resource, which is also tracked for usage).
Regarding claim 4, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Although Calegari teaches the server receiving user keyboard and mouse inputs to execute 2D/3D graphics applications and to open visualization sessions to control them remotely as described in paragraph 248, Calegari fails to teach: wherein the visualization filters include user filtering actions provided by the client and wherein the compute hardware resource is configured to operate as a remote data/analysis server of the scientific visualization system to execute the filtering actions to create visualizations.
The analogous art Sherman teaches:
wherein the visualization filters include user filtering actions provided by the client (col. 6 lines 40-50, “visualization requested. . . the client system can request filtering, sorting, drilling-down, tooltip generation, and aggregation of data. . .”)
and wherein the compute hardware resource is configured to operate as a remote data/analysis server of the scientific visualization system to execute the filtering actions to create visualizations (Sherman: col 7 lines 7-30, “the client . . . request for a new visual representation of a given data set. . . For example, . . . the user desires a visual representation of election results by county. A user is displaying a visualization of senate election results by county (e.g., about 3000 data points), filtered for a particular election year. Filtering senate election results by a particular election year (e.g., 1986) reduces the number of data points by one third, which is approximately 1000 data points. The requested visual representation is a county map of the United States, where the counties are color coded by political affiliation. Counties without data are left blank.. . .”; NOTE: As shown in Fig 1, the server processes the filtering request from the client, and because the server is physically separated from the client, therefore, the server operating as a remote data/analysis executing filtering actions associated with the visualization requested. Sherman’s visualizations are scientific because the example visualizations disclosed by Sherman are statistics based.)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Calegari and Sherman and include: wherein the visualization filters include user filtering actions provided by the client and wherein the compute hardware resource is configured to operate as a remote data/analysis server of the scientific visualization system to execute the filtering actions to create visualizations.
The reason for doing so is because “it is time-consuming and potentially expensive to send a very large data set the client device to be rendered. However, it may be feasible to transmit a relatively small data set or aggregated data” (Sherman: col 6 lines 44-47).
Regarding claim 5, depending on 4,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 4,
Sherman further teaches:
wherein the visualizations include one of two dimensional or three-dimensional graphs (Sherman: col 11 lines 22-25, “render an image for the requested type of visual representation. For example, if the user requested the data to be presented in the form of a bar graph”; NOTE: A rendered image is inherently a 2D image. A rendered and displayed bar graph image is at least a 2D bar graph ).
Regarding claim 6, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Calegari further teaches:
wherein the compute hardware resource is a central processing unit node (Calegari: ¶95, “. . . The hardware circuit can be an integrated circuit. Examples of a processor comprise, but are not limited to, a central processing unit, a graphic processor, an application-specific integrated circuit (ASIC), and a programmable logic circuit. . .”, ¶22, “. . . a service that can “fit” in a virtual machine or a container. . .aggregates the power of several tens, hundreds or thousands of these physical servers (nodes).”).
Regarding claim 7, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Sherman further teaches:
wherein the one or more geometry streams (NOTE: The visual data as taught by Sherman transmitted to the client for client side rendering as discussed in the rejection of claim 1)
include mesh geometry and scalar data (Sherman: col 7 lines 45-67, “includes all data necessary to fully render the requested visualization. For example, a user requests a visualization of the average life span of people by the city where they live. The lifespan averages are represented by circles displayed on a map, where the size of the circle indicates the amount above or below the national average for each particular city. . . the server could produce visual data by calculating the size, location, and color of the circles. . .”; NOTE: The mesh geometry defines the data necessary to display the shape, size, and location of shapes when rendering objects in 3D. Sherman teaches that geometrical data is included for rendering and displaying shapes over a data stream so the client can perform rendering of shapes in the client side. In order to render the shape corresponding to the geometry of the circle displayed on a map, the client’s system must know the coordinates, and the edges, and other geometric information of the circles in order to display them on specific locations with corresponding sizes on a map. The scalar data defines the different sizes of the circles as an indication to the amount above or below the national average scaling the size of the circles accordingly).
Regarding claim 8, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Sherman further teaches:
wherein the one or more geometry streams contain increasingly complex level-of-detail meshes (Sherman: col 1 lines 29-30, “. . . The time and processing power needed to render images increases as the size and complexity of data sets increases. . .”; NOTE: As discussed above, the mesh geometry defines the geometry and placement of the circles displayed on the map. Sherman’s system analyzes the client’s resource capabilities and increase the complexity of data sets accordingly, which increases the complex level-of-detail of the meshes. The higher the client’s resource capability, the complexity of data sets to include mesh geometry increases to match the client’s processing power.).
Regarding claim 9, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Calegari further teaches:
wherein the streamed visualization data (Calegari: ¶248 video stream)
includes interactive automated reports or parameters for mesh processing (Calegari: ¶248, “”a remote visualization server 80 allows user clients 2 to execute interactive 2D/3D graphics applications on remote servers . . . to open visualization sessions to control them remotely. Thus, all of the computing and rendering of the graphics applications, including the 3D rendering, is carried out on the server side on dedicated resources. The keyboard and mouse inputs from the user client are transferred to the server which, in return, encodes the graphics scene in pixels and returns the data in the form of video stream to the user client. . .”; NOTE: The process of 3D rendering constitutes mesh processing. The interactive automated reports or parameters are the user keyboard and mouse inputs, which makes the visualization interactive.).
Regarding claim 10, depending on 1,
The combination of Calegari and Sherman teaches:
The non-transitory computer readable medium of claim 1,
Calegari further teaches:
wherein the streamed visualization data includes volumetric data (NOTE: In reference to Calegari paragraph 248, 3D rendering is carried out on the server side and transmits video stream to the client. 3D rendered objects have volumes, because Calegari’s system can render in 3D, the volumetric data is inherently included.).
Although Calegari teaches eliminating redundancy which reduces the amount of data transmitted to the client as streamed visualization data (video stream), Calegari fails to teach: reduced representations that enable interactive slicing in a browser of the client.
The analogous art Sherman teaches:
reduced representations (Sherman: col 6 lines 43-49, “. . . visualization requested. . . transmit a relatively small data set or aggregated data. Furthermore, the server also determines the requested type of operation. For example, the client system can request filtering, sorting, drilling-down, tooltip generation, and aggregation of data. . .”; NOTE: A request to drill-down data to be able to transmit small data set constitutes to a reduced representation of the visualization data requested.)
that enable interactive slicing in a browser of the client (Sherman: col 7 lines 8-10, “. . . the client system detects user input indicating a request for a new visual representation of a given data set. Based on the detected user input, the client system sends a request to the server system. . .”; col 8 lines 29-34, “. . . a user loads an interactive visualization in a browser, a request is sent to the server system that includes the “name” of the requested visualization and a summary of the client capabilities. . .”; NOTE: The user loads a visualization request using a browser. The user using the client device can send a request to “drill-down” the data for the interactive visualization in a browser making. The process of the user requesting to “drill-down” data for visualization interactively via user inputs constitutes to the enabling interactive slicing in a browser of a client.)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Calegari and Sherman and include: wherein the streamed visualization data includes reduced representations that enable interactive slicing in a browser of the client.
The reason of doing so is because “it is time-consuming and potentially expensive to send a very large data set the client device to be rendered. However, it may be feasible to transmit a relatively small data set or aggregated data” (Sherman: col 6 lines 44-46).
Regarding method claims 12-19 respectively, method claims 12-19 are drawn to the methods corresponding to the program configuration of using same as claimed in CRM claims 2-9 respectively. Therefore, method claims 12-19 respectively corresponds to the program configuration in CRM claims 2-9 respectively, and are rejected for the same reasons of obviousness as used above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK GALERA whose telephone number is (571)272-5070. The examiner can normally be reached Mon-Fri 0800-1700 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK P GALERA/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights