DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 7-9,11-15, 18 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Double Patenting
Claim 1 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of copending Application No. 18651187 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because both the independent claim 1 in this case is a broader than the one in application No. 18651187. The claim 1 in this case refers to a “video encoder” while the claim 1 of the reference application refers to an “HDR video encoder.”
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 103
Claim(s) 1,2,4,6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schmieder (US-20100146127-A1) in view of Zhu(US-20150063451-A1)
Regarding claim 1, Schmieder discloses A computing system comprising: a processor; and computer storage memory having computer-executable instructions stored thereon which, when executed by the processor(Fig. 1 discloses all the parts necessary for an example computer system), configure the computing system to perform operations comprising: obtaining, at a video driver of a remote desktop server, graphical data for presenting in association with a remote desktop client device(para.[0003] “Graphics data from a user mode desktop application operating in user-mode session space is send through a display driver operating in kernel-mode session space;” para.[0029] “In the embodiment where the graphics data is to be sent across a communications network via a remote desktop protocol (RDP), the driver 204 partially encodes the received data into at least one RDP graphics protocol data unit (PDU).” Here the drier corresponds to the “display driver” and performs the same functional steps as “obtaining…graphical data.”); providing the graphical data to a video encoder logically separated from the video driver(para. [0036] “The shared memory graphics reflector 308 comprises a memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them. This shared memory graphics reflector 308 marshals all received data and sends it to a graphics reflector 310 in user-mode system space. The reflectors ensure that a normally complex task of exchanging data between the user session space and the system space proceeds in an efficient manner.” The claim requires that the driver provides the graphics data to an encoder that is logically separated from the driver. This reference explicitly has the graphics data pass from the kernel-mode display driver to an RDP encoder process in user-mode system space using a reflector that marshals data across the boundary aligning with the claim element.); and providing the encoded graphical data to a remote access protocol stack in a terminal service of the remote desktop server for transmitting the encoded graphical data to the remote desktop client device( para.[0041] “When the RDP output scheduler and encoder 314 has encoded the received graphics data into at least one RDP PDU, it sends each PDU to the user-mode transport 316. The user-mode transport 316 sends the PDU to the intended recipient 318 of the PDU. For instance, where a user on a client machine 318 has a RDP session with the present machine, the user-mode transport 316 sends the PDU to the client 318 in accordance with RDP across a communications network.”)
However, Schmieder alone does not fully disclose encoding, via the video encoder, the graphical data; The combination of Schmieder and Zhu does disclose encoding, via the video encoder, the graphical data(Zhu para.[0040] discloses “For example, video content 214 is passed to a video encoder 220, shown as performing an encoding according to an MPEG-based codec, such as MPEG-4 AVC/264.” Further, Schmieder para.[0040] discloses “ The output scheduler component 314 is responsible for consuming the virtual channel data and the graphics marshaled data and encoding that data on its own set of threads into the RDP format. This architecture allows the encoding for each RDP session present on the machine to be performed by a single component on a finite and optimal set of threads.” Schmieder teaches the encoder process encodes graphics data into an RDP format. Zhu explicitly teaches that , within an RDP pipeline, video content is encoded by a “video encoder” using codecs “such as MPEG-4 AVC/264.” Zhu supplies the explicit “video encoder” teaching that can be used in Schmieders encoder process.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Zhu into the teachings of Schmieder in order to improve compression efficiency and compatibility with decoder support.
Regarding claim 2, the combination of Schmieder and Zhu disclose all the elements of claim 1 as discussed above. Schmieder also discloses wherein the graphical data comprises data rendered by a graphics processing unit or a central processing unit based on an instruction provided by the video driver(para.[0027] “An application in user-mode session space 202, such as a word processor or web browser first generates graphics data. Where the application 202 is a word processor, this graphics data may be text or an image to be displayed on the screen in a particular alignment, such as centered and bolded on a page of the document being edited. This graphics data is then sent to a device driver 204 in accordance with a device driver interface (DDI) 206 for processing. The driver 204 and DDI 206 exist in kernel-mode session space. A device driver is a component or computer program that allows a high-level computer program, such as a word processor, to communicate with a hardware device, such as a computer printer or a computer monitor. The driver 204 typically communicates with the associated hardware through a communications subsystem of the computer to which the hardware is physically connected. The corresponding DDI 206 is a form of an application programming interface (API) that is specific to the driver. The program 202 makes a call in accordance with the DDI 206, which is translated by the DDI 206 and the driver 204 into a communication understood by the corresponding hardware.” Schmieder discloses an instruction path involving the driver with the application making a call to the DDI that is translated by the driver analogous to “instructions provided by the video driver.”)
Regarding claim 4, the combination of Schmieder and Zhu disclose all the elements of claim 1 as discussed above. Schmieder also discloses wherein the graphical data is provided to the video encoder via an encoding interface (para. [0036] “In the embodiment where the graphics data is to be sent across a communications network via a remote desktop protocol (RDP), the driver 304 partially encodes the received data into at least one command that can be understood by the RDP encoder process. A The driver 306 sends each command to a shared memory graphics reflector 308 that exists in kernel mode session space. The shared memory graphics reflector 308 comprises a memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them. This shared memory graphics reflector 308 marshals all received data and sends it to a graphics reflector 310 in user-mode system space.” Schmieder describes a specific interface mechanism between the kernel driver side and the user mode encoder process. The shared memory graphics reflectors marshal graphics commands across the boundary and deliver them into the encoder process. The reflectors and share memory mapping are an interface layer mediating driver to encoder delivery aligning with the claim element.)
Regarding claim 6, the combination of Schmieder and Zhu disclose all the elements of claim 1 as discussed above. Schmieder also discloses wherein the video encoder includes remote access protocol code to communicate the encoded graphical data (para. [0029] “In the embodiment where the graphics data is to be sent across a communications network via a remote desktop protocol (RDP), the driver 204 partially encodes the received data into at least one RDP graphics protocol data unit (PDU). A RDP PDU is a unit of data that is specified in the RDP protocol. The driver 204 then sends each RDP PDU to a system-space kernel mode driver 208 that implements the rest of the encoding.” Schmieder’s RDP encoder process performs protocol specific packaging and transmission aligning with the claim element.)
Claim 3,5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schmieder as modified by Zhu as applied to claim 1 above, and further in view of Collins(US-20090285496-A1).
Regarding claim 3, the combination of Schmieder and Zhu disclose all the elements of claim 1 as discussed above. However, the combination does not fully disclose wherein the encoded graphical data is provided from the video encoder to the video driver for providing to the remote access protocol stack via a file input/output communication channel.
The combination of Schmieder, Zhu and Collins does disclose wherein the encoded graphical data is provided from the video encoder to the video driver for providing to the remote access protocol stack via a file input/output communication channel(Collins states in para.[0067] “The encoder 204 in many embodiments, inputs image data or display graphics and encodes or compresses the data or graphics according to an encoding or compression algorithm. In one embodiment, the encoder 204 encodes the inputted image data or display graphics according to the encoding methods and processes described herein. In other embodiments, the encoder 204 can both encode the image data and compress the image data according to any of the following types of compression: lossless, lossy, Lempel-Ziv, Huffman, Adaptive Huffman, Golomb, or any other type of compression consistent and operative with the encoding methods described herein. In some embodiments, the encoder 204 transmits an encoded and/or compressed image to an application/desktop delivery system 222 for transmission to a remote computing machine 201; … While FIG. 2 depicts the encoder 204 as communicating with the frame buffer 202, cache 214 and the application/desktop delivery system 222; the encoder 204 can communicate with any number of components on the local computing machine 200 including the applications 220. In some embodiments, the encoder 204 can be included within the frame buffer 202.” Further Schmieder teaches in para. [0030] “Such virtual channel data is sent directly through a file input/output (I/O) subsystem driver 216 that exists in kernel-mode session space. Similar as to how the device driver 204 processed its received data into at least one RDP PDU, the I/O subsystem 216 may apply framing, compression and encryption to the virtual channel data received to produce at least one RDP PDU, and send each PDU to the system space RDP kernel mode driver 208. “Collins provides the encoded graphical data and RDP delivery context. Schmieder provides explicit file input/output communication channel techniques for moving channel data into the RDP driver. It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Collins in the the combination of teachings of Schmieder and Zhu in order to provide the encoded graphical data for transmission via a file I/O communication.
Regarding claim 5, the combination of Schmieder and Zhu disclose all the elements of claim 1 as discussed above. However, the combination does not fully discloses wherein the graphical data is obtained at the video driver via a frame buffer that stores data rendered by a graphics processing unit or central processing unit.
Collins does disclose wherein the graphical data is obtained at the video driver via a frame buffer that stores data rendered by a graphics processing unit or central processing unit(para.[0065] “Executing on the local computing machine 200 is a frame buffer 202 communicating with an encoder 204 over a first connection L10. The first connection L10 can be any communicative connection established between the frame buffer 202 and the encoder 204. In some embodiments, the frame buffer 202 can be any buffer able to contain graphics, graphics commands, images or any other graphical or media content. … While the frame buffer 202 is depicted as communicating only with the applications 220 and the encoder 204, in other embodiments the frame buffer 202 can communicate with any number of system components within the local computing machine 202 such as the cache 214 and the application/desktop delivery system 222.” Collins explicitly discloses the frame buffer and that it stores pixel color data for an image, where the pixel data comes from an image renderer. This aligns with the “frame buffer that stores data rendered” portion of the claim that can be used by the renderer in Schmieder.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Collins into the combination of teachings of Schmieder and Zhu in order to improve the rendering pipeline as the source that the driver taught by Schmieder to obtain the graphical data.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over Schmieder as modified by Zhu as applied to claim 1 above, and further in view of Raduchel (US-20180063555-A1) and Jia(US-20130294689-A1).
Regarding claim 10 the combination of Schmieder and Zhu disclose all the elements of claim 1 as discussed above. However, they do not fully disclose wherein the video encoder comprises a hardware or software component used to encode video content that includes high dynamic range information.
The combination of Schmieder, Zhu, Raduchel, and Jia do disclose wherein the video encoder comprises a hardware or software component used to encode video content that includes high dynamic range information (Raduchel teaches in para. [0089] “In some implementations, the server systems 300A and 300B can include a separate encoder, e.g., a software-based encoder or a hardware-based encoder, which is physically located outside of the GPU chip 310. The separate encoder can perform encoding operations to generate the encoded video data 301 without using memory bandwidth of the graphics memory 320.” This is explicit disclosure that the encoder can be software or hardware based to perform the encoding operations that generate video data. Jia discloses the encoding in Fig.1 showing an example of an HDR image encoder.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Raduchel and Jia into the combination of teachings of Schmieder and Zhu in order to a have a system to better adapt HDR information.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schmieder (US-20100146127-A1) in view of Zhu (US-20150063451-A1) and Abdo (US-20100228871-A1)
Regarding claim 16, the combination of Schmieder discloses One or more computer storage media having computer-executable instructions embodied thereon that, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising: obtaining, at a video driver of a remote desktop server, graphical data for presenting in association with a remote desktop client device ( Schmieder in para.[0003] “Graphics data from a user mode desktop application operating in user-mode session space is send through a display driver operating in kernel-mode session space;” and para.[0029] “In the embodiment where the graphics data is to be sent across a communications network via a remote desktop protocol (RDP), the driver 204 partially encodes the received data into at least one RDP graphics protocol data unit (PDU).” Here the driver corresponds to the “display driver” and performs the same functional steps as “obtaining…graphical data.”); providing the graphical data to a video encoder logically separated from the video driver via an encoding interface (Schmieder teaches in para. [0036] “In the embodiment where the graphics data is to be sent across a communications network via a remote desktop protocol (RDP), the driver 304 partially encodes the received data into at least one command that can be understood by the RDP encoder process. A The driver 306 sends each command to a shared memory graphics reflector 308 that exists in kernel mode session space. The shared memory graphics reflector 308 comprises a memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them. This shared memory graphics reflector 308 marshals all received data and sends it to a graphics reflector 310 in user-mode system space.” Schmieder describes a driver in kernel-mode session space providing graphics data across a boundary to a user-mode “RDP encoder process,” using shared memory reflectors. The shared memory reflects are the interface that connect the driver to a logically separate encoder process.); providing, via the video driver, the encoded graphical data to a remote desktop protocol stack in a terminal service of the remote desktop server and transmitting, via the remote desktop protocol stack, the encoded graphical data to the remote desktop client device for presentation via the remote desktop client device (para. [0041] “When the RDP output scheduler and encoder 314 has encoded the received graphics data into at least one RDP PDU, it sends each PDU to the user-mode transport 316. The user-mode transport 316 sends the PDU to the intended recipient 318 of the PDU. For instance, where a user on a client machine 318 has a RDP session with the present machine, the user-mode transport 316 sends the PDU to the client 318 in accordance with RDP across a communications network.” Schmieder explicitly transmats encoded RDP PDUs to the client using RDP transport functionality, aligning with the claim element.)
However, Schmieder alone does not disclose encoding, via the video encoder, the graphical data; providing, via the video driver, the encoded graphical data to a remote desktop protocol stack in a terminal service of the remote desktop server. The combination of Schmieder and Zhu does disclose encoding, via the video encoder, the graphical data(Zhu para.[0040] discloses “For example, video content 214 is passed to a video encoder 220, shown as performing an encoding according to an MPEG-based codec, such as MPEG-4 AVC/264.” Further, Schmieder para.[0040] discloses “ The output scheduler component 314 is responsible for consuming the virtual channel data and the graphics marshaled data and encoding that data on its own set of threads into the RDP format. This architecture allows the encoding for each RDP session present on the machine to be performed by a single component on a finite and optimal set of threads.” Schmieder teaches the encoder process encodes graphics data into an RDP format. Zhu explicitly teaches that , within an RDP pipeline, video content is encoded by a “video encoder” using codecs “such as MPEG-4 AVC/264.” Zhu supplies the explicit “video encoder” teaching that can be used in Schmieders encoder process.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Zhu into the teachings of Schmieder in order to improve compression efficiency and compatibility with decoder support.
However, the combination of Schmieder and Zhu still does not disclose providing, via the video driver, the encoded graphical data to a remote desktop protocol stack in a terminal service of the remote desktop server. The combination of Schmieder and Abdo does disclose providing, via the video driver, the encoded graphical data to a remote desktop protocol stack in a terminal service of the remote desktop server(Schmieder teaches in para. [0036] “The shared memory graphics reflector 308 comprises a memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them. This shared memory graphics reflector 308 marshals all received data and sends it to a graphics reflector 310 in user-mode system space. The reflectors ensure that a normally complex task of exchanging data between the user session space and the system space proceeds in an efficient manner.” The claim requires that the driver provides the graphics data to an encoder that is logically separated from the driver. This reference explicitly has the graphics data pass from the kernel-mode display driver to an RDP encoder process in user-mode system space using a reflector that marshals data across the boundary aligning with the claim element. However, Schmieder does not mention doing the providing “via the video encoder”. Abdo teaches this in para.[0038] “RDPDD encodes the DDI callbacks into RDP drawing orders. The encoding drawing orders are placed into the order heap 305 in memory that is shared with the RDPWD driver. A “RDP batching engine” scans the orders in the order heap to determine if there are any orders that are logically related. Any related orders are wrapped with “begin” and “end” markers. The RDPWD driver associated with the stack that is connected to the client 310 extracts the drawing orders from the order heap. It bulk compresses the drawing orders and wraps them within RDP transport structures, then sends them down the stack and to the client 310 across a communications network 308.” This provides Schmieder with architecture that is explicitly provides via the video encoder.)
Claim(s) 17,19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schmieder as modified by Zhu and Abdo as applied to claim 16 above, and further in view of Sampath(US-20080209048-A1).
Regarding claim 17, the combination of Schmieder, Zhu and Abdo discloses all the elements of claim 16 as discussed above. However, they do not disclose wherein the video driver corresponds with a remote desktop session, and the video driver instantiates the video encoder based on receiving an indication to use the video encoder in association with the remote desktop session
Sampath does disclose wherein the video driver corresponds with a remote desktop session, and the video driver instantiates the video encoder based on receiving an indication to use the video encoder in association with the remote desktop session (para. [0032] “The program Win32k.sys identifies various display drivers, and loads actual instances of display drivers in separate session spaces. The actual instances are loaded once the program Win32k.sys initiates a low level graphics infrastructure 112 to determine if we need to open the mirror driver;” para. [0029] “In the case of remote sessions, the program win32k.sys passes the handle down to an actual display driver 208, for example, Remote Desktop Protocol Display Driver (RDPDD) or Remote Desktop Protocol Encoder Display Driver (RDPENCDD). The handle may be used to make an “IOCTL” call to the video port requesting for “IOCTL_VIDEO_QUERY_NUM_AVAIL_MODES”. In other words, this call is frrom Win32k.sys to the video port and asks for the capabilities of a device as to how many modes does the device support.” Abdo describes remote terminal server sessions being instantiated, and explicitly states that display driver instances are loaded in separate session spaces, and that a mirror driver is “loaded with a session space” of the server, aligning with the claim element. Abdo also discloses that the system determines “if we need to open the mirror driver” and then loads the instance. We can treat the “need to open” as “indication to use” aligning with the claim element.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Sampath into the combination of teachings of Schmieder, Zhu and Abdo in order to adapt the per-session driver loading logic in order to manage remote session display components in a terminal server environment.
Regarding claim 19, the combination of Schmieder, Zhu and Abdo disclose all the elements of claim 16 as discussed above. Zhu also discloses wherein the video encoder comprises a high efficiency video coding encoder or an advanced video coding encoder (para. [0035] “In particular examples, the encoder 110 can be compliant with one or more codecs such as an MPEG-4 AVC/H.264 or HEVC/H.265 codec. Other types of standards-based encoding schemes or codecs could be used as well.” This shows explicit disclosure of AVC/H.264 being an advanced video encoder.)
Claim(s) 20 is rejected under 35 U.S.C. 103 as being unpatentable over Schmieder as modified by Zhu and Abdo as applied to claim 16 above, and further in view of Raduchel (US-20180063555-A1) and Jia(US-20130294689-A1).
Regarding claim 20, the combination of Schmieder, Zhu and Abdo disclose all the elements of claim 16 as discussed above. However, the combination does not disclose wherein the video encoder comprises a hardware of software component used to encode video content that includes high dynamic range information.
The combination of Schmieder, Zhu, Abdo, Raduchel and Jia disclose (Raduchel teaches in para.[0089] “In some implementations, the server systems 300A and 300B can include a separate encoder, e.g., a software-based encoder or a hardware-based encoder, which is physically located outside of the GPU chip 310. The separate encoder can perform encoding operations to generate the encoded video data 301 without using memory bandwidth of the graphics memory 320.” This is explicit disclosure that the encoder can be software or hardware based to perform the encoding operations that generate video data. Jia discloses the encoding in Fig.1 showing an example of an HDR image encoder.) It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the teachings of Raduchel and Jia into the combination of teachings of Schmieder, Zhu and Abdo in order to a have a system to better adapt HDR information.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRIS ALEJANDRO PUNTIER whose telephone number is (703)756-1893. The examiner can normally be reached M-F 7:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRIS ALEJANDRO PUNTIER/Examiner, Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616