Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-28 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Applicant’s amended claims recite “wherein the frame index is determined based on whether the web browser supports knowing the frame index”. Applicant’s disclosure does not disclose determining the frame index using a technique selected based on whether the web browser supports “knowing” the frame index, as broadly claimed, but explicitly and specifically “precisely knowing the current frame index before it is rendered” using a function call which calculates an exact frame index. As one of ordinary skill in the art would know, e.g. as taught by Debugging, page 11, browsers also support imprecisely knowing the current frame index before it is rendered, but Applicant’s disclosure only contemplates using browser functions which allow precisely knowing the current the current frame index before it is rendered, indicating that the scope of the amended independent claims contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Depending claims are rejected under the same rationale.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 and 25-28 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Applicant’s amended claims recite “wherein the frame index is determined based on whether the web browser supports knowing the frame index, and wherein if the support is not provided by the web browser: calculating the frame index from the video data in the frame”. As discussed in the above 35 U.S.C. 112(a) written description rejections, Applicant’s disclosure indicates that the web browser supports “precisely knowing the current frame index before it is rendered” using a function call which calculates an exact frame index. Applicant’s amended claims do not require that the browser support precisely knowing the frame index, such that imprecise methods provided by the browser are within the claim scope, which would include imprecise methods calculated based on video data in the frame, leaving the claim scope indefinite, i.e. the frame index would not be determined “based on whether the web browser supports knowing the frame index”, because the same imprecise method would be used regardless of whether the web browser supports knowing the frame index.
Depending claims do not clarify this issue and are similarly rejected, with exception of claims 21-24 which clarify the scope of the respective web browser support, i.e. as in claims 21-22 the browser support is limited to using a requestVideoFrameCallback method rather than imprecise methods, and in claims 23-24 the frame index is calculated based on pixel colors in the frame rather than any imprecise method based on video data in the frame.
For purposes of applying prior art, the claims will be interpreted as still reciting “precisely”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 12, 13, 16, 19, 21-24, and 26-28 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2017/0295379 A1 (hereinafter Sun) in view of U.S. Patent Application Publication 2017/0032562 A1 (hereinafter Block) in view of U.S. Patent Application Publication 2020/0211230 A1 (hereinafter Zhao) in view of “HTML5 - Browser and Feature Detection” by Sascha P. Corti (hereinafter Corti) in view of “Debugging” website on webrtcforthecurious.com (hereinafter Debugging).
Regarding claim 1, the limitations “a method comprising: executing in a web browser application hosted by a computing device, a program configured to render video in the web browser; identifying a plurality of shaders associated with the program; instructing a graphics processing unit (GPU) how to render video in the web browser based on the plurality of shaders associated with the program” are taught by Sun (Sun, e.g. abstract, paragraphs 1-156, discloses a system for efficient video decoding for web browser based interfaces by performing one or more video decoding and/or rendering operations using shaders operating on GPU(s) of the same computing system operating the web browser interface. Sun, e.g. paragraphs 32, 33, 48-52, 89-99, teaches that the web browser receives video data and shader program code for performing the decoding/rendering operations, i.e. the claimed web browser executing a program to render video, which identifies a plurality of shaders, and further, e.g. paragraphs 49-51, using an API to provide the shader(s) to the GPU for use in decoding/rendering video content, i.e. the claimed instructing a GPU how to render video based on the plurality of shaders.)
The limitations “accessing, by the program, video data and metadata associated with the video to be rendered … and communicating the video data associated with the video to be rendered … to the GPU for viewing by a user proximate a display device coupled to the GPU” are taught by Sun (Sun, e.g. paragraphs 64-69, teaches that the received video bitstreams include both coded video data and syntax elements including protocol/format specific metadata, i.e. the claimed accessing video data and metadata associated with the video. Sun further teaches, e.g. paragraphs 40, 42, 49-52, 67, that the video bitstream data is provided to the GPU for performing the decoding and/or rendering operations to generate output frames for display on a display device viewed by a user, i.e. the claimed communicating the video data to the GPU to be rendered for viewing by a user proximate to a display device coupled to the GPU.)
The limitations “wherein one or more instructions associated with the instructing are stored at least in part in one or more uniform structures … embedding the metadata into at least one uniform structure in a frame-specific synchronous manner corresponding to a current frame of the video … communicating the … uniform structures to the GPU … wherein the GPU renders the video data at based at least in part on the metadata … embedded in the uniform structures” are not explicitly taught by Sun (Sun, e.g. paragraphs 60, 63, 70, 79-81, teaches that the decoding and/or rendering operations performed by the shader may rely on decoding parameters that may be set on a per-tile, per-slice, or per-picture/frame basis, i.e. the protocol/format specific metadata included in the video bitstream includes at least some metadata that is synchronized to individual frames and used in rendering the video data. Sun does not explicitly address the use of shader uniform variables, per se, i.e. the claimed uniform structures.) However, this limitation is taught by Block (Block, e.g. abstract, paragraphs 41-116, describes a system for developing and using web applications for rendering and compositing synchronized 3D graphics with decoded video frames to present interactive visual aids along with a previously generated video. Block, e.g. paragraphs 57-59, teaches that the application runs in a web browser and uses a GPU to perform graphics processing, including executing a rendering loop once for each displayed frame, e.g. paragraphs 68, 69, 94, 97, 99-104, figure 15, using a Video Model class 545 to control video playback, decoding, and time tracking, e.g. paragraphs 66, 67, and a Moments Model 543 to create and modify 3D scene/object properties in synchronization with the current frame/time of the video, e.g. paragraphs 66, 70-72, 83-85, 95, 96, 98, 100, 102, i.e. the Moment objects comprise properties/metadata which are specified in a frame-specific synchronous manner corresponding to particular frames of the video. Further, Block teaches that both the video playback, e.g. paragraph 76, as well as the 3D graphics object rendering, e.g. paragraphs 58, 79-81, 87, 96, 102, can be performed using shaders operating on the GPU, where the above noted frame-specific synchronized properties/metadata are specified using shader uniform variables, e.g. paragraphs 79, 96, 102, figures 10, 11, i.e. as claimed, the metadata is embedded into at least one uniform structure in a frame-specific synchronous manner corresponding to a current frame of video.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sun’s efficient web browser video decoding system to include Block’s synchronized 3D graphics rendering and compositing technique in order to support synchronous playback of interactive 3D visual aids composited with playback of a video. In the modified system, Sun’s efficient video decoding using shaders would be performed without modification, e.g. Block, paragraph 76, suggests an analogous embodiment in which shaders perform video decoding/playback, and frame-specific metadata would be stored in shader uniform variables and updated in synchronization with the current video frame/time, e.g. Block, paragraphs 70, 79-81, 96, 100, 102, i.e. as claimed, the frame-specific metadata/instructions for rendering are embedded/stored in uniform structures in a frame-specific synchronous manner corresponding to the current frame of the video.
The limitations “wherein the embedded metadata includes an exact frame index of the video to be rendered before the current frame is rendered, [wherein if the web browser does not support precisely knowing the frame index]; precisely calculating the frame index from video data in the frame … wherein the GPU renders the video data based at least in part on the metadata, including the frame index, embedded in the uniform structures” are not explicitly taught by Sun in view of Block (In Sun’s modified system, as taught by Block, e.g. paragraphs 79-81, 96, 102, as discussed above, the frame-specific metadata would be stored in shader uniform variables and updated in synchronization with the current video frame/time, i.e. embedding the metadata into uniform structure(s) in a frame-specific synchronous manner. Block, e.g. paragraph 68, teaches that different browsers may function differently and require different appropriate function calls, and analogously, different graphics libraries may be used having different functionality, e.g. paragraphs 57, 94, indicating the Three.js library is merely exemplary. That is, as implied by Block, different browsers/libraries may require executing different functionality depending on the functions supported by the web browser, i.e. executing different embedding processes based on the functions, i.e. properties, determined to be supported by the web browser. Further, Block, e.g. paragraphs 70-72, 94-102, teaches that the Moments Model tracks playback time of the video to update the status of the moments based on the time in order to synchronize the moments with the video playback, where the times are synchronized to video frames, e.g. paragraphs 83-86, i.e. Block’s GPU shaders render the video in part based on the metadata associated with a particular frame index/time. While Block’s implicit teaching corresponds to the broader function of embedding metadata determined by different mechanisms due to differing browser support, and Block’s GPU shaders render the video based on metadata associated with a particular frame index/time, neither Sun nor Block explicitly address including an exact frame index of the video in the metadata embedded in the uniform structure determined by precisely calculating the frame index from video data in the frame.) However, this limitation is taught by Zhao (Zhao, e.g. abstract, paragraphs 43-88 describes a web browser based rendering system which renders 3D mesh objects that are textured using synchronized video frames decoded from a video where the synchronization is based on corresponding frame numbers, e.g. paragraphs 50-52, 55, 56, 64-67. Zhao, e.g. paragraphs 64-68, teaches that the HTML5 video element may not provide APIs to precisely determine the current frame number for synchronization, and further teaches that the frame number for each video frame texture can be added to the texture by converting the frame number to a 16-bit binary number and overwriting a 64x4 strip of pixels using 4x4 pixels for each bit, allowing the GPU detect the frame number to maintain synchronization, i.e. as in depending claim 23, a current frame index is encoded into the video frame texture by overwriting pixels to store the frame index which is then read by the GPU shader to calculate the precise frame number of the modified video frame texture. It is further noted that one of ordinary skill in the art would have understood that Zhao’s description of paragraphs 64-66 indicates that the determined current frame number is provided as part of the input data to the hardware decoder, i.e. when the API for precisely determining the frame number is provided for the HTML5 video element, the web browser application would still have to provide the frame number provided by the API to the hardware decoder using the frame number for selecting the corresponding decoded data for rendering the frame.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Sun’s efficient web browser video decoding system, including Block’s synchronized 3D graphics rendering and compositing technique, to use Zhao’s fallback technique for encoding the precise frame number of each frame into the corresponding frame texture which is read by the GPU shader in order to support frame synchronization in web browsers which do not support precisely knowing the frame index, as taught by Zhao. That is, in Sun’s modified system, Zhao’s fallback frame index encoding/calculating technique would be used as in depending claim 23 when the web browser does not support the precise method. Further, as noted above, Block, e.g. paragraphs 70-72, 94-102, teaches that the Moments Model tracks playback time of the video to update the status of the moments based on the time in order to synchronize the moments with the video playback, where the times are synchronized to video frames, e.g. paragraphs 83-86, such that in Sun’s modified system, the GPU shaders render the video in part based on the metadata associated with a particular frame index/time, where Block’s frame-specific metadata stored in shader uniform variables and updated in synchronization with the current video frame/time would further include the determined frame number of the current frame as taught by Zhao, i.e. as claimed, the GPU renders the video based at least in part on the metadata, including the frame index, embedded in the uniform structures.
The limitations “wherein the embedded metadata includes an exact frame index of the video to be rendered before the current frame is rendered, wherein the frame index is determined based on whether the web browser supports precisely knowing the frame index, and wherein if the support is not provided by the web browser: precisely calculating the frame index from video data in the frame” are implicitly taught by Sun in view of Block and Zhao (As noted above, as implied by Block, different browsers/libraries may require executing different functionality depending on the functions supported by the web browser, i.e. executing different embedding processes based on the functions, i.e. properties, determined to be supported by the web browser. Further, as discussed in the modification above, in Sun’s modified system, Zhao’s fallback frame index encoding/calculating technique would be used as in depending claim 23 when the web browser does not support the precise method. While Zhao, e.g. paragraph 65, implies that the use of the frame encoding technique is only required when the web browser does not provide the precise method, i.e. the technique is introduced after noting that the precise method may not be provided, Zhao does not explicitly discuss the system operation when the precise method is provided. In the interest of compact prosecution, Corti is cited for teaching that one of ordinary skill in the art would have known that web applications may be required to determine which function(s) are supported by a particular web browser and execute preferred or fallback code depending on what functions are determined to be supported, and Debugging is cited for teaching that the HTML5 function requestVideoFrameCallback, when supported, provides the precise frame index, in contrast to older workarounds/fallback techniques for estimating the frame index.) However, this limitation is taught by Corti in view of Debugging (Corti, e.g. pages 1-10, describes Browser and Feature detection for writing web applications using HTML5, and further teaches, e.g. pages 5-10 that performing per-feature detection and selection of preferred or fallback code improves the resulting web application compatibility with different browsers in comparison to merely detecting the browser type, i.e. as noted above, one of ordinary skill in the art would have known that web applications may be required to determine which function(s) are supported by a particular web browser and execute preferred or fallback code depending on what functions are determined to be supported. Further, Debugging, Section “Actual video time in browser”, explicitly discloses that the ‘requestVideoFrameCallback()’ API standard is a relatively new feature providing a reliable time stamp of the current video frame, and previous workarounds depended on using video.currentTime which were not precise, i.e. Debugging teaches that one of ordinary skill in the art would have recognized that Zhao’s fallback technique would only be used when it is determined that the web browser executing the web application does not support the precise method, analogous to Corti’s feature detection and code selection technique.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Sun’s efficient web browser video decoding system, including Block’s synchronized 3D graphics rendering and compositing technique, using Zhao’s fallback technique for encoding the precise frame number of each frame when the web browser does not support the precise method, to use a web application performing Corti’s feature detection and code selection technique, and further to detect whether a method of precisely knowing the current frame index for perfect synchronization is supported by the web browser as taught by Corti, Debugging, and Zhao, and use the preferred precise method when it is provided as taught by Corti and Debugging and implied By Zhao, or Zhao’s fallback technique when the preferred precise method is not provided as taught by Corti, Zhao, and Debugging. In Sun’s modified system, Block’s web application would be written using Corti’s feature detection and code selection technique to select code using functions which are supported by the browser. Further, in Sun’s modified system, the detection of browser features/properties would include determining that the claimed precise method is provided as required in depending claim 21, where the claimed precise method would be used to provide synchronized metadata as in depending claim 22 when it is provided, and Zhao’s fallback frame index encoding/calculating technique would be used when it is not provided, as recited broadly in claim 1 and more specifically in depending claim 23, where said fallback/workaround technique could include monitoring a timestamp event on the video object as in depending claim 24, as taught by Debugging.
Regarding claims 2-4, the limitations “wherein accessing video data and metadata associated with the video to be rendered includes accessing video data and metadata from at least one file and/or at least one data stream” are taught by Sun (Sun, e.g. paragraphs 59, 64, indicates that the source of the video bitstream comprising the coded video data and syntax elements including protocol/format specific metadata may be either a file or a data stream. With respect to claim 4 requiring accessing the video data and metadata from at least one file and at least one data stream, it is noted that the independent claim does not require that the web browser is playing “a video”, singular, but rather “video”, such that the claim scope would include a single web browser playing back multiple videos from multiple sources, i.e. as is conventional with modern web browsers (including before Applicant’s effective filing date), a web browser may present two or more tabs/windows playing back video from multiple sources, i.e. one tab playing back a video accessed from a locally stored file, and a second tab playing back a video accessed through a network interface, i.e. a data stream. It is further noted that this interpretation appears to be consistent with Applicant’s disclosure, which does not appear to discuss an example of a single video being rendered by accessing both a stored file and a data stream, but rather that the system is capable of rendering videos from both file or streaming sources.)
Regarding claim 5, the limitation “wherein the metadata associated with the video to be rendered is synchronized with individual frames of the video” is taught by Sun in view of Block (As discussed in the claim 1 rejection above, in Sun’s modified system, as taught by Block, e.g. paragraphs 79-81, 96, 102, as discussed above, the frame-specific metadata would be stored in shader uniform variables and updated in synchronization with the current video frame/time, i.e. embedding the metadata into uniform structure(s) in a frame-specific synchronous manner, where said metadata corresponds to the claimed metadata associated with the video and synchronized with individual frames of the video. Further, Sun, e.g. paragraphs 60, 63, 70, 79-81, teaches that the decoding and/or rendering operations performed by the shader may rely on decoding parameters that may be set on a per-tile, per-slice, or per-picture/frame basis, i.e. the protocol/format specific metadata included in the video bitstream includes at least some metadata that is synchronized to individual frames.)
Regarding claims 12 and 19, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 1 above, with Sun, e.g. paragraphs 38-41, teaching that the system may be embodied in an apparatus with processors executing programs stored on non-transitory media.
Regarding claim 13, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claims 2, 3 above.
Regarding claim 16, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 5 above.
Note: in the interest of compact prosecution, it is noted that depending claims 21-24 recite contingent limitations, see MPEP 2111.04 II, and are therefore inherently anticipated by the combination as mapped to claim 1 above. That is, with respect to claims 21, 22, if the precise method is not supported, then the contingent limitations of claim 21 and 22 are not required to be in the prior art combination in order to read on the claim. Analogously, with respect to depending claims 23, 24, if the precise method is supported, then the contingent limitations of claims 23 and 24 are not required to be in the prior art combination in order to read on the claim.
Regarding claims 21 and 22, the limitations “wherein the precisely knowing an exact frame index of the video to be rendered is performed by a requestVideoFrameCallback method when the browser supports precisely knowing the frame index”, “executing the requestVideoFrameCallback method to determine a current frame index of the current frame of the video, wherein the requestVideoFrameCallback method is passed an object that contains information to calculate the current frame index; the browser performing a lookup on the metadata for the current frame index; and the browser binding the metadata to at least one uniform structure of the uniform structures” are taught by Sun in view of Block, Zhao, and Debugging (As discussed in the claim 1 rejection above, in Sun’s modified system, the detection of browser features/properties would include determining that the claimed precise method is provided as required in depending claim 21, where the claimed precise method would be used to provide synchronized metadata as in depending claim 22 when it is provided. As taught by Zhao and Debugging, the precise method is preferred due to it’s precision, such that when said precise method is determined to be available using Conti’s feature detection, the precise method would be used by Block’s Video Model and Moment Model components performing video frame/time tracking/monitoring for performing the synchronous update of the metadata stored in the shader uniform variables, e.g. paragraphs 67, 95, 98, 100, 102. Further, Debugging, section Actual video time in browser, indicates that the ‘requestVideoFrameCallback’ method includes the rtpTimestamp field, i.e. the method is passed an object which contains information to calculate the current frame index. Finally, as taught by Block, the metadata used to update the shader uniform variables is selected based on the tracked/monitored video frame/time and the metadata defining each respective moment/graphical object, e.g. paragraphs 70, 102, i.e. as claimed, the browser performs a lookup on the metadata for the current frame index and binds the metadata to at least one uniform structure.)
Regarding claim 23, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 1 above, i.e. as discussed in the claim 1 rejection, Zhao’s fallback frame index encoding/calculating technique would be used as when it is determined that the precise method is not provided, where Zhao, e.g. paragraphs 64-68, teaches that the HTML5 video element may not provide APIs to precisely determine the current frame number for synchronization, and further teaches that the frame number for each video frame texture can be added to the texture by converting the frame number to a 16-bit binary number and overwriting a 64x4 strip of pixels using 4x4 pixels for each bit, allowing the GPU to detect the frame number to maintain synchronization, i.e. as in depending claim 23, a current frame index is encoded into the video frame texture by overwriting pixels to store the frame index which is then read by the GPU shader to calculate the precise frame number of the modified video frame texture.
Regarding claim 24, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 1 above, i.e. as discussed in the claim 1 rejection, said fallback/workaround technique could include monitoring a timestamp event on the video object as taught by Debugging.
Regarding claim 26, the limitation “wherein the frame index is used as an index to access-frame specific metadata included in the metadata” is taught by Sun in view of Block and Zhao (As discussed in the modification of the claim 1 rejection above, Block, e.g. paragraphs 70-72, 94-102, teaches that the Moments Model tracks playback time of the video to update the status of the moments based on the time in order to synchronize the moments with the video playback, where the times are synchronized to video frames, e.g. paragraphs 83-86, such that in Sun’s modified system, the GPU shaders render the video in part based on the metadata associated with a particular frame index/time, where Block’s frame-specific metadata stored in shader uniform variables and updated in synchronization with the current video frame/time would further include the determined frame number of the current frame as taught by Zhao, i.e. as claimed, the GPU renders the video based at least in part on the metadata, including the frame index, embedded in the uniform structures. That is, as taught by both Block and Zhao, the frame index is used for synchronization, which with respect to Block’s frame-specific metadata stored in shader uniform variables, corresponds to using the frame index as an index to access frame-specific metadata included in the metadata.)
Regarding claims 27 and 28, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 26 above.
Claims 6, 7, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2017/0295379 A1 (hereinafter Sun) in view of U.S. Patent Application Publication 2017/0032562 A1 (hereinafter Block) in view of U.S. Patent Application Publication 2020/0211230 A1 (hereinafter Zhao) in view of “HTML5 - Browser and Feature Detection” by Sascha P. Corti (hereinafter Corti) in view of “Debugging” website on webrtcforthecurious.com (hereinafter Debugging) as applied to claims 1, 12, and 19 above, and further in view of “Streaming 360-Degree Videos Using Super-Resolution” by Mallesham Dasari, et al. (hereinafter Dasari).
Regarding claim 6, the limitation “wherein the plurality of shaders store parameters associated with a neural network” is not explicitly taught by Sun (Sun, e.g. paragraphs 83, 84, teaches that the shaders performing the video decoding and/or rendering operations may perform post processing for purposes including filtering, upsampling, and color format changes, as well as, e.g. paragraph 87, that additional decoding modules could be added to the pipeline, but does not explicitly address the use of neural network(s) used in the shader module(s) performing decoding and/or rendering operations.) However, this limitation is taught by Dasari (Dasari, e.g. abstract, sections 1, 3-5, discloses the PARSEC system for streaming 360 degree videos using super-resolution processing performed by trained neural nets, i.e. DNNs. Dasari, e.g. section 1, paragraphs 3, 4, section 3, paragraphs 1, 4-8, section 4, section 5, paragraphs 3, 4, teaches that the 360 degree video segments are divided into tiles, where each tile is downsampled and compressed into a low resolution representation of the tile, and a super-resolution micro-model DNN is trained for each segment tile, e.g. as shown in figure 3, where the client device selects tiles to download and/or generate based on viewport prediction, video quality of experience, network capacity, and compute capacity, e.g. figure 4, section 4 A. That is, Dasari teaches, e.g. section 1, paragraph 4, section 4 A, paragraph 1, section 5, paragraph 4, figure 4, that the client downloads a set of compressed low resolution tiles for each 360 video segment, and a subset of the compressed low resolution tiles are used to generate the high resolution version of the compressed low resolution tile using the associated super-resolution micro-model DNN downloaded with the tile, i.e. parameters representing the trained neural network for the corresponding segment tile, corresponding to the claimed parameters associated with a neural network. Further, Dasari, e.g. section V, subsection Impact of compute capacity, indicates that the super-resolution processing is performed using client device GPUs, i.e. shader programs executing on the GPU. It is additionally noted with respect to claim 7, that the super-resolution micro-model DNN/parameters would correspond to frame-specific metadata, i.e. each segment tile is associated with a subset of the frames of the video, e.g. spans of 1, 2, or 3 seconds, as in section V, subsection “Streaming longer video segments”. Finally, it is additionally noted with respect to claim 11, that the super-resolution micro-model DNN/parameters would correspond to the claimed parameters of a differentiable computation graph, i.e. Applicant’s disclosure, paragraph 64, indicates that the models generated by TensorFlow correspond to the claimed differentiable computation graph, and Dasari, e.g. section V, paragraph 4, indicates that the DNN micro-models are trained using TensorFlow, referring to reference 10, “TensorFlow: A system for large-scale machine learning” by Martin Abadi, et al., which describes the output of TensorFlow as being the claimed differentiable computation graph parameters, e.g. section 3, figures 2, 3, where a computation graph is constructed for evaluating differentiable functions, the result of which includes a set of parameters, indicating that one of ordinary skill in the art would have recognized that Dasari’s super-resolution micro-model DNN/parameters correspond to the claimed parameters of a differentiable computation graph.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Sun’s efficient web browser video decoding system, including Block’s synchronized 3D graphics rendering and compositing technique, using Zhao’s fallback technique for encoding the precise frame number of each frame when the web browser does not support the precise method, using a web application performing Corti’s feature detection and code selection technique, detecting whether precisely knowing the current frame index is supported as taught by Corti, Debugging, and Zhao, using the precise method when it is provided or Zhao’s fallback technique when it is not provided as taught by Corti, Zhao, and Debugging, to support Dasari’s 360 degree video streaming technique by including shader module(s) for performing Dasari’s super-resolution micro-model DNNs as one of Sun’s decoding and/or rendering operations performed by shaders operated on the GPU in order to support decoding and display of 360 degree videos streamed using Dasari’s 360 degree video streaming technique, i.e. Sun’s system is intended to support effectively any type of video decoding by adding, omitting, and/or replacing the decoding modules to support the desired decompression, e.g. paragraph 87, and including shader module(s) for performing Dasari’s super-resolution micro-model DNNs as one of Sun’s shader modules would allow the system to support decoding and display of 360 degree videos streamed using Dasari’s 360 degree video streaming technique. Sun’s unmodified system would already support downloading and decoding of encoded video analogous to Dasari’s compressed low resolution segment tiles, such that Sun’s modified system would additionally perform the selection and downloading of tiles and micro-model DNN parameters based on the viewport, quality, network capacity and compute capacity as in Dasari, section 4 A, and for the subset of tiles selected for generating the high resolution tile version, Sun’s modified system would use a shader module for performing Dasari’s super-resolution micro-model DNN using the associated super-resolution micro-model DNN/parameters downloaded with the tile, corresponding to the claimed plurality of shaders storing parameters associated with a neural network.
Regarding claim 7, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 6 above, i.e. as noted in the claim 6 rejection, for the subset of tiles selected for generating the high resolution tile version, Sun’s modified system would use a shader module for performing Dasari’s super-resolution micro-model DNN using the associated super-resolution micro-model DNN/parameters downloaded with the tile, corresponding to the claimed plurality of shaders storing parameters associated with a neural network. Also noted in the claim 6 rejection, the super-resolution micro-model DNN/parameters would correspond to frame-specific metadata, i.e. each segment tile is associated with a subset of the frames of the video, e.g. spans of 1, 2, or 3 seconds, as in section V, subsection “Streaming longer video segments”.
Regarding claims 14 and 20, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claims 6 and 7 above.
Claims 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2017/0295379 A1 (hereinafter Sun) in view of U.S. Patent Application Publication 2017/0032562 A1 (hereinafter Block) in view of U.S. Patent Application Publication 2020/0211230 A1 (hereinafter Zhao) in view of “HTML5 - Browser and Feature Detection” by Sascha P. Corti (hereinafter Corti) in view of “Debugging” website on webrtcforthecurious.com (hereinafter Debugging) as applied to claims 1 and 12 above, and further in view of U.S. Patent Application Publication 2020/0027261 A1 (hereinafter Briggs).
Regarding claim 8, the limitation “wherein the plurality of shaders store at least one of an array of 3x3 rotation matrices, 4x4 rotation matrices, or other parameterizations of a pose" is not explicitly taught by Sun (Sun, e.g. paragraph 87, teaches that additional decoding modules could be added to the pipeline for supporting other types of video decoding, and Sun, e.g. paragraph 38, describes the GPU, including that many shader routines perform matrix and vector operations, but Sun does not specifically address the use of rotation matrices or pose parameterization by the shaders. Further, while Block, e.g., 108, indicates that the animation of objects may include rotation, Block also does not explicitly address the use of rotation matrices or pose parameterization by the shaders.) However, this limitation is taught by Briggs (Briggs, e.g. abstract, paragraphs 1-80, discloses a system for rendering 360 degree videos with depth content using shaders executed by a GPU. Briggs, e.g. paragraphs 22, 32-37, figure 3, describes the 360 depth video format, which includes, for each frame, a video image and a depth image, where rendering is performed by the GPU using vertex shaders for displacing vertices of the environment model surface based on the depth image, e.g. paragraphs 22, 38-42, 48-58, and pixel shaders performing ray marching to sample the video image textured onto the environment model surface, e.g. paragraphs 22, 38, 39, 46, 59-76. Further, Briggs, e.g. paragraphs 65, 67-69, figure 10 indicates that the pixel shaders can use a lookup table to determine the direction, rd , of the ray being marched, i.e. storing an array of parameterizations of pose.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Sun’s efficient web browser video decoding system, including Block’s synchronized 3D graphics rendering and compositing technique, using Zhao’s fallback technique for encoding the precise frame number of each frame when the web browser does not support the precise method, using a web application performing Corti’s feature detection and code selection technique, detecting whether precisely knowing the current frame index is supported as taught by Corti, Debugging, and Zhao, using the precise method when it is provided or Zhao’s fallback technique when it is not provided as taught by Corti, Zhao, and Debugging, to support Briggs’ 360 depth video format by including Briggs’ shader modules for performing environment model vertex displacement and pixel/fragment ray marching in Sun’s decoding and/or rendering operations performed by shaders operated on the GPU in order to support decoding and display of Briggs’ 360 depth videos, i.e. Sun’s system is intended to support effectively any type of video decoding by adding, omitting, and/or replacing the decoding modules to support the desired decompression, e.g. paragraph 87, and including shader modules for performing rendering for Briggs’ 360 degree depth videos in Sun’s set of shader modules would allow the system to support decoding and display of videos stored using Briggs’ 360 depth video format. Sun’s unmodified system would already support downloading and decoding of encoded video analogous to the video image part of Briggs’ 360 depth format, such that Sun’s modified system would additionally execute Briggs’ vertex and fragment shaders based on the user’s current viewpoint when playing 360 depth videos, where the fragment shaders would rely on Briggs’ ray direction lookup table, as discussed above, corresponding to the claimed shaders storing an array of parameterizations of pose.
Regarding claim 15, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 8 above.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2017/0295379 A1 (hereinafter Sun) in view of U.S. Patent Application Publication 2017/0032562 A1 (hereinafter Block) in view of U.S. Patent Application Publication 2020/0211230 A1 (hereinafter Zhao) in view of “HTML5 - Browser and Feature Detection” by Sascha P. Corti (hereinafter Corti) in view of “Debugging” website on webrtcforthecurious.com (hereinafter Debugging) as applied to claim 1 above, and further in view of U.S. Patent Application Publication 2020/0027261 A1 (hereinafter Briggs) in view of U.S. Patent Application Publication 2013/0106855 A1 (hereinafter Urbach).
Regarding claim 9, the limitation “wherein the plurality of shaders store a near and a far value in units of the video rendering system” is implicitly taught by Sun as modified by Briggs in the claim 8 rejection above (As discussed in the claim 8 rejection above, one of ordinary skill in the art would have found it obvious to modify Sun’s efficient web browser video decoding system to support Briggs’ 360 depth video format by including Briggs’ shader modules for performing environment model vertex displacement and pixel/fragment ray marching in Sun’s decoding and/or rendering operations performed by shaders operated on the GPU in order to support decoding and display of Briggs’ 360 depth videos. Also noted in the above claim 8 rejection, Sun’s modified system would additionally execute Briggs’ vertex and fragment shaders based on the user’s current viewpoint when playing 360 depth videos, where the fragment shaders would rely on Briggs’ ray direction lookup table to perform the ray marching in the fragment shader. Further, Briggs, e.g. paragraph 43, teaches that vertices which are too near or far from the viewpoint should be discarded, and also, e.g. paragraphs 69-80, that the ray marching process proceeds until an intersection with the depth surface is determined, but does not address termination of the ray marching process in the situation where the vertices of the depth surface are so distant that they are discarded, such that the ray marching process would not necessarily identify an intersection point with the depth surface. While Briggs does not explicitly address this situation, one of ordinary skill in the art would have known, and Urbach, discussed below, explicitly teaches, that ray marching processes can be constrained to a search space using minimum and maximum ray length values, i.e. the claimed near and far values in units of the rendering system.) However, this limitation is taught by Urbach (Urbach, e.g. abstract, paragraphs 1-70, discloses a system for rendering which includes using a GPU to perform ray tracing against an environment map model textured with video frames, e.g. paragraphs 29-57. Urbach, e.g. paragraph 44, explains that the search for the intersection can be accelerated by determining the minimum and maximum distance values that bound the search space, i.e. determining near and far values in units of the video rendering system.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Sun’s efficient web browser video decoding system, including Block’s synchronized 3D graphics rendering and compositing technique, using Zhao’s fallback technique for encoding the precise frame number of each frame when the web browser does not support the precise method, using a web application performing Corti’s feature detection and code selection technique, detecting whether precisely knowing the current frame index is supported as taught by Corti, Debugging, and Zhao, using the precise method when it is provided or Zhao’s fallback technique when it is not provided as taught by Corti, Zhao, and Debugging, supporting Briggs’ 360 depth video format by including Briggs’ shader modules for performing environment model vertex displacement and pixel/fragment ray marching in Sun’s set of shader modules, to use Urbach’s accelerated intersection search technique of tracing the ray between the minimum and maximum distances bounding the search space to constrain the search space of Briggs’ pixel/fragment ray marching. As noted above, Briggs does not explicitly address this situation where the vertices of the depth surface are so distant that they are discarded, but in the modified system wherein Briggs’ pixel/fragment shader performing ray marching, using a linear search, e.g. Briggs, paragraph 70, is modified to use Urbach’s limited search space between the minimum and maximum distances, in the situation where the vertices of the depth surface are so distant that they are discarded, the search will still be limited to the maximum distance, i.e. will not continue indefinitely. Further, in the modified system, the pixel/fragment shaders performing Briggs’ pixel/fragment ray marching within Urbach’s limited search space would store the minimum and maximum distances defining said search space, i.e. the claimed near and far values in units of the video rendering system.
Claims 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2017/0295379 A1 (hereinafter Sun) in view of U.S. Patent Application Publication 2017/0032562 A1 (hereinafter Block) in view of U.S. Patent Application Publication 2020/0211230 A1 (hereinafter Zhao) in view of “HTML5 - Browser and Feature Detection” by Sascha P. Corti (hereinafter Corti) in view of “Debugging” website on webrtcforthecurious.com (hereinafter Debugging) as applied to claims 1 and 12 above, and further in view of U.S. Patent Application Publication 2015/0103252 A1 (hereinafter Rane).
Regarding claim 10, the limitation “wherein the plurality of shaders store a gamma correction value used for per-frame gamma correction” is not explicitly taught by Sun (Sun, e.g. paragraphs 83, 84, teaches that the shaders performing the video decoding and/or rendering operations may perform post processing for purposes including filtering, upsampling, and color format changes, as well as, e.g. paragraph 87, that additional decoding modules could be added to the pipeline, and while one of ordinary skill in the art would know that gamma correction is commonly performed in image processing pipelines, Sun does not explicitly teach performing gamma correction, or by extension storing a gamma correction value used for per-frame gamma correction.) However this limitation is taught by Rane (Rane, e.g. abstract, paragraphs 2-70, discloses a system for performing video decoding with gamma correction using a GPU, which includes receiving encoded video, e.g. paragraphs 24, 35, decoding the video frames using the color adjustment unit (CAU) to perform the gamma correction on each frame based on the corresponding gamma correction value 182, e.g. paragraphs 35-37, and providing the gamma corrected frame for display, e.g. paragraph 37. Rane, e.g., paragraphs 18, 19, 25-34, 36, describe the function of the gamma function units used to perform the per-frame gamma correction, which receive the reciprocal gamma value 171 used for decoding to perform the gamma correction for each channel of each pixel of the output frame. Rane, e.g. paragraphs 38, 47, 48, 69, 70, teaches that the CAU performing the per-frame gamma correction can be a thread operating on a parallel processing unit, as well as performed by a programmable pipeline stage in a GPU, i.e. analogous to Sun’s video decoding and/or rendering operations, Rane’s gamma correction can be performed by a GPU shader program.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Sun’s efficient web browser video decoding system, including Block’s synchronized 3D graphics rendering and compositing technique, using Zhao’s fallback technique for encoding the precise frame number of each frame when the web browser does not support the precise method, using a web application performing Corti’s feature detection and code selection technique, detecting whether precisely knowing the current frame index is supported as taught by Corti, Debugging, and Zhao, using the precise method when it is provided or Zhao’s fallback technique when it is not provided as taught by Corti, Zhao, and Debugging, to include Rane’s gamma correction processing as one of Sun’s decoding and/or rendering operations performed by shaders operated on the GPU in order to support decoding of gamma-encoded video, as taught by Rane, i.e. Sun’s system is intended to support effectively any type of video decoding by adding, omitting, and/or replacing the decoding modules to support the desired decompression, e.g. paragraph 87, and including Rane’s gamma correction processing as one of Sun’s shader modules would allow the system to support decompression of gamma encoded video. As noted above, in the modified system, Rane’s gamma correction, which is analogous to Sun’s video decoding and/or rendering operations, would be implemented using a shader program executed by the GPU, corresponding to the claim requirement that the shaders store a gamma correction value (either of Rane’s values 182, 171, would correspond to the claimed value) used for performing gamma correction for each decoded video frame.
Regarding claim 17, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 10 above.
Claim 11 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2017/0295379 A1 (hereinafter Sun) in view of U.S. Patent Application Publication 2017/0032562 A1 (hereinafter Block) in view of U.S. Patent Application Publication 2020/0211230 A1 (hereinafter Zhao) in view of “HTML5 - Browser and Feature Detection” by Sascha P. Corti (hereinafter Corti) in view of “Debugging” website on webrtcforthecurious.com (hereinafter Debugging) as applied to claims 1 and 12 above, and further in view of “Streaming 360-Degree Videos Using Super-Resolution” by Mallesham Dasari, et al. (hereinafter Dasari) in view of “TensorFlow: A system for large-scale machine learning” by Martin Abadi, et al. (hereinafter Abadi).
Regarding claim 11, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 6 above, i.e. as noted in the claim 6 rejection, for the subset of tiles selected for generating the high resolution tile version, Sun’s modified system would use a shader module for performing Dasari’s super-resolution micro-model DNN using the associated super-resolution micro-model DNN/parameters downloaded with the tile, corresponding to the claimed plurality of shaders storing parameters associated with a neural network. Also noted in the claim 6 rejection, the super-resolution micro-model DNN/parameters would correspond to the claimed parameters of a differentiable computation graph, i.e. Applicant’s disclosure, paragraph 64, indicates that the models generated by TensorFlow correspond to the claimed differentiable computation graph, and Dasari, e.g. section V, paragraph 4, indicates that the DNN micro-models are trained using TensorFlow, referring to reference 10, “TensorFlow: A system for large-scale machine learning” by Martin Abadi, et al., which describes the output of TensorFlow as being the claimed differentiable computation graph parameters, e.g. Abadi, section 3, figures 2, 3, where a computation graph is constructed for evaluating differentiable functions, the result of which includes a set of parameters, indicating that one of ordinary skill in the art would have recognized that Dasari’s super-resolution micro-model DNN/parameters correspond to the claimed parameters of a differentiable computation graph.
Regarding claim 18, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 11 above.
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2017/0295379 A1 (hereinafter Sun) in view of U.S. Patent Application Publication 2017/0032562 A1 (hereinafter Block) in view of U.S. Patent Application Publication 2020/0211230 A1 (hereinafter Zhao) in view of “HTML5 - Browser and Feature Detection” by Sascha P. Corti (hereinafter Corti) in view of “Debugging” website on webrtcforthecurious.com (hereinafter Debugging) as applied to claims 1 and 12 above, and further in view of U.S. Patent Application Publication 2018/0332267 A1 (hereinafter Hesla).
Regarding claim 25, the limitations “minimizing vestibulo-ocular conflict in the video, the minimizing further comprising: tracking a motion and rotation of a camera recording the video; storing the camera’s position and rotation in each frame of the video; rendering the video in a shader; and the shader applying the camera's position and rotation in each frame to a scene geometry associated with the video rendering for each frame, wherein the rendering minimizes the vestibulo-ocular conflict by using each frame’s metadata storing the camera’s position and rotation and the uniform structures” are not explicitly taught by Sun (Although, as discussed in the claim 1 rejection above, Sun teaches rendering the video using shader(s), Sun does not address using motion and rotation data recorded for each frame of the video to minimize vestibulo-ocular conflict.) However, this limitation is taught by Hesla (Hesla describes an omnidirectional camera for capturing 360 degree videos, e.g. abstract, paragraphs 2-117. Hesla teaches that the recording device includes cameras for capturing the 360 degree videos, e.g. paragraphs 23-30, 34, 35, 42-55, and location and orientation sensors for capturing motion data characterizing the motion of the camera in correspondence to the 360 degree video frames, e.g. paragraphs 32, 33, 38, 40, 49, 50, 58-60, 68-73, where the location and orientation data is stored along with the 360 degree video data, e.g. paragraphs 38, 39, i.e. the claimed storing of the camera’s position and rotation in each frame of the video. Hesla further teaches that the location/orientation data can be used to stabilize the 360 degree video by removing transient motions identified from the location/orientation data, e.g. paragraphs 38, 41, 58-67, 74-81, i.e. as claimed, the motion data is used by the rendering process to minimize the vestibulo-ocular conflict by counteracting motion of the recording device. Finally, Hesla teaches that the processing to calculate the stabilized trajectory used to playback the recorded 360 degree video in a stabilized manner may be performed by an external/playback device, e.g. paragraphs 41, 64, 74, 75, 79-81, i.e. as claimed, the minimization is applied by the shader of the playback device rendering the video using frame synchronized position and rotation data.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Sun’s efficient web browser video decoding system, including Block’s synchronized 3D graphics rendering and compositing technique, using Zhao’s fallback technique for encoding the precise frame number of each frame when the web browser does not support the precise method, using a web application performing Corti’s feature detection and code selection technique, detecting whether precisely knowing the current frame index is supported as taught by Corti, Debugging, and Zhao, using the precise method when it is provided or Zhao’s fallback technique when it is not provided as taught by Corti, Zhao, and Debugging, to perform Hesla’s video stabilization technique for 360 degree videos captured using Hesla’s omnidirectional camera in order to present stabilized 360 degree videos to the user, i.e. one of ordinary skill in the art would recognize that Sun’s system is compatible with 360 degree videos, and Hesla teaches that the video stabilization can be performed by an external processing system such as Sun’s system, e.g. paragraphs 41, 64, 74, 75, 79-81, where one of ordinary skill in the art would be motivated to ensure that Sun’s system performs the video stabilization because unstabilized videos may have various drawbacks such as important footage being out of view, sudden movements and/or vibrations, noted by Hesla, e.g. paragraphs 5, 38, 76, 77. In Sun’s modified system, as discussed above with respect to Hesla’s omnidirectional camera and 360 degree videos, the received video metadata would include the claimed position and rotation of the camera used to counteract motion thereof, and Sun’s modified system would perform the video stabilization during rendering with the GPU shaders by calculating a stabilized trajectory for use during playback of corresponding 360 degree videos captured by Hesla’s omnidirectional camera, corresponding to the claimed minimizing vestibulo-ocular conflict by rendering using frame synchronized metadata comprising the camera position and rotation.
Response to Arguments
Applicant’s arguments, see page 10, filed 1/21/26, with respect to the 35 U.S.C. 112(b) rejection of claim 25 have been fully considered and are persuasive. The 35 U.S.C. 112(b) rejection of claim 25 has been withdrawn.
Applicant's arguments filed 1/21/26 have been fully considered but they are not persuasive.
Applicant’s arguments with respect to Block are limited to considering paragraphs 9, 18, 19, 66, and 68, without actually addressing the paragraphs cited for teaching synchronization based on both video time and specific frames corresponding to the current frame or frame-specific video metadata frames. Applicant asserts that Block does not describe that the timing information is embedded in metadata, but Applicant’s remarks fail to suggest how the cited portions of Block which discuss the frame-specific metadata fail to teach the claim limitations, and in particular fail to address Block’s explanation of the compositing performed by rendering loop in paragraphs 92-102, which includes updating the frame-specific metadata in the shader uniform variables based on the video frame. Therefore this argument cannot be considered persuasive.
Applicant’s remarks with respect to Zhao generically assert that Zhao does not teach the entire claim limitation to which the combination of references are mapped, but fails to contradict the above discussed rationale that one of ordinary skill in the art would understand that Zhao’s description of paragraphs 64-66 indicates that the determined current frame number is provided as part of the input data to the hardware decoder, i.e. when the API for precisely determining the frame number is provided for the HTML5 video element, the web browser application would still have to provide the frame number provided by the API to the hardware decoder using the frame number for selecting the corresponding decoded data for rendering the frame. Therefore this argument also cannot be considered persuasive.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Applicant’s remarks do not actually address the combination of references, and instead merely show that no reference individually anticipates the entire claim limitation to which multiple references are mapped in combination. That is, aside from Applicant’s remarks with Block not being persuasive, as noted above, Applicant indicates that Block is silent with respect to embedding a frame index in the metadata, but does not actually address the combination of Sun modified in view of Block and Zhao, where Zhao’s web browser application has a fallback technique for determining the frame index which is used for synchronization with the hardware decoder, i.e. Zhao’s synchronization is based on directly providing the frame index to the hardware decoder to use for selecting the correctly decoded mesh data for the current frame. As noted above, Applicant’s remarks fail to contradict this rationale. Instead, Applicant’s remarks then erroneously conclude that because none of the references individually teach the entire claim limitation to which the combination of references are mapped, the combination of references cannot teach the entire claim limitation. However, because the combination of references teach the entire claim limitation, Applicant’s remarks cannot be considered persuasive.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT BADER whose telephone number is (571)270-3335. The examiner can normally be reached 11-7 m-f.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at 571-272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT BADER/Primary Examiner, Art Unit 2611