DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This is in response to applicant’s amendment/response filed on 05/11/2026, which has been entered and made of record. Claims 1 and 16 have been amended. Claim 8 has been cancelled. Claim 21 has been added. Claims 1-7 and 9-21 are pending in the application.
Response to Arguments
Applicant’s arguments with respect to Claim 1 have been fully considered but they are not persuasive.
Regarding Claim 1, Applicant submits that “The art cited fails to teach and/or suggest at least this combination of features, and particularly the feature of "determining a selected resolution based on a combination of a perceived proximity of the camera to the display screen and a size of the display screen.”" (Remarks, p. 9).
The examiner disagrees with Applicant’s arguments.
Phillips recites “bandwidth allocation for a given display or stream may be allocated or adjusted based on the size of the display(s) in question, spatial distance between a position of a display and a position of the user's XR device (e.g., an XR headset)” (¶6). Further, ¶60 discloses the resolution of the video stream being based on the assigned bandwidth, and reciting “. . . more bandwidth may be allocated to displays 102b and 102c and less bandwidth may be allocated to displays 102a, 102d, and 102e, e.g., while displays 102b and 102c are in the FOV. As a result, for instance, the football games on displays 102b and 102c might be streamed at, e.g., 3840×2160 resolution and 1.84 Mbps each, while the football game on display 102a might be streamed at, e.g., 1280×720 resolution and 2350 kbps and news programs on virtual displays 102d and 102e might be streamed at, e.g., 720×480 resolution and 1750 kbps each.” Therefore, Phillips discloses adjusting the resolution of the video stream based on the perceived proximity and the size of the display(s).
Applicant’s arguments with respect to Claim 10 have been fully considered but they are not persuasive.
Regarding Claim 10, Applicant submits that “At most, Philips may therefore disclose that separate audio and video streams may be sent. However, this is not an explicit or implicit indication that the live streaming server itself splits audiovisual data into an audio stream or a video stream. Rather, at most, Philips' live streaming server may at most already receive separate audio and video streams. Accordingly, Philips fails to teach and/or suggest at least the feature of claim 10 of: "splitting, by the server, the audiovisual data into a video component and an audio component associated with the video component." (Remarks, p. 10).
The examiner disagrees with Applicant’s premises and conclusion.
Phillips disclose receiving audiovisual content such as movies and/or television programs (¶32). Zhou teaches sending audio data and video data separately. It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to split the received audiovisual data (taught by Phillips) to send audio component and video component separately. The suggestions/motivations would have been that “In this way, the second client may play the audio data when displaying the target resource. Thus, although the second client of the audience pauses the displaying of the live streaming picture, the anchor can communicate with the audience through audio, which may not cause leakage of the personal privacy of the anchor and may not interrupt the communication between the anchor and the audience.” (¶55)
Applicant’s arguments with respect to claim(s) 16 and new claim 21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s arguments directed to amended limitations of claim 16 and new claim 21 have been addressed in the detail rejection below with new reference by Nadler.
The arguments regarding dependent claims for the virtue of their dependency are moot because the independent claims are not allowable.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4, and 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phillips (US 20240430501 A1), and in view of Chen et al. (US 20240144621 A1).
Regarding Claim 1, Phillips discloses A system (ABST reciting “Systems and methods”), comprising:
a processor; (¶33 reciting “the computing device can be any device comprising a processor and memory,”) and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations (¶34 reciting “The methods and/or any instructions for performing any of the embodiments discussed herein may be encoded on computer-readable media.” ¶72), the operations comprising:
detecting that video stream data that is displayable on a display screen within a rendered representation of three-dimensional (3D) virtual environment is not within a viewport of an extended reality display device that is viewing part of the 3D virtual environment; (¶41 reciting “FIG. 2 shows another example environment in which bandwidth allocation for streams on virtual displays . . . The environment comprises a room 200. The room 200 comprises an extended reality device, . . ., virtual televisions 208b, 208c, 208d, 208e, 208f”. Further, ¶42 reciting “The augmented reality device 202 determines an extent to which each of the plurality of displays falls within the field of view of the extended reality display device. . . The determined extent to which of the displays 208a, 208b, 208c, 208d, 208e, 208f fall within the field of view defined by segments 206a, 206b, 206c, 206d”. In addition, ¶3 disclosing a rendered representation of 3D virtual environment, and reciting “Extended reality (XR) systems, such as augmented reality (AR) and virtual reality (VR) systems, can allow viewers to place virtual display screens in a spatially mapped environment”) and
in response to the detecting, taking action to halt downloading of the video stream data; (Fig. 2 showing virtual televisions 208d, 208e and 208f are not within a viewport of a camera. ¶42 reciting “displays 208d, 208e and 208f may be assigned a weighting of zero.” Further, ¶47 reciting “A stream that is paused may have a weight of zero associated with it. The weight of zero removes the device stream from the bandwidth allocation.” In other words, in response to the detecting of virtual TVs 208d, 208e, 208f are not within a viewport, halt downloading the video stream data.)
evaluating the viewport of the camera to determine whether the display screen has changed from not being within the viewport of the camera to being currently within the viewport of the camera; (Phillips, ¶66 reciting “If it is determined at 1006 that the user paused for more than the threshold limit of time, for each virtual and physical display, weights for bandwidth allocation are recalculated based on the field of vision of the user, as explained in reference to FIGS. 1, 2 and 8. For example, if a user is primarily watching their favorite team on display 102b, but turns to focus on display 102c as the game ends, 102d when there is breaking news, and/or potentially display 102e to see how the market is reacting, the user will pause for more than a threshold and the bandwidth should be readjusted to provide the user with a better viewing experience within their new view of vision.”) and
in response to the detecting that the display screen is currently within the viewport of the camera:
determining a selected resolution based on a combination of a perceived proximity of the camera to the display screen and a size of the display screen, and
taking action to resume downloading of the video stream data at the selected resolution.
(Phillips, ¶6 reciting “bandwidth allocation for a given display or stream may be allocated or adjusted based on the size of the display(s) in question, spatial distance between a position of a display and a position of the user's XR device (e.g., an XR headset)” Further, ¶60 disclosing the resolution of the video stream being based on the assigned bandwidth, and reciting “. . . more bandwidth may be allocated to displays 102b and 102c and less bandwidth may be allocated to displays 102a, 102d, and 102e, e.g., while displays 102b and 102c are in the FOV. As a result, for instance, the football games on displays 102b and 102c might be streamed at, e.g., 3840×2160 resolution and 1.84 Mbps each, while the football game on display 102a might be streamed at, e.g., 1280×720 resolution and 2350 kbps and news programs on virtual displays 102d and 102e might be streamed at, e.g., 720×480 resolution and 1750 kbps each.” Therefore, Phillips discloses adjusting the resolution of the video stream based on the size of the display(s) and the perceived proximity. In addition, Phillips ¶66 disclosing resuming downloading of the video stream data.)
Phillips discloses a head-mounted extended realty device used for the XR environment. However, Phillips does not explicitly disclose the viewport of a camera of the device.
Chen teaches “Systems and methods are provided for creating enhanced VR content.” (ABST). More specifically, Chen teaches “systems and methods may produce an enhanced image using information such as metadata used by the HMD to render an immersive experience and create views. . . Systems and methods may generate an enhanced image using metadata of the VR environment with the remote rendering device. For example, a scene or “location” within the VR environment as well as the position of the camera, the viewport of the camera, the motion or vector of the camera and other information to “re-create” the image or video may be sent to the remote rendering device.” (¶58).
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to combine the teachings from Phillips and Chen to use a viewport of a camera of a device, such as HMD. The suggestions/motivations would have been to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 2, Phillips in view of Chen discloses The system of claim 1, wherein the taking of the action to halt the downloading of the video stream data comprises communicating a request to a server. (Phillips, ¶49 reciting “A request to pause a stream will occur at the device itself and such requests may be transmitted to a back-end system for controlling the switching of a stream from a live mode to a pause mode.”)
Regarding Claim 4, Phillips in view of Chen discloses The system of claim 1, wherein the operations further comprise detecting that the display screen is currently within the viewport, and in response to the detecting that the display screen is currently within the viewport, taking action to resume downloading of the video stream data. (Phillips, ¶66 reciting “If it is determined at 1006 that the user paused for more than the threshold limit of time, for each virtual and physical display, weights for bandwidth allocation are recalculated based on the field of vision of the user, as explained in reference to FIGS. 1, 2 and 8. For example, if a user is primarily watching their favorite team on display 102b, but turns to focus on display 102c as the game ends, 102d when there is breaking news, and/or potentially display 102e to see how the market is reacting, the user will pause for more than a threshold and the bandwidth should be readjusted to provide the user with a better viewing experience within their new view of vision.”)
Regarding Claim 6. Phillips in view of Chen discloses The system of claim 4, wherein the taking of the action to resume the downloading of the video stream data results in downloading resumed video stream data, and wherein the operations further comprise adjusting the resolution of the resumed video stream data based on perceived proximity of the camera to the display screen within the 3D environment. (Phillips, ¶13 disclosing adjusting bandwidth based on the distance from the XR device (i.e. camera within the XR device) to the display screen, and reciting “the bandwidth amount assigned to at least one of the plurality of content streams is adjusted based on a distance from the XR device to each display of the plurality of displays.” Further, ¶60 disclosing the resolution of the video stream being based on the assigned bandwidth, and reciting “. . . more bandwidth may be allocated to displays 102b and 102c and less bandwidth may be allocated to displays 102a, 102d, and 102e, e.g., while displays 102b and 102c are in the FOV. As a result, for instance, the football games on displays 102b and 102c might be streamed at, e.g., 3840×2160 resolution and 1.84 Mbps each, while the football game on display 102a might be streamed at, e.g., 1280×720 resolution and 2350 kbps and news programs on virtual displays 102d and 102e might be streamed at, e.g., 720×480 resolution and 1750 kbps each.” Therefore, Phillips discloses adjusting the resolution of the video stream based on the perceived proximity.)
Claim(s) 16 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phillips (US 20240430501 A1), and in view of Chen et al. (US 20240144621 A1), and further in view of Nadler (US 20160094837 A1).
Regarding Claim 16, Phillips in view of Chen discloses A non-transitory machine-readable medium, comprising executable instructions that, when executed by at least one processor, facilitate performance of operations (¶34 reciting “The methods and/or any instructions for performing any of the embodiments discussed herein may be encoded on computer-readable media.” ¶72), the operations comprising:
downloading streamed video from a server; (¶41 reciting “a modem 212 transmits content and data to and from the devices 202, 208a, 208b, 208c, 208d, 208e, 208f via network 214 to a server 216.”)
rendering a representation of a three-dimensional (3D) virtual environment from a perspective of a viewport of a camera viewing at least part of the 3D virtual environment (Phillips, ¶3 disclosing a rendered representation of 3D virtual environment, and reciting “Extended reality (XR) systems, such as augmented reality (AR) and virtual reality (VR) systems, can allow viewers to place virtual display screens in a spatially mapped environment”. In addition, Chen teaches “systems and methods may produce an enhanced image using information such as metadata used by the HMD to render an immersive experience and create views. . . a scene or “location” within the VR environment as well as the position of the camera, the viewport of the camera, the motion or vector of the camera and other information to “re-create” the image or video may be sent to the remote rendering device.” (¶58)), wherein the representation includes a video display screen, within the viewport, on which the streamed video is rendered, (Phillis, ¶41 reciting “FIG. 2 shows another example environment in which bandwidth allocation for streams on virtual displays . . . The environment comprises a room 200. The room 200 comprises an extended reality device, . . ., virtual televisions 208b, 208c, 208d, 208e, 208f”) and wherein the camera and the video display screen are associated with a first perceived proximity to one another that determines a first resolution at which the streamed video is rendered on the video display screen; (Phillips, ¶13 disclosing adjusting bandwidth based on the distance from the XR device (i.e. camera within the XR device) to the display screen, and ¶60 disclosing the resolution of the video stream being based on the assigned bandwidth, and reciting “. . . more bandwidth may be allocated to displays 102b and 102c and less bandwidth may be allocated to displays 102a, 102d, and 102e, e.g., while displays 102b and 102c are in the FOV. As a result, for instance, the football games on displays 102b and 102c might be streamed at, e.g., 3840×2160 resolution and 1.84 Mbps each, while the football game on display 102a might be streamed at, e.g., 1280×720 resolution and 2350 kbps and news programs on virtual displays 102d and 102e might be streamed at, e.g., 720×480 resolution and 1750 kbps each.” Therefore, Phillips discloses the resolution of the video stream based on the perceived proximity.)
determining that the first perceived proximity has changed to a second perceived proximity that corresponds to a second resolution at which the streamed video is to be rendered on the video display screen, wherein the first perceived proximity is different from the second perceived proximity, and wherein the first resolution is different from the second resolution (Phillips, ¶13 disclosing adjusting bandwidth based on the distance from the XR device (i.e. camera within the XR device) to the display screen; and ¶37 disclosing a viewer may periodically turn her head and focus on different virtual displays. Further, ¶66 recites “if a user is primarily watching their favorite team on display 102b, but turns to focus on display 102c as the game ends, 102d when there is breaking news, and/or potentially display 102e to see how the market is reacting, the user will pause for more than a threshold and the bandwidth should be readjusted to provide the user with a better viewing experience within their new view of vision. At 1012, for each virtual and physical display, the distance to each display and size of display are determined. At 1012, based on FoV, distance, and size, the bandwidth is adjusted, as explained in reference to FIGS. 1 and 2.” Thus, Phillips discloses a new resolution of a stream is requested periodically.), and in response:
requesting that the streamed video be downloaded from the server at the second resolution;
receiving the streamed video at the second resolution from the server; and
rendering the streamed video on the video display screen in the representation of the 3D virtual environment;
(Phillips, ¶49 disclosing a request to change the streaming may be transmitted to a back-end system (i.e. server). ¶13 disclosing adjusting bandwidth based on the distance from the XR device (i.e. camera within the XR device) to the display screen; and ¶60 disclosing changing the resolution of the video stream based on the assigned bandwidth, and reciting “. . . more bandwidth may be allocated to displays 102b and 102c and less bandwidth may be allocated to displays 102a, 102d, and 102e, e.g., while displays 102b and 102c are in the FOV. As a result, for instance, the football games on displays 102b and 102c might be streamed at, e.g., 3840×2160 resolution and 1.84 Mbps each, while the football game on display 102a might be streamed at, e.g., 1280×720 resolution and 2350 kbps and news programs on virtual displays 102d and 102e might be streamed at, e.g., 720×480 resolution and 1750 kbps each.”)
and
determining that the video display screen is no longer within the viewport, and in response, requesting that the downloading of the streamed video from the server be halted. (Phillips, Fig. 2 showing virtual televisions 208d, 208e and 208f are not within a viewport of a camera. ¶42 reciting “displays 208d, 208e and 208f may be assigned a weighting of zero.” Further, ¶47 reciting “A stream that is paused may have a weight of zero associated with it. The weight of zero removes the device stream from the bandwidth allocation.” In other words, in response to the detecting of virtual TVs 208d, 208e, 208f are not within a viewport, halt downloading the video stream data.)
(The suggestions/motivations for the combination of Phillips and Chen would have been the same as that of Claim 1 rejections.)
However, Phillips in view of Chen does not explicitly disclose determining that the second perceived proximity corresponds to a second true pixel size different from a first true pixel size associated with the first perceived proximity, determining a second resolution that corresponds to the second proximity, wherein the first true pixel size s different from the second true pixel size.
It is well known in the art the pixel size is related to the perceived proximity. In addition, Nadler teaches “the near distance viewing which requires very small pixel sizes to retain the illusion of reality.” (¶53). Therefore, Nadler teaches a perceived proximity is associated a pixel size. Thus the pixel sizes associated with different perceived proximity are different.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to combine the teachings from Phillips in view of Chen and Nadler to retain the illusion of reality.
Regarding Claim 18. Phillips in view of Chen and Nadler discloses The non-transitory machine-readable medium of claim 16, wherein the operations further comprise, after the downloading of the streamed video has been halted, determining that the video display screen is again within the viewport, and in response, requesting that the downloading of the streamed video from the server be resumed. (See Claim 16 and 4 rejections for detailed analysis.)
Regarding Claim 19. Phillips in view of Chen and Nadler discloses The non-transitory machine-readable medium of claim 18, wherein the operations further comprise, after the downloading of the streamed video has been halted, determining that the video display screen is again within the viewport, determining a third perceived proximity of the camera to the video display screen that corresponds to a third resolution at which the streamed video is to be rendered on the video display screen, wherein the third resolution comprises one of: the first resolution, the second resolution, or a different resolution from the first resolution and the second resolution, and requesting that the streamed video from the server be downloaded at the third resolution. (Phillips discloses a new resolution of a stream is requested periodically and the resolution of a stream depends on the distance from the camera of the XR device to the display screen. ¶13 disclosing adjusting bandwidth based on the distance from the XR device (i.e. camera within the XR device) to the display screen; and ¶37 disclosing a viewer may periodically turn her head and focus on different virtual displays. Further, ¶66 recites “if a user is primarily watching their favorite team on display 102b, but turns to focus on display 102c as the game ends, 102d when there is breaking news, and/or potentially display 102e to see how the market is reacting, the user will pause for more than a threshold and the bandwidth should be readjusted to provide the user with a better viewing experience within their new view of vision. At 1012, for each virtual and physical display, the distance to each display and size of display are determined. At 1012, based on FoV, distance, and size, the bandwidth is adjusted, as explained in reference to FIGS. 1 and 2.”)
Regarding Claim 20. Phillips in view of Chen and Nadler discloses The non-transitory machine-readable medium of claim 16, wherein the operations further comprise, after the downloading of the streamed video has been halted, determining that the video display screen is again within the viewport, determining a third perceived proximity of the camera to the video display screen that corresponds to a third resolution at which the streamed video is to be rendered on the video display screen, wherein the third resolution comprises one of: the first resolution, the second resolution, or a different resolution from the first resolution and the second resolution, and requesting that the downloading of the streamed video from the server be resumed at the third resolution. (Phillips discloses a new resolution of a stream is requested periodically and the resolution of a stream depends on the distance from the camera of the XR device to the display screen. ¶13 disclosing adjusting bandwidth based on the distance from the XR device (i.e. camera within the XR device) to the display screen; and ¶37 disclosing a viewer may periodically turn her head and focus on different virtual displays. Further, ¶66 recites “if a user is primarily watching their favorite team on display 102b, but turns to focus on display 102c as the game ends, 102d when there is breaking news, and/or potentially display 102e to see how the market is reacting, the user will pause for more than a threshold and the bandwidth should be readjusted to provide the user with a better viewing experience within their new view of vision. At 1012, for each virtual and physical display, the distance to each display and size of display are determined. At 1012, based on FoV, distance, and size, the bandwidth is adjusted, as explained in reference to FIGS. 1 and 2.”)
Claim(s) 3 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phillips (US 20240430501 A1), in view of Chen et al. (US 20240144621 A1), and further in view of Zhou (US 20220286716 A1).
Regarding Claim 3. Phillips in view of Chen discloses The system of claim 1, wherein the taking of the action to halt the downloading of the video stream data results in halted video stream data.
However, Phillips in view of Chen does not explicitly disclose wherein the operations further comprise receiving downloaded audio stream data that is associated with the halted video stream data, and playing the audio stream data associated with the halted video stream data.
Zhou teaches “a method and an apparatus for live streaming” (ABST). More specifically, ¶54 recites “pausing by the live streaming sever acquisition of the live streaming data from the first client may include pausing by the live streaming sever acquisition of image data from the first client. . . ., the method may further include maintaining acquisition of audio data from the first client; and sending the audio data to the second client to play the audio data during the displaying of the target resource on the second client.”
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the system (taught by Phillips in view of Chen) to maintain receiving and playing associated audio data (taught by Zhou). The suggestions/motivations would have been that “In this way, the second client may play the audio data when displaying the target resource. Thus, although the second client of the audience pauses the displaying of the live streaming picture, the anchor can communicate with the audience through audio, which may not cause leakage of the personal privacy of the anchor and may not interrupt the communication between the anchor and the audience.” (¶55), and to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 17. Phillips in view of Chen and Zhou discloses The non-transitory machine-readable medium of claim 16, wherein the operations further comprise downloading streamed audio, associated with the streamed video, from the server, and outputting the streamed audio independent of whether the downloading of the streamed video is ongoing or has been halted. (see Claim 16 and 3 rejections for detailed analysis.)
Claim(s) 5 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phillips (US 20240430501 A1), in view of Chen et al. (US 20240144621 A1), and further in view of Dessero et al. (US 20250308187 A1).
Regarding Claim 5, Phillips in view of Chen discloses The system of claim 4.
However, Phillips in view of Chen does not explicitly disclose wherein the operations further comprise rendering the video stream data, via a graphics processing unit, as texture data in the rendered representation of the 3D virtual environment.
It is well known in the art to render VR contents via a GPU. In addition, Dessero teaches “displaying a representation of respective media in a three-dimensional environment” (ABST). More specifically, ¶184 recites “the controller 110 renders AR or VR frames 562”. In addition, ¶115 teaches the controller 110 includes graphics processing unit, and recites “the controller 110 includes one or more processing units 202 (e.g., microprocessors, application-specific integrated-circuits (ASICs), field-programmable gate arrays (FPGAs), graphics processing units (GPUs), central processing units (CPUs), processing cores, and/or the like)”.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the system (taught by Phillips in view of Chen) to render a representation of media in a virtual 3D environment via a GPU (taught by Dessero). The suggestions/motivations would have been to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 7. Phillips in view of Chen and Dessero discloses The system of claim 6, wherein the operations further comprise taking further action to readjust the resolution of the resumed video stream data based on a change in the perceived proximity of the camera to the display screen within the 3D environment. (Phillips discloses adjusting the resolution of the video stream based on the proximity. Dessero, ¶232 teaching displaying the representation of the media at a different distance, and reciting “representation 702 is displayed in three-dimensional environment 700 . . . computer system 101 displays representation 702 at a different distance (e.g., closer or farther) relative to a location corresponding to the current viewpoint of the user of computer system . . . ”. The suggestions/motivations would have been the same as that of Claim 5 rejections.)
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phillips (US 20240430501 A1), in view of Chen et al. (US 20240144621 A1), and further in view of Miyanaga (US 20230088340 A1).
Regarding Claim 9. Phillips in view of Chen discloses The system of claim 8.
Phillips discloses the weights (associated with evaluating of the viewport) may be updated at regular periods (¶42). However, Phillips in view of Chen does not explicitly disclose wherein the evaluating of the viewport of the camera occurs at a rate of once per rendering frame.
Miyanaga teaches “In a case where the progress of the game in the virtual space is managed by a frame (sampling a game state at appropriate time intervals and synchronizing at a frequency of sampling), the primary copy is updated at a frame rate (for example, 20 times per second)” (¶133). In other words, Miyanaga teaches updating a virtual object at a rate of once per rendering frame.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to combine the teachings from Phillips in view of Chen and Miyanaga to evaluate the viewport of the camera for updating at a rate of once per rendering frame. The suggestions/motivations would have been “in order to reduce the transmission frequency of the information” (¶133), and to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Claim(s) 10-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phillips (US 20240430501 A1), and in view of Zhou (US 20220286716 A1).
Regarding Claim 10, Phillips discloses A method (ABST reciting “Systems and methods ”), comprising:
obtaining, at a server comprising at least one processor, audiovisual data; (¶32 reciting “A content item may include audio, video, text and/or any other media content. . . Video includes audiovisual content such as movies and/or television programs. . . Content items may, for example, be streamed to physical computing devices.” Further, ¶41 reciting “a modem 212 transmits content and data to and from the devices 202, 208a, 208b, 208c, 208d, 208e, 208f via network 214 to a server 216.” In addition, ¶33 reciting “the computing device can be any device comprising a processor and memory”)
streaming, by the server, video at a first resolution to the client device; (¶36 reciting “each stream may be set to an initial bandwidth. For example, the football games on displays 102a, 102b, and 102c might be streamed at, e.g., 1920×1080 resolution and 4300 kbps each, while the news programs on virtual displays 102d and 102e might be streamed at, e.g., 1280×720 resolution and 2350 kbps each.”)
streaming, by the server, audio to the client device; (¶32 reciting “A content item may include audio, . . . Audio includes audio-only content,”)
receiving a request from the client device to stream the video component at a second resolution that is different from the first resolution; and
in response to the request, streaming, the video component at the second resolution.
(¶42 disclosing a client device 202 determines the bandwidth. Further, ¶60 disclosing changing the resolution of the video stream based on the assigned bandwidth, and reciting “. . . more bandwidth may be allocated to displays 102b and 102c and less bandwidth may be allocated to displays 102a, 102d, and 102e, e.g., while displays 102b and 102c are in the FOV. As a result, for instance, the football games on displays 102b and 102c might be streamed at, e.g., 3840×2160 resolution and 1.84 Mbps each, while the football game on display 102a might be streamed at, e.g., 1280×720 resolution and 2350 kbps and news programs on virtual displays 102d and 102e might be streamed at, e.g., 720×480 resolution and 1750 kbps each.” In addition, ¶49 disclosing a request to change the streaming may be transmitted to a back-end system (i.e. server))
However, Phillips does not explicitly disclose splitting, by the server, the audiovisual data into a video component and an audio component associated with the video component; streaming the video component; streaming the audio component.
Zhou teaches “a method and an apparatus for live streaming” (ABST). More specifically, ¶41-42 teaches acquiring the target resource including images, audio and video, sending the acquired target resource to a client. Further, ¶54 recites “the method may further include maintaining acquisition of audio data from the first client; and sending the audio data to the second client to play the audio data during the displaying of the target resource on the second client.” In other words, Zhou teaches sending separate target resource, such as audio or video, to a client, which implies the video component and audio component have been split from the audiovisual data.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the system (taught by Phillips) to send acquired target resource, such as the video component or the audio component, to the client (taught by Zhou). The suggestions/motivations would have been that “In this way, the second client may play the audio data when displaying the target resource. Thus, although the second client of the audience pauses the displaying of the live streaming picture, the anchor can communicate with the audience through audio, which may not cause leakage of the personal privacy of the anchor and may not interrupt the communication between the anchor and the audience.” (¶55), and to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 11, Phillips in view of Zhou discloses The method of claim 10, wherein the request is a first request from the client device, and further comprising receiving, by the server, a second request from the client device to stream the video component at a third resolution that is different from the second resolution, and in response to the second request, streaming, by the server, the video component at the third resolution. (Phillips, ¶37 disclosing a viewer may periodically turn her head and focus on different virtual displays. Further, ¶60 discoing the bandwidth amounts assigned to the content streams are adjusted based on which displays are in the field of vision of the user; and resolution is adjusted accordingly. Furthermore, ¶66 recites “if a user is primarily watching their favorite team on display 102b, but turns to focus on display 102c as the game ends, 102d when there is breaking news, and/or potentially display 102e to see how the market is reacting, the user will pause for more than a threshold and the bandwidth should be readjusted to provide the user with a better viewing experience within their new view of vision. At 1012, for each virtual and physical display, the distance to each display and size of display are determined. At 1012, based on FoV, distance, and size, the bandwidth is adjusted, as explained in reference to FIGS. 1 and 2.” Thus, Phillips discloses a new resolution of a stream is requested periodically.)
Regarding Claim 12, Phillips in view of Zhou discloses The method of claim 10, wherein the request is a first request from the client device, and further comprising receiving, by the server, a second request from the client device to halt the streaming of the video component, and in response to the request, halting the streaming of the video component to the client device, and continuing the streaming of the audio component to the client device. (Zhou, ¶35 reciting “In S202, in a case that the first client receives a live streaming pausing operation of an anchor, the first client sends a live streaming pausing command to the live streaming server.” Further, ¶54 reciting “pausing by the live streaming sever acquisition of the live streaming data from the first client may include pausing by the live streaming sever acquisition of image data from the first client. . . ., the method may further include maintaining acquisition of audio data from the first client; and sending the audio data to the second client to play the audio data during the displaying of the target resource on the second client.” The suggestions/motivations would have been the same as that of Claim 10 rejections.)
Regarding Claim 13, Phillips in view of Zhou discloses The method of claim 12, wherein the video component comprises a first video component that is halted, wherein the audio component comprises a first audio component that continues to be streamed, and further comprising streaming, by the server, a second video component to the client device, and streaming, by the server, a second audio component to the client device, while the first video component is halted and the first audio component continues to be streamed. (Phillips, ¶42 reciting “The augmented reality device 202 determines an extent to which each of the plurality of displays falls within the field of view of the extended reality display device. . . displays 208d, 208e and 208f may be assigned a weighting of zero.” Further, ¶47 reciting “A stream that is paused may have a weight of zero associated with it. The weight of zero removes the device stream from the bandwidth allocation.” In other words, in response to the detecting of virtual TVs 208d, 208e, 208f are not within a viewport, halt downloading the video stream data while the streaming to TVs 208a, 208b, 208c continues. In addition, Zhou teaches halting the video component and continuing streaming video component (see Claim 12 rejections). The suggestions/motivations would have been the same as that of Claim 10 rejections.)
Regarding Claim 14, Phillips in view of Zhou discloses The method of claim 12, further comprising receiving, by the server, a third request from the client device to resume the streaming of the video component, and in response to the request, resuming the streaming of the video component to the client device. (Phillips, ¶66 reciting “If it is determined at 1006 that the user paused for more than the threshold limit of time, for each virtual and physical display, weights for bandwidth allocation are recalculated based on the field of vision of the user, as explained in reference to FIGS. 1, 2 and 8. For example, if a user is primarily watching their favorite team on display 102b, but turns to focus on display 102c as the game ends, 102d when there is breaking news, and/or potentially display 102e to see how the market is reacting, the user will pause for more than a threshold and the bandwidth should be readjusted to provide the user with a better viewing experience within their new view of vision.”)
Regarding Claim 15, Phillips in view of Zhou discloses The method of claim 14, wherein the third request from the client device is associated with a request to stream the video component at a third resolution, and wherein the resuming of the streaming of the video component to the client device comprises streaming the video component at the third resolution. (See claim 14 and 11 rejections for detailed analysis)
Claim(s) 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phillips (US 20240430501 A1), and in view of Zhou (US 20220286716 A1), nd further in view of Nadler (US 20160094837 A1).
Regarding Claim 21. Phillips in view of Zhou discloses The method of claim 10.
However, Phillips in view of Zhou does not explicitly disclose wherein the second resolution is associated with a second true pixel size different from a first true pixel size associated with the first resolution.
It is well known in the art the pixel size is related to the perceived proximity. In addition, Nadler teaches “the near distance viewing which requires very small pixel sizes to retain the illusion of reality.” (¶53). Therefore, Nadler teaches a perceived proximity is associated a pixel size. Thus the pixel sizes associated with different perceived proximity are different.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to combine the teachings from Phillips in view of Zhou and Nadler to retain the illusion of reality.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YI WANG whose telephone number is (571)272-6022. The examiner can normally be reached 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YI WANG/Primary Examiner, Art Unit 2619
Prosecution Insights