Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the application filed September 5, 2025, claims 22-42 are presented for examination. Claims 22, 30 and 37 are independent claims.
Priority
Examiner acknowledges the claims for domestic priority under 35 U.S. C. 119 (e) to provisional patent application 62512365, which was filed May 30, 2017.
Oath/Declaration
The Office acknowledges receipt of a properly signed Oath/Declaration submitted September 5, 2025.
Drawings
The drawings filed September 5, 2025 are accepted by the examiner.
Abstract
The abstract filed September 5, 2025 is accepted by the examiner.
Claim Objection
Claims 22-42 are objected to because of the following minor informality:
Claims 22-42 are misnumbered, please see correction below:
PNG
media_image1.png
820
698
media_image1.png
Greyscale
PNG
media_image2.png
982
766
media_image2.png
Greyscale
PNG
media_image3.png
1002
714
media_image3.png
Greyscale
PNG
media_image4.png
1150
778
media_image4.png
Greyscale
PNG
media_image5.png
944
718
media_image5.png
Greyscale
PNG
media_image6.png
1104
768
media_image6.png
Greyscale
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the "right to exclude" granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428,46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046,29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Omum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CPR 3.73(b).
Claims 22-42 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-20 of application No. 18421780 Patent 12429695 and claims 1-20 of application 16844869 (US Patent 10914957 B1). Although the conflicting claims are not identical, they are not patentably distinct from each because of their similarity to the limitations claimed in application No. 18421780 Patent 12429695 and application 16844869 (US Patent 10914957 B1).
This is an obviousness-type double patenting rejection.
Application 19320968
Patent 12429695B2
22, 30 and 37. A method, comprising: performing, by one or more processors: receiving motion vectors, wherein the motion vectors include head motion vectors determined based at least in part on information captured by one or more sensors; rendering, based at least in part on the received motion vectors, a frame comprising one or more layers at different depths overlaid on a base layer; and encoding the frame along with the received motion vectors corresponding to the layers.
1 and 11. A method, comprising: performing, by one or more processors: receiving motion vectors, wherein the motion vectors include head motion vectors determined based at least in part on information captured by one or more sensors; rendering, based at least in part on the received motion vectors, a frame comprising one or more layers at different depths overlaid on a base layer; and encoding the rendered frame along with associated motion vectors corresponding to the one or more layers of the rendered frame.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 22, 23, 24, 30, 31, 32, 37, 38 and 39 are rejected under 35 U.S.C. 103 as being unpatentable over Jung et al. (US 20170337897 Al) in view of Osman (US 20180311583 A1).
As to Claim 22:
Jung et al. discloses a head-mounted device (Jung, see Abstract, where Jung discloses that a head mounted display (HMD) device including a communication processor configured to communicate with a mobile terminal; a display configured to display a content received from the mobile terminal; and a controller configured to in response to an event generated on the mobile terminal, selectively display an object on the display corresponding to the event generated on the mobile terminal, based on a type of the displayed content), comprising: one or more processors (Jung, see controller 180 in figure 4A) configured to: receive video frames from one or more cameras (Jung, see paragraph [0049], where Jung discloses that as shown in FIG. 3A, the camera 221 may be provided at the body 200. For instance, the camera 221 may be disposed on one surface (e.g., a front surface) of the HMD body 200. The camera 221 is disposed near at least one of a left eye and a right eye, and is formed to capture (receive or input) a front image. Since the camera 221 is arranged near the eye to capture a front image, the camera 221 may acquire a scene at which a user stares, as an image); receive video frames of virtual content; and overlay or composite objects included in the video frames of the virtual content with objects included in the video frames (Jung, see paragraph [0058], where Jung discloses that the display unit 251 may project an image to a user's eyes using a prism. The prism may be formed of a transmissive material such that the user can see both the projected image and a general view at a front side (a viewing angle). An image output through the display unit 251 may be seen in an overlapped manner with a general view. The HMD 200 can provide an augmented reality (AR) which provides a single image by overlapping a real image or background with a virtual image using a characteristic of the display unit 251) from the one or more cameras (Jung, see paragraph [0049], where Jung discloses that the camera 221 is arranged near the eye to capture a front image, the camera 221 may acquire a scene at which a user stares, as an image) to generate a virtual view of an environment of the head-mounted device (Jung, see paragraph [0036], where Jung discloses a virtual space image displayed on a display unit of the HMD 200 can be an image stored in the HMD 200 or stored in the mobile terminal 100 which wirelessly communicates with the HMD 200. For instance, in case of a virtual space image stored in the HMD 200, the HMD 200 can execute an image processing and a rendering process to process the virtual space image, and may output video information generated after the image processing the rendering process to the display unit. Further, in case of a virtual space image stored in the mobile terminal 100, the mobile terminal 100 can execute an image processing and a rendering process, and may transmit video information generated after the image processing the rendering process to the HMD 200. Then, the HMD 200 can output the video information received from the mobile terminal 100); and provide the virtual view for display to a user of the head-mounted device (Jung, see 901 in figure 9A), wherein in response to a failure to receive the video frames of the virtual content (Jung, see paragraph [0120], where Jung discloses that if a stream downloading or a play of the video content 703 being played is stopped due to a network disconnection, the execution state of the video content 703 can be changed, and a notification icon 740" indicating occurrence of a corresponding event can be displayed).
Jung differs from the claimed subject matter in that Jung does not explicitly disclose that the one or more processors are further configured to: continue to provide the virtual view for display to the user, wherein the virtual view provided to the user during the failure to receive the video frames of the virtual content comprises objects included in the video frames from the one or more cameras.
However in an analogous art, Osman discloses that the one or more processors (Osman, see microcontroller 268 in figure 2D) are further configured to: continue to provide the virtual view for display to the user (Osman, see claim mapping in figure 3E, where Osman discloses a provided virtual view even while the video frame is blocked by a virtual content object (soccer ball), wherein the virtual view provided to the user during the failure to receive the video frames of the virtual content comprises objects included in the video frames from the one or more cameras (Osman, see claim mapping in figure 3E and paragraphs [0133] and [0134], where Osman discloses that similarly, at time T2, the at least one camera of the HMD detects the ball at a different angle/location as the ball continues its upward ascent to a distance of d2 and the game processing module translates the different angle/location of the ball to the corresponding location on the screen where to render the image of the ball. FIG. 3F illustrates the location 304b where the image of the ball is rendered (the location 304b may correlate with the height d2) while the remaining portions of the screen continue to render the interactive scene from the game. When the ball falls down, the movement of the ball in the downward direction is captured till the ball moves out of the capturing range of the cameras. Thus, FIG. 3D illustrates the scene rendered on the screen of the HMD at time T0 as well as at time T5, FIG. 3E illustrates the scene with the ball covering a portion 304a of the screen at time T1
(corresponding to ball ascending) as well as at time T4 (corresponding to ball descending) and FIG 3F illustrates the scene with the ball covering a portion 304b of the scene at time T2 (corresponding to ball ascending) as well as at time T3 (corresponding to ball ascending). In this embodiment, the HMD screen is considered to have transitioned from a non-transparent mode to a simulated transparent mode. In some embodiments, the image of the ball may be rendered in the foreground while the interactive scenes may be rendered in the background. Alternately, the image of the ball may be rendered in the background while the interactive scenes may be rendered in the foreground. In these embodiments, the HMD screen is considered to have transitioned from a non-transparent mode to a semi-transparent mode. The game processing module, thus, provides a user with a way to completely immerse in a game while providing a peek into scenes from the real-world based on triggering of external events or based on input from the user, such as gaze shifting, or other such visual cues explicitly or implicitly provided by the user).
PNG
media_image7.png
1264
1008
media_image7.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the invention of Jung with Osman. One would be motivated to modify Jung by disclosing that the one or more processors are further configured to: continue to provide the virtual view for display to the user, wherein the virtual view provided to the user during the failure to receive the video frames of the virtual content comprises objects included in the video frames from the one or more cameras as taught by Jeong, and thereby incorporating sophisticated operations and mathematics to produce a very realistic game experience (Osman, see paragraph [0003]).
As to Claim 23:
Jung in view of Osman discloses that the head-mounted device of claim 22, wherein the virtual view provided to the user during the failure to receive the video frames of the virtual content comprises: a message indicating that a connection for receiving the virtual content has been lost (Jung, see message 740 in figure 7(b) indicating network disconnection and therefore indicating that a connection for receiving the virtual content has been lost).
PNG
media_image8.png
694
546
media_image8.png
Greyscale
As to Claim 24:
Jung in view of Osman discloses that the head-mounted device of claim 22, wherein in response to the failure to receive the video frames of the virtual content, the one or more processors are further configured to: generate, at the head-mounted device, the virtual content comprising the one or more objects that are to be overlaid or composited with the objects included in the video frames from the one or more cameras to generate the virtual view of the environment of the head-mounted device (Osman, see claim mapping in figure 3D and 3E and paragraphs [0133] and [0134], where Osman discloses that similarly, at time T2, the at least one camera of the HMD detects the ball at a different angle/location as the ball continues its upward ascent to a distance of d2 and the game processing module translates the different angle/location of the ball to the corresponding location on the screen where to render the image of the ball. FIG. 3F illustrates the location 304b where the image of the ball is rendered (the location 304b may correlate with the height d2) while the remaining portions of the screen continue to render the interactive scene from the game. When the ball falls down, the movement of the ball in the downward direction is captured till the ball moves out of the capturing range of the cameras. Thus, FIG. 3D illustrates the scene rendered on the screen of the HMD at time T0 as well as at time T5, FIG. 3E illustrates the scene with the ball covering a portion 304a of the screen at time T1 (corresponding to ball ascending) as well as at time T4 (corresponding to ball descending) and FIG 3F illustrates the scene with the ball covering a portion 304b of the scene at time T2 (corresponding to ball ascending) as well as at time T3 (corresponding to ball ascending). In this embodiment, the HMD screen is considered to have transitioned from a non-transparent mode to a simulated transparent mode. In some embodiments, the image of the ball may be rendered in the foreground while the interactive scenes may be rendered in the background. Alternately, the image of the ball may be rendered in the background while the interactive scenes may be rendered in the foreground. In these embodiments, the HMD screen is considered to have transitioned from a non-transparent mode to a semi-transparent mode. The game processing module, thus, provides a user with a way to completely immerse in a game while providing a peek into scenes from the real-world based on triggering of external events or based on input from the user, such as gaze shifting, or other such visual cues explicitly or implicitly provided by the user).
PNG
media_image9.png
1094
950
media_image9.png
Greyscale
As to Claim 30:
Jung et al. discloses a method (Jung, see Abstract, where Jung discloses that a head mounted display (HMD) device including a communication processor configured to communicate with a mobile terminal; a display configured to display a content received from the mobile terminal; and a controller configured to in response to an event generated on the mobile terminal, selectively display an object on the display corresponding to the event generated on the mobile terminal, based on a type of the displayed content), comprising: receiving, at a head-mounted device, video frames from one or more cameras of the head mounted device (Jung, see paragraph [0049], where Jung discloses that as shown in FIG. 3A, the camera 221 may be provided at the body 200. For instance, the camera 221 may be disposed on one surface (e.g., a front surface) of the HMD body 200. The camera 221 is disposed near at least one of a left eye and a right eye, and is formed to capture (receive or input) a front image. Since the camera 221 is arranged near the eye to capture a front image, the camera 221 may acquire a scene at which a user stares, as an image); receiving, at the head-mounted device, video frames of virtual content; overlaying or compositing objects included in the video frames of the virtual content with objects included in the video frames (Jung, see paragraph [0058], where Jung discloses that the display unit 251 may project an image to a user's eyes using a prism. The prism may be formed of a transmissive material such that the user can see both the projected image and a general view at a front side (a viewing angle). An image output through the display unit 251 may be seen in an overlapped manner with a general view. The HMD 200 can provide an augmented reality (AR) which provides a single image by overlapping a real image or background with a virtual image using a characteristic of the display unit 251) from the one or more cameras (Jung, see paragraph [0049], where Jung discloses that the camera 221 is arranged near the eye to capture a front image, the camera 221 may acquire a scene at which a user stares, as an image) to generate a virtual view of an environment of the head-mounted device (Jung, see paragraph [0036], where Jung discloses a virtual space image displayed on a display unit of the HMD 200 can be an image stored in the HMD 200 or stored in the mobile terminal 100 which wirelessly communicates with the HMD 200. For instance, in case of a virtual space image stored in the HMD 200, the HMD 200 can execute an image processing and a rendering process to process the virtual space image, and may output video information generated after the image processing the rendering process to the display unit. Further, in case of a virtual space image stored in the mobile terminal 100, the mobile terminal 100 can execute an image processing and a rendering process, and may transmit video information generated after the image processing the rendering process to the HMD 200. Then, the HMD 200 can output the video information received from the mobile terminal 100); providing the virtual view for display to a user of the head-mounted device (Jung, see 901 in figure 9A); and in response to a failure to receive additional video frames of the virtual content (Jung, see paragraph [0120], where Jung discloses that if a stream downloading or a play of the video content 703 being played is stopped due to a network disconnection, the execution state of the video content 703 can be changed, and a notification icon 740" indicating occurrence of a corresponding event can be displayed).
Jung differs from the claimed subject matter in that Jung does not explicitly disclose continuing to provide a virtual view for display to the user, wherein the virtual view provided to the user during the failure comprises objects included in the video frames from the one or more cameras.
However in an analogous art, Osman discloses continuing to provide a virtual view for display to the user (Osman, see claim mapping in figure 3E, where Osman discloses a provided virtual view even while the video frame is blocked by a virtual content object (soccer ball)), wherein the virtual view provided to the user during the failure comprises objects included in the video frames from the one or more cameras Osman, see claim mapping in figure 3E and paragraphs [0133] and [0134], where Osman discloses that similarly, at time T2, the at least one camera of the HMD detects the ball at a different angle/location as the ball continues its upward ascent to a distance of d2 and the game processing module translates the different angle/location of the ball to the corresponding location on the screen where to render the image of the ball. FIG. 3F illustrates the location 304b where the image of the ball is rendered (the location 304b may correlate with the height d2) while the remaining portions of the screen continue to render the interactive scene from the game. When the ball falls down, the movement of the ball in the downward direction is captured till the ball moves out of the capturing range of the cameras. Thus, FIG. 3D illustrates the scene rendered on the screen of the HMD at time T0 as well as at time T5, FIG. 3E illustrates the scene with the ball covering a portion 304a of the screen at time T1 (corresponding to ball ascending) as well as at time T4 (corresponding to ball descending) and FIG 3F illustrates the scene with the ball covering a portion 304b of the scene at time T2 (corresponding to ball ascending) as well as at time T3 (corresponding to ball ascending). In this embodiment, the HMD screen is considered to have transitioned from a non-transparent mode to a simulated transparent mode. In some embodiments, the image of the ball may be rendered in the foreground while the interactive scenes may be rendered in the background. Alternately, the image of the ball may be rendered in the background while the interactive scenes may be rendered in the foreground. In these embodiments, the HMD screen is considered to have transitioned from a non-transparent mode to a semi-transparent mode. The game processing module, thus, provides a user with a way to completely immerse in a game while providing a peek into scenes from the real-world based on triggering of external events or based on input from the user, such as gaze shifting, or other such visual cues explicitly or implicitly provided by the user).
PNG
media_image7.png
1264
1008
media_image7.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the invention of Jung with Osman. One would be motivated to modify Jung by disclosing continuing to provide a virtual view for display to the user, wherein the virtual view provided to the user during the failure comprises objects included in the video frames from the one or more cameras as taught by Jeong, and thereby incorporating sophisticated operations and mathematics to produce a very realistic game experience (Osman, see paragraph [0003]).
.
As to Claim 31:
Jung in view of Osman discloses that the method of claim 30, wherein the virtual view provided to the user during the failure comprises: a message indicating that a connection for receiving the virtual content has been lost (Jung, see message 740 in figure 7(b) indicating network disconnection and therefore indicating that a connection for receiving the virtual content has been lost).
PNG
media_image8.png
694
546
media_image8.png
Greyscale
As to Claim 32:
Jung in view of Osman discloses that the method of claim 30, wherein said continuing to provide the virtual view during the failure comprises: generating, at the head-mounted device, the virtual content comprising the one or more objects that are to be overlaid or composited with the objects included in the video frames from the one or more cameras (Osman, see claim mapping in figure 3D and 3E and paragraphs [0133] and [0134], where Osman discloses that similarly, at time T2, the at least one camera of the HMD detects the ball at a different angle/location as the ball continues its upward ascent to a distance of d2 and the game processing module translates the different angle/location of the ball to the corresponding location on the screen where to render the image of the ball. FIG. 3F illustrates the location 304b where the image of the ball is rendered (the location 304b may correlate with the height d2) while the remaining portions of the screen continue to render the interactive scene from the game. When the ball falls down, the movement of the ball in the downward direction is captured till the ball moves out of the capturing range of the cameras. Thus, FIG. 3D illustrates the scene rendered on the screen of the HMD at time T0 as well as at time T5, FIG. 3E illustrates the scene with the ball covering a portion 304a of the screen at time T1 (corresponding to ball ascending) as well as at time T4 (corresponding to ball descending) and FIG 3F illustrates the scene with the ball covering a portion 304b of the scene at time T2 (corresponding to ball ascending) as well as at time T3 (corresponding to ball ascending). In this embodiment, the HMD screen is considered to have transitioned from a non-transparent mode to a simulated transparent mode. In some embodiments, the image of the ball may be rendered in the foreground while the interactive scenes may be rendered in the background. Alternately, the image of the ball may be rendered in the background while the interactive scenes may be rendered in the foreground. In these embodiments, the HMD screen is considered to have transitioned from a non-transparent mode to a semi-transparent mode. The game processing module, thus, provides a user with a way to completely immerse in a game while providing a peek into scenes from the real-world based on triggering of external events or based on input from the user, such as gaze shifting, or other such visual cues explicitly or implicitly provided by the user).
PNG
media_image9.png
1094
950
media_image9.png
Greyscale
.
As to Claim 37:
Jung et al. discloses that one or more non-transitory, computer-readable, storage media storing program instructions that, when executed using one or more processors, cause the one or more processors to (Jung, see controller 180 in figure 4A): receive video frames from one or more cameras (Jung, see paragraph [0049], where Jung discloses that as shown in FIG. 3A, the camera 221 may be provided at the body 200. For instance, the camera 221 may be disposed on one surface (e.g., a front surface) of the HMD body 200. The camera 221 is disposed near at least one of a left eye and a right eye, and is formed to capture (receive or input) a front image. Since the camera 221 is arranged near the eye to capture a front image, the camera 221 may acquire a scene at which a user stares, as an image); receive video frames of virtual content; and overlay or composite objects included in the video frames of the virtual content with objects included in the video frames (Jung, see paragraph [0058], where Jung discloses that the display unit 251 may project an image to a user's eyes using a prism. The prism may be formed of a transmissive material such that the user can see both the projected image and a general view at a front side (a viewing angle). An image output through the display unit 251 may be seen in an overlapped manner with a general view. The HMD 200 can provide an augmented reality (AR) which provides a single image by overlapping a real image or background with a virtual image using a characteristic of the display unit 251) from the one or more cameras (Jung, see paragraph [0049], where Jung discloses that the camera 221 is arranged near the eye to capture a front image, the camera 221 may acquire a scene at which a user stares, as an image) to generate a virtual view of an environment (Jung, see paragraph [0036], where Jung discloses a virtual space image displayed on a display unit of the HMD 200 can be an image stored in the HMD 200 or stored in the mobile terminal 100 which wirelessly communicates with the HMD 200. For instance, in case of a virtual space image stored in the HMD 200, the HMD 200 can execute an image processing and a rendering process to process the virtual space image, and may output video information generated after the image processing the rendering process to the display unit. Further, in case of a virtual space image stored in the mobile terminal 100, the mobile terminal 100 can execute an image processing and a rendering process, and may transmit video information generated after the image processing the rendering process to the HMD 200. Then, the HMD 200 can output the video information received from the mobile terminal 100); provide the virtual view for display to a user (Jung, see 901 in figure 9A); and in response to a failure to receive the video frames of the virtual content (Jung, see paragraph [0120], where Jung discloses that if a stream downloading or a play of the video content 703 being played is stopped due to a network disconnection, the execution state of the video content 703 can be changed, and a notification icon 740" indicating occurrence of a corresponding event can be displayed).
Jung differs from the claimed subject matter in that Jung does not explicitly disclose continue to provide the virtual view for display to the user, wherein the virtual view provided to the user during the failure to receive the video frames of the virtual content comprises objects included in the video frames from the one or more cameras.
However in an analogous art, Osman discloses continue to provide the virtual view for display to the user (Osman, see claim mapping in figure 3E, where Osman discloses a provided virtual view even while the video frame is blocked by a virtual content object (soccer ball)), wherein the virtual view provided to the user during the failure to receive the video frames of the virtual content comprises objects included in the video frames from the one or more cameras Osman, see claim mapping in figure 3E and paragraphs [0133] and [0134], where Osman discloses that similarly, at time T2, the at least one camera of the HMD detects the ball at a different angle/location as the ball continues its upward ascent to a distance of d2 and the game processing module translates the different angle/location of the ball to the corresponding location on the screen where to render the image of the ball. FIG. 3F illustrates the location 304b where the image of the ball is rendered (the location 304b may correlate with the height d2) while the remaining portions of the screen continue to render the interactive scene from the game. When the ball falls down, the movement of the ball in the downward direction is captured till the ball moves out of the capturing range of the cameras. Thus, FIG. 3D illustrates the scene rendered on the screen of the HMD at time T0 as well as at time T5, FIG. 3E illustrates the scene with the ball covering a portion 304a of the screen at time T1 (corresponding to ball ascending) as well as at time T4 (corresponding to ball descending) and FIG 3F illustrates the scene with the ball covering a portion 304b of the scene at time T2 (corresponding to ball ascending) as well as at time T3 (corresponding to ball ascending). In this embodiment, the HMD screen is considered to have transitioned from a non-transparent mode to a simulated transparent mode. In some embodiments, the image of the ball may be rendered in the foreground while the interactive scenes may be rendered in the background. Alternately, the image of the ball may be rendered in the background while the interactive scenes may be rendered in the foreground. In these embodiments, the HMD screen is considered to have transitioned from a non-transparent mode to a semi-transparent mode. The game processing module, thus, provides a user with a way to completely immerse in a game while providing a peek into scenes from the real-world based on triggering of external events or based on input from the user, such as gaze shifting, or other such visual cues explicitly or implicitly provided by the user).
PNG
media_image7.png
1264
1008
media_image7.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the invention of Jung with Osman. One would be motivated to modify Jung by disclosing continuing to provide a virtual view for display to the user, wherein the virtual view provided to the user during the failure comprises objects included in the video frames from the one or more cameras as taught by Jeong, and thereby incorporating sophisticated operations and mathematics to produce a very realistic game experience (Osman, see paragraph [0003]).
As to Claim 38:
Jung in view of Osman discloses that the one or more non-transitory, computer-readable storage media of claim 37, wherein the virtual view provided to the user during the failure comprises: a message indicating that a connection for receiving the virtual content has been lost (Jung, see message 740 in figure 7(b) indicating network disconnection and therefore indicating that a connection for receiving the virtual content has been lost).
PNG
media_image8.png
694
546
media_image8.png
Greyscale
As to Claim 39:
Jung in view of Osman discloses that the one or more non-transitory, computer-readable, storage media of claim 37, wherein in response to a failure to receive the video frames of the virtual content the program instructions, when executed using the one or more processors, further cause the one or more processors to: generate the virtual content comprising one or more objects that are to be overlaid or composited with the objects included in the video frames from the one or more cameras to generate the virtual view of an environment of the head-mounted device (Osman, see claim mapping in figure 3D and 3E and paragraphs [0133] and [0134], where Osman discloses that similarly, at time T2, the at least one camera of the HMD detects the ball at a different angle/location as the ball continues its upward ascent to a distance of d2 and the game processing module translates the different angle/location of the ball to the corresponding location on the screen where to render the image of the ball. FIG. 3F illustrates the location 304b where the image of the ball is rendered (the location 304b may correlate with the height d2) while the remaining portions of the screen continue to render the interactive scene from the game. When the ball falls down, the movement of the ball in the downward direction is captured till the ball moves out of the capturing range of the cameras. Thus, FIG. 3D illustrates the scene rendered on the screen of the HMD at time T0 as well as at time T5, FIG. 3E illustrates the scene with the ball covering a portion 304a of the screen at time T1 (corresponding to ball ascending) as well as at time T4 (corresponding to ball descending) and FIG 3F illustrates the scene with the ball covering a portion 304b of the scene at time T2 (corresponding to ball ascending) as well as at time T3 (corresponding to ball ascending). In this embodiment, the HMD screen is considered to have transitioned from a non-transparent mode to a simulated transparent mode. In some embodiments, the image of the ball may be rendered in the foreground while the interactive scenes may be rendered in the background. Alternately, the image of the ball may be rendered in the background while the interactive scenes may be rendered in the foreground. In these embodiments, the HMD screen is considered to have transitioned from a non-transparent mode to a semi-transparent mode. The game processing module, thus, provides a user with a way to completely immerse in a game while providing a peek into scenes from the real-world based on triggering of external events or based on input from the user, such as gaze shifting, or other such visual cues explicitly or implicitly provided by the user).
PNG
media_image9.png
1094
950
media_image9.png
Greyscale
Allowable Subject Matter
Claims 25, 26, 27, 28, 29, 33, 34, 35, 36, 40, 41 and 42 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Referring to claims 25, 33 and 40, the following is a statement of reasons for the indication of allowable subject matter: the prior art fail to suggest limitations “wherein the one or more processors are configured to: upon detection of a loss of reception of the video frames of the virtual content, route the video frames received from the one or more cameras to a direct-to-display processing pipeline, wherein the virtual view provided for display to the user during the failure comprises an output of the direct-to-display pipeline”.
Referring to claims 26, 34 and 41, the following is a statement of reasons for the indication of allowable subject matter: the prior art fail to suggest limitations “decode a given received video frame of the virtual content; store the given received video frame of the virtual content to a previous frame buffer; and in response to a failure to receive a next video frame of the virtual content, read the given video frame from the previous frame buffer and use the given video frame to synthesize virtual content for the next video frame”.
Referring to claims 27, 35 and 42, and dependent claims 28 and 36 the following is a statement of reasons for the indication of allowable subject matter: the prior art fail to suggest limitations “receive, for each of the respective video frames from the one or more cameras, a corresponding pre-determined motion vector; receive, for each of the respective video frames of the virtual content, a corresponding pre-determined motion vector; and use the pre-determined motion vectors to overlay or composite the objects included in the video frames of the virtual content with the objects included in the video frames from the one or more cameras to generate the virtual view of the environment of the head-mounted device”.
Referring to claim 29, the following is a statement of reasons for the indication of allowable subject matter: the prior art fail to suggest limitations “wherein the one or more processors are further configured to: receive information indicating a level of pupil dilation of the user of the head-mounted device; and modulate a brightness of the objects from the virtual content that are overlaid or composited with the objects from the one or more cameras based on the level of pupil dilation of the user”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to
applicant's disclosure. Lee (US 20200358981 A1) discloses an electronic device includes a camera; a display; at least one sensor; a communication unit configured to establish wireless communication with another electronic device using at least one protocol; and a processor configured to be functionally connected to the camera, the display, the at least one sensor, and the communication unit, wherein the processor is configured to perform a call with the other electronic device, detect a state change of the electronic device based on sensing information sensed by the at least one sensor .
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NELSON ROSARIO whose telephone number is (571)270-1866. The examiner can normally be reached on Monday through Friday, 7:30am- 5:00pm EST. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Eason can be reached on (571) 270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NELSON M ROSARIO/Primary Examiner, Art Unit 2624
Prosecution Insights