DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 9, and 16 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 8, and 14 of U.S. Patent No. 12,217,356, respectively. Although the claims at issue are not identical, they are not patentably distinct from each other because they are generic to all that is recited in US Patent 12,217,356. That is, the above claims are anticipated by claims of US Patent 12,217,356, and are therefore an obvious variant thereof. This will be described in reference to the table below; emphasis has been added in bold to the corresponding elements.
Instant Application 19/000495
US Patent 12,217,356
Claim 1:A system, comprising:
a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, the computing device configured to:
generate a real-time, rendered digital scene for presentation on the display;
wherein the computing device generates the digital scene based on a scene captured by the camera such that one or more objects shown in the digital scene each have suitable perspective for the location of the moving camera as received from the one or more trackers.
Claim 1:
A system, comprising:
a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, the computing device configured to:
receive the location of the moving camera from the one or more trackers, and
generate a real-time, rendered digital scene for presentation on the display;
wherein the computing device generates the digital scene such that one or more objects shown in the digital scene each have suitable perspective for the location of the moving camera as received from the one or more trackers; and
wherein the moving camera captures the digital scene.
Claim 9:
A method performed by a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, the method comprising:
generating a real-time, rendered digital scene for presentation on the display;
wherein one or more objects shown in the digital scene are based on a scene captured by the camera and have suitable perspective for the location of the moving camera as received from the one or more trackers.
Claim 8: A method performed by a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, the method comprising:
receiving, from the one or more trackers, the location of the moving camera; and
generating a real-time, rendered digital scene for presentation on the display;
wherein one or more objects shown in the digital scene have suitable perspective for the location of the moving camera as received from the one or more trackers; and
capturing the digital scene using the moving camera.
Claim 16:A non-transitory machine-readable medium storing instructions which, when executed by a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, will cause the computing device to perform actions comprising:
generating a real-time, rendered digital scene for presentation on the display;
wherein the computing device generates the digital scene based on a scene captured by the camera such that one or more objects shown in the digital scene have suitable perspective for the location of the moving camera as received from the one or more trackers.
Claim 14:
A non-transitory machine-readable medium storing instructions which, when executed by a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display, will cause the computing device to perform actions comprising:
receiving the location of the moving camera from the one or more trackers; and
generating a real-time, rendered digital scene for presentation on the display;
wherein the computing device generates the digital scene such that one or more objects shown in the digital scene have suitable perspective for the location of the moving camera as received from the one or more trackers; and
capturing the digital scene using the moving camera.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-22 are rejected under 35 U.S.C. 103 as being unpatentable over Sanders et al. (US Pub. 2015/0350628), hereinafter Sanders, in view of Smits (US Pub. 2009/0096994).
Regarding claim 1, Sanders, in view of Smits teaches the Sanders discloses a system comprising: a computing device in communication with both a display and one or more trackers that determine a location of a moving camera relative to the display (Fig. 2; Paragraph [0033]: immersion computer system 202 may also include a processing system 206 comprising one or more processing cores and one or more memory devices. The processing system 206 can coordinate communications between the various subsystems. A display driver 210 and a sound driver 218 can be coupled to a display output 212 and a sound output 214, respectively. The display driver 210 takes the 2-D images received from the interactive content system and formats the images to be displayed on one or more display devices 228 for the user; Paragraphs [0035]-[0036]: immersive environment 230 may also include one or more tracking devices 224 that are used to track the location of the user within the immersive environment 230. Various methods can be used to track the location of the user within the real-world coordinates of the immersive environment 230. These real-world coordinates can then be translated into a position in the 3-D virtual scene that is being processed by the interactive content system. A virtual camera in the 3-D virtual scene can be positioned at the location of the user, and in some cases, the virtual camera can be oriented in a direction of the sight line of the user. Real-time rendering of the virtual 3-D scene from the perspective of the user will be discussed in greater detail below…In some embodiments, the tracking devices 224 may include a plurality of cameras configured together to capture most or all of the immersive environment 230. The plurality of cameras can track fiducial markers on the user, such as Scotchlight balls, QR-like digital codes, or other visually distinguishable graphics. Some embodiments may perform a full motion capture of the user, such that the movements of the user can be used to animate a digital character;Paragraph [0047]: as the user takes a few steps forward, and object displayed by the display devices 228 in front of the user will grow bigger. The size and other details of the projected images can be calculated by ascertaining the position of the user in the immersive environment, and rendering the 3-D virtual scene in real-time from the perspective of the user in the immersive environment. The picture plane of the rendered scene can be oriented and positioned in the 3-D virtual scene relative to the camera position so as to be congruent with the position and orientation of the display devices 228 and the position and orientation of the user in the immersive environment), the computing device configured to: generate a real-time, rendered digital scene for presentation on the display (Paragraphs [0045]-[0048]: size, shape, and/or location of the chroma-key background 304 can be changed by the computer system based on the location of the camera 308 and the location of the subject 302. If the location of the camera 308 and the location of the subject 302 are known, then the computer system can calculate a projection of the subject 302 onto the display device 306. The computer system can then use the projection of the subject 302 to generate an appropriately sized chroma-key background 304… as the camera 308, the subject 302, and/or an object 310 in the 3-D virtual scene move relative to each other, the chroma-key background 304 may tend to interfere with the interaction between the subject 302 and the object 310 in the virtual 3-D scene. However, by minimizing the size of the chroma-key background 304 and moving it relative to the position of the cameras 308 and/or subject 302, the interference with the interaction between the subject 302 and the object 310 can be minimized. As described above, the computer system can determine how to move and resize the chroma-key background based on the location of the subject 302. In some cases, the volumetric representation of subject 302 (e.g., an actor) can simply be inserted into the virtual 3-D scene in the computer system and rendered in the chroma-key color as part of the scene projected onto the display device 306. As the subject 302 moves, the computer system determines, at interactive rates, which portions of an environment of the virtual 3-D scene are to be replaced by the chroma-key background 304 and which portions are to be revealed from previous replacement at any given moment in time. In other cases, the chroma-key background can be inserted after the entirety of the 3-D scene is rendered onto the 2-D image that is projected on the display device 306 as a 2-D silhouette. In either case, the size of the subject may be recorded a priori or measured as images of the subject 302 are captured. The size of the chroma-key background can be increased or decreased in real-time based on the size of the subject 302, the location of the physical camera, and the location of the subject 302 relative to the display device 306. By making the chroma-key background conform as closely to the shape of the projection of the subject 302 on the display device 306 as possible, the amount of the visual representation of the virtual 3-D scene that is obscured by the chroma-key background can be minimized. In some embodiments, certain scene elements, such as characters or objects with which the subject 302 may interact, may be specially designated in the 3-D virtual scene. When generating the chroma-key background, the computer can ensure that no portion of these designated elements are covered by the chroma-key background. While this may make extracting the images of the subject 302 from the projected scene more difficult (because the chroma-key background is not behind the subject 302), it may also prevent the chroma-key background from interfering between the subject 302 and CGI characters in the scene; Paragraph [0065]: 3-D virtual scene 904 may be comprised of multiple characters, backgrounds, textures, lighting effects, animations, and/or the like. The real-time rendering engine 906 can receive information from the subject recording system 912 including the location/orientation of the subject and the location/orientation of the camera 914. The real-time rendering engine 906 can place the virtual camera at the location of the subject with the same orientation and generate a real-time render of the 3-D virtual scene 900. This real-time stream of images may be displayed on the display devices 910 of the virtual-reality environment 908. Based on the location of the subject and the location/orientation of the camera, the real-time rendering engine 906 can insert a dynamic chroma-key background in the visual representation of the 3-D virtual scene for display on the display devices 910).
Sanders does not explicitly disclose wherein the computing device generates the digital scene based on a scene captured by the camera such that one or more objects shown in the digital scene each have suitable perspective for the location of the moving camera as received from the one or more trackers.
However, Smits teaches immersive display wherein user perspective can be dynamically and in real-time updated and projected (Paragraphs [0121]-[0123]), further comprising wherein the computing device generates the digital scene based on a scene captured by the camera such that one or more objects shown in the digital scene each have suitable perspective for the location of the moving camera as received from the one or more trackers (Fig. 6B; Paragraph [0122]: As described for FIG. 6A, this capability is useful in immersive applications, for example, video games, where user perspective can be dynamically and in real-time updated and projected; Paragraph [0123]: FIG. 6B shows an embodiment of a IPD response to different viewing perspectives. In this embodiment an illustrative image of a car is projected by the IP) onto a screen. When user 604 is at position-1, with a line of sight 624, which is substantially perpendicular to the screen, that is, substantially parallel with a center-line of projection from the IPD to the screen, a front-view image 620 of the car is automatically projected by the IPD. The position of user 604, in this case, position-1, relative to the center-line of projection is determined by information communicated via position sensor 602 (see FIG. 6A). When user 604 changes his position to position-2, with a line of sight 626, which is at an angle with respect to the center-line of projection, that is, not parallel with the center-line, the IPD detects the new position via information provided by position sensor 602. Subsequently, the IPD adjusts the image of the car to project the correct perspective image 622 of the car on the screen, as if user 604 is looking at a physical car in real world in 3-D; Paragraph [0125]: the IPD renders a scene by continuous real-time adjustments of the projected image by referencing the image as viewed through a position camera aligned with the viewer's perspective, as described above with respect to FIGS. 6A and 6B. In one embodiment, the position camera may be focused on a field of view within the screen, representing a focus area of the viewer/user). Smits teaches that this will allow for adjusting the image of an object to project the correct perspective of the object on the screen, as if user is looking at the object in real world in 3-D and that this enables projection of images in an immersive environment (Paragraph [0123]). Therefore, it would have been obvious to one of ordinary skill in the art to have modified Sanders with the features above as taught by Smits.
Regarding claim 2, Sanders, in view of Smits teaches the system of claim 1, Sanders discloses further comprising: the display (Fig. 2; Paragraph [0033]: immersion computer system 202 may also include a processing system 206 comprising one or more processing cores and one or more memory devices. The processing system 206 can coordinate communications between the various subsystems. A display driver 210 and a sound driver 218 can be coupled to a display output 212 and a sound output 214, respectively. The display driver 210 takes the 2-D images received from the interactive content system and formats the images to be displayed on one or more display devices 228 for the user); the one or more trackers (Fig. 9; Paragraph [0064]: subject recording system 912 may comprise a camera 914 and a subject tracking device 916. The camera can be used to capture images and flash for video sequences of the subject in the virtual-reality environment 908. The camera 914 may also include position controls and sensors that detect and/or control the location, orientation, and focal characteristics of the camera 914. The subject tracking device 916 can be used to track the location of the subject and/or the orientation of the head of the subject (i.e., where the subject is looking) The subject tracking device 916 can include GPS, radar, depth sensors, infrared emissions and detections, visual tracking systems, RFIDs, and/or the like); and the moving camera (Fig. 9; Paragraphs [0035]-[0036]: immersive environment 230 may also include one or more tracking devices 224 that are used to track the location of the user within the immersive environment 230. Various methods can be used to track the location of the user within the real-world coordinates of the immersive environment 230. These real-world coordinates can then be translated into a position in the 3-D virtual scene that is being processed by the interactive content system. A virtual camera in the 3-D virtual scene can be positioned at the location of the user, and in some cases, the virtual camera can be oriented in a direction of the sight line of the user. Real-time rendering of the virtual 3-D scene from the perspective of the user will be discussed in greater detail below…In some embodiments, the tracking devices 224 may include a plurality of cameras configured together to capture most or all of the immersive environment 230. The plurality of cameras can track fiducial markers on the user, such as Scotchlight balls, QR-like digital codes, or other visually distinguishable graphics. Some embodiments may perform a full motion capture of the user, such that the movements of the user can be used to animate a digital character; Paragraph [0047]: as the user takes a few steps forward, and object displayed by the display devices 228 in front of the user will grow bigger. The size and other details of the projected images can be calculated by ascertaining the position of the user in the immersive environment, and rendering the 3-D virtual scene in real-time from the perspective of the user in the immersive environment. The picture plane of the rendered scene can be oriented and positioned in the 3-D virtual scene relative to the camera position so as to be congruent with the position and orientation of the display devices 228 and the position and orientation of the user in the immersive environment).
Regarding claim 3, Sanders, in view of Smits teaches the system of claim 1, wherein the moving camera moves relative to the display (Sanders: Paragraphs [0035]-[0036]: immersive environment 230 may also include one or more tracking devices 224 that are used to track the location of the user within the immersive environment 230. Various methods can be used to track the location of the user within the real-world coordinates of the immersive environment 230. These real-world coordinates can then be translated into a position in the 3-D virtual scene that is being processed by the interactive content system. A virtual camera in the 3-D virtual scene can be positioned at the location of the user, and in some cases, the virtual camera can be oriented in a direction of the sight line of the user. Real-time rendering of the virtual 3-D scene from the perspective of the user will be discussed in greater detail below…In some embodiments, the tracking devices 224 may include a plurality of cameras configured together to capture most or all of the immersive environment 230. The plurality of cameras can track fiducial markers on the user, such as Scotchlight balls, QR-like digital codes, or other visually distinguishable graphics. Some embodiments may perform a full motion capture of the user, such that the movements of the user can be used to animate a digital character;Paragraph [0047]: as the user takes a few steps forward, and object displayed by the display devices 228 in front of the user will grow bigger. The size and other details of the projected images can be calculated by ascertaining the position of the user in the immersive environment, and rendering the 3-D virtual scene in real-time from the perspective of the user in the immersive environment. The picture plane of the rendered scene can be oriented and positioned in the 3-D virtual scene relative to the camera position so as to be congruent with the position and orientation of the display devices 228 and the position and orientation of the user in the immersive environment), the one or more trackers track the location of the moving camera when the moving camera moves (Sanders: Paragraph [0039]: tracking technologies can constantly monitor the position of the subject 202 within the virtual-reality environment 200. This location can be provided in real-time to a rendering engine in a computer system to update the position of the virtual camera in the 3-D virtual scene. The computer system can perform a real-time render on the virtual 3-D scene from the updated camera position and provide a streaming video sequence of images to be displayed on the display devices 206), and the computing device generates the digital scene such that the one or more objects shown in the digital scene each maintain suitable perspective for the location of the moving camera when the camera moves (Smits: Fig. 6B; Paragraph [0122]: As described for FIG. 6A, this capability is useful in immersive applications, for example, video games, where user perspective can be dynamically and in real-time updated and projected; Paragraph [0123]: FIG. 6B shows an embodiment of a IPD response to different viewing perspectives. In this embodiment an illustrative image of a car is projected by the IP) onto a screen. When user 604 is at position-1, with a line of sight 624, which is substantially perpendicular to the screen, that is, substantially parallel with a center-line of projection from the IPD to the screen, a front-view image 620 of the car is automatically projected by the IPD. The position of user 604, in this case, position-1, relative to the center-line of projection is determined by information communicated via position sensor 602 (see FIG. 6A). When user 604 changes his position to position-2, with a line of sight 626, which is at an angle with respect to the center-line of projection, that is, not parallel with the center-line, the IPD detects the new position via information provided by position sensor 602. Subsequently, the IPD adjusts the image of the car to project the correct perspective image 622 of the car on the screen, as if user 604 is looking at a physical car in real world in 3-D; Paragraph [0125]: the IPD renders a scene by continuous real-time adjustments of the projected image by referencing the image as viewed through a position camera aligned with the viewer's perspective, as described above with respect to FIGS. 6A and 6B. In one embodiment, the position camera may be focused on a field of view within the screen, representing a focus area of the viewer/user).
Regarding claim 4, Sanders, in view of Smits teaches the system of claim 1, Sanders discloses wherein the moving camera captures a composite image including a physical object disposed between the camera and the display, the one or more trackers determine the location of the physical object with respect to the moving camera and the display (Paragraph [0047]: as the user takes a few steps forward, and object displayed by the display devices 228 in front of the user will grow bigger. The size and other details of the projected images can be calculated by ascertaining the position of the user in the immersive environment, and rendering the 3-D virtual scene in real-time from the perspective of the user in the immersive environment. The picture plane of the rendered scene can be oriented and positioned in the 3-D virtual scene relative to the camera position so as to be congruent with the position and orientation of the display devices 228 and the position and orientation of the user in the immersive environment; Paragraph [0052]: the 3-D virtual scene can be rendered by the interactive content system from the perspective of the location of the user in the immersive environment. For example, the location of the user can be tracked and/or the direction of the user's gaze can be determined by sensors, gaze detection, tracking devices, and/or the like. The location of the user's head and/or the direction of the user's gaze can be used to position and orient a virtual camera in the 3-D virtual scene. Two-dimensional images can then be rendered from the perspective of the virtual camera and displayed on the display devices; Paragraph [0057]: the user-controlled character may interact with other users who are within the immersive environment surrounded by the display devices 228. In one aspect, the system may process the movements to determine whether the hologram of the character has interacted with other assets. Based on such a determination, the system may cause modifications to the other assets and/or character. For example, the user may cause the hologram of the character to swing a bat at a hologram of a baseball. Responsive to the swing, the system may alter the trajectory of the baseball hologram that is displayed. In one aspect, a user controlling a character may be outside of the immersive environment surrounded by the display devices 228 in a second different area. One or more sensors of the system may be directed at the second different area in order to capture the movement of the user), and the computing device generates the digital scene such that at least one of the one or more objects shown in the digital scene is positioned within the scene based on the location of the physical object as determined by the one or more trackers (Paragraph [0020]: captured performance of the subject may thereafter be inserted into or otherwise included in a rendered 3-D virtual scene. Such a 3-D virtual scene may be included as part of viewable content (e.g., a movie, television programming, video game, online video content) that may be presented to one or more viewers. For instance, an actor may perform within a virtual-reality environment that simulates a scene from a movie. His or her performance may be captured by one or more cameras directed or trained on the actor. The captured performance of the actor may then be composited with one or more digital objects and scenery to create a 3-D virtual scene for inclusion in a final production version of the movie to be shown to viewing audiences; Paragraph [0028]: embodiments described herein not only elicit a more emotional and life-like response from an actor, but these embodiments also allow actors to see and interact with real-time graphics (e.g., graphics displayed at an interactive frame-rate) and be illuminated correctly to match the illumination of the post-production render. These embodiments allow for a minimized matte-edge detail so that only a region directly behind the actor is illuminated with a chroma-key color for real-time and/or extraction and compositing. Illuminating the acting space with controlled lighting, LEDs, projectors, and/or any other light sources programmed with computer imagery can provide an immersive experience. By tracking the position of the actor and the position the camera and mapping these positions to the 3-D virtual scene, the proper inferred perspective of the actor can be used to generate projected images that provide proper graphic illumination. In short, the actor is able to realistically interact other characters and objects in the 3-D virtual scene, and the projected environment provides illumination characteristics that are consistent with the 3-D virtual scene; Paragraphs []0032]-[0033]: the term “chroma-key background” can refer to the any background used for compositing two images or video streams together based on color hue differences. For example, a green screen or blue screen may be commonly used as a chroma-key background…FIG. 1A illustrates a subject in a virtual-reality environment, according to some embodiments. The virtual-reality environment 101 may be comprised of a real-world space that provides some form of interaction with a 3-D virtual scene displayed by the virtual-reality environment 101. In this embodiment, the virtual-reality environment 101 is comprised of one or more display devices 102. These display devices 102 may include screens onto which images of elements of the 3-D virtual scene may be projected. These display devices 102 may also include active displays, such as LED panels, LCD panels, or other active display technologies. In some embodiments (not shown) the virtual-reality environment 101 may additionally or alternatively include physical devices that are used to present or suggest elements of the 3-D virtual scene. For example, robotic elements may be included that are controlled by actions or motions defined in the 3-D virtual scene. In another example, lasers can project intersecting light beams that creates images or holograms within the virtual-reality environment 101 to suggest locations or movements of elements of the 3-D virtual scene).
Regarding claim 5, Sanders, in view of Smits teaches the system of claim 4, Sanders discloses wherein the physical object is movable with respect to the display, the one or more trackers track the location of the physical object when the physical object moves (Paragraph [0039]: As the subject 202 moves within the virtual-reality environment 200, the display devices 206 can display updated images rendered in real-time from the 3-D virtual scene. As described above, tracking technologies can constantly monitor the position of the subject 202 within the virtual-reality environment 200. This location can be provided in real-time to a rendering engine in a computer system to update the position of the virtual camera in the 3-D virtual scene. The computer system can perform a real-time render on the virtual 3-D scene from the updated camera position and provide a streaming video sequence of images to be displayed on the display devices 206), and the computing device generates the digital scene such that the at least one of the one or more objects shown in the digital scene is repositioned within the scene based on the location of the physical object as the physical object moves (Paragraph [0024]: the chroma-key background may replace a portion of the 3-D virtual scene directly behind the subject from the perspective of the physical environment. For example an actor can stand in the virtual-reality environment in a movie studio while images of robots and aliens are projected on screens that surround the actor, such that the actor feels as though they are in an alien environment actually interacting with CGI characters and/or objects. While shooting the movie scene, the actor can talk to the robotic/alien characters and freely move around the scene. At the same time, the computer system can track the actor's movements and dynamically resize the alien environment and characters as the actor moves around the scene. The computer system can also simultaneously project a green screen screen image on a portion of the projection screens so that the camera can capture the actor's performance against the green screen instead of having the recorded images of the actor blend into the alien background).
Regarding claim 6, Sanders, in view of Smits teaches the system of claim 5, Sanders discloses wherein the physical object is an actor (Paragraph [0024]: the chroma-key background may replace a portion of the 3-D virtual scene directly behind the subject from the perspective of the physical environment. For example an actor can stand in the virtual-reality environment in a movie studio while images of robots and aliens are projected on screens that surround the actor, such that the actor feels as though they are in an alien environment actually interacting with CGI characters and/or objects. While shooting the movie scene, the actor can talk to the robotic/alien characters and freely move around the scene. At the same time, the computer system can track the actor's movements and dynamically resize the alien environment and characters as the actor moves around the scene. The computer system can also simultaneously project a green screen screen image on a portion of the projection screens so that the camera can capture the actor's performance against the green screen instead of having the recorded images of the actor blend into the alien background).
Regarding claim 7, Sanders, in view of Smits teaches the system of claim 1, Sanders discloses wherein the one or more trackers comprises the moving camera (Paragraphs [0035]-[0036]: immersive environment 230 may also include one or more tracking devices 224 that are used to track the location of the user within the immersive environment 230. Various methods can be used to track the location of the user within the real-world coordinates of the immersive environment 230. These real-world coordinates can then be translated into a position in the 3-D virtual scene that is being processed by the interactive content system. A virtual camera in the 3-D virtual scene can be positioned at the location of the user, and in some cases, the virtual camera can be oriented in a direction of the sight line of the user. Real-time rendering of the virtual 3-D scene from the perspective of the user will be discussed in greater detail below…In some embodiments, the tracking devices 224 may include a plurality of cameras configured together to capture most or all of the immersive environment 230. The plurality of cameras can track fiducial markers on the user, such as Scotchlight balls, QR-like digital codes, or other visually distinguishable graphics. Some embodiments may perform a full motion capture of the user, such that the movements of the user can be used to animate a digital character).
Regarding claim 8, Sanders, in view of Smits teaches the system of claim 1, Sanders discloses wherein the moving camera captures a composite image including a live action actor disposed between the camera and the display, the one or more objects shown in the digital scene include an animated actor (Paragraph [0047]: as the user takes a few steps forward, and object displayed by the display devices 228 in front of the user will grow bigger. The size and other details of the projected images can be calculated by ascertaining the position of the user in the immersive environment, and rendering the 3-D virtual scene in real-time from the perspective of the user in the immersive environment. The picture plane of the rendered scene can be oriented and positioned in the 3-D virtual scene relative to the camera position so as to be congruent with the position and orientation of the display devices 228 and the position and orientation of the user in the immersive environment; Paragraph [0052]: the 3-D virtual scene can be rendered by the interactive content system from the perspective of the location of the user in the immersive environment. For example, the location of the user can be tracked and/or the direction of the user's gaze can be determined by sensors, gaze detection, tracking devices, and/or the like. The location of the user's head and/or the direction of the user's gaze can be used to position and orient a virtual camera in the 3-D virtual scene. Two-dimensional images can then be rendered from the perspective of the virtual camera and displayed on the display devices; Paragraph [0057]: the user-controlled character may interact with other users who are within the immersive environment surrounded by the display devices 228. In one aspect, the system may process the movements to determine whether the hologram of the character has interacted with other assets. Based on such a determination, the system may cause modifications to the other assets and/or character. For example, the user may cause the hologram of the character to swing a bat at a hologram of a baseball. Responsive to the swing, the system may alter the trajectory of the baseball hologram that is displayed. In one aspect, a user controlling a character may be outside of the immersive environment surrounded by the display devices 228 in a second different area. One or more sensors of the system may be directed at the second different area in order to capture the movement of the user), and the display comprises light emitting diodes (LEDs) to present the digital scene (Paragraph [0028]: embodiments described herein not only elicit a more emotional and life-like response from an actor, but these embodiments also allow actors to see and interact with real-time graphics (e.g., graphics displayed at an interactive frame-rate) and be illuminated correctly to match the illumination of the post-production render. These embodiments allow for a minimized matte-edge detail so that only a region directly behind the actor is illuminated with a chroma-key color for real-time and/or extraction and compositing. Illuminating the acting space with controlled lighting, LEDs, projectors, and/or any other light sources programmed with computer imagery can provide an immersive experience. By tracking the position of the actor and the position the camera and mapping these positions to the 3-D virtual scene, the proper inferred perspective of the actor can be used to generate projected images that provide proper graphic illumination. In short, the actor is able to realistically interact other characters and objects in the 3-D virtual scene, and the projected environment provides illumination characteristics that are consistent with the 3-D virtual scene).
Regarding claim 9, the limitations of this claim substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Regarding claim 10, the limitations of this claim substantially correspond to the limitations of claim 3; thus they are rejected on similar grounds.
Regarding claim 11, the limitations of this claim substantially correspond to the limitations of claim 4; thus they are rejected on similar grounds.
Regarding claim 12, the limitations of this claim substantially correspond to the limitations of claim 5; thus they are rejected on similar grounds.
Regarding claim 13, the limitations of this claim substantially correspond to the limitations of claim 6; thus they are rejected on similar grounds.
Regarding claim 14, the limitations of this claim substantially correspond to the limitations of claim 7; thus they are rejected on similar grounds.
Regarding claim 15, the limitations of this claim substantially correspond to the limitations of claim 8; thus they are rejected on similar grounds.
Regarding claim 16, the limitations of this claim substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Regarding claim 17, the limitations of this claim substantially correspond to the limitations of claim 3; thus they are rejected on similar grounds.
Regarding claim 18, the limitations of this claim substantially correspond to the limitations of claim 4; thus they are rejected on similar grounds.
Regarding claim 19, the limitations of this claim substantially correspond to the limitations of claim 5; thus they are rejected on similar grounds.
Regarding claim 20, the limitations of this claim substantially correspond to the limitations of claim 6; thus they are rejected on similar grounds.
Regarding claim 21, the limitations of this claim substantially correspond to the limitations of claim 7; thus they are rejected on similar grounds.
Regarding claim 22, the limitations of this claim substantially correspond to the limitations of claim 8; thus they are rejected on similar grounds.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lovemelt et al. (US Pub. 2019/0098230) teaches a display screen providing a real-time, third-person perspective display of aspects of the interior of the environment, which may include a virtual background composited with video.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW D SALVUCCI whose telephone number is (571)270-5748. The examiner can normally be reached M-F: 7:30-4:00PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW SALVUCCI/Primary Examiner, Art Unit 2613
Prosecution Insights