DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
1. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/22/2026 has been entered.
Claim Rejections - 35 USC § 103
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over (NPL: DHP19: Dynamic Vision Sensor 3D Human Pose Dataset, 2019, Enrico Calabres), and in view of West, US 20130181901 A1, and further in view of Koch, US 2015/0294492 A1.
5. As per claim 1, Calabres discloses: A system comprising:
a first set of display panels at least partially enclosing an area; (Calabres, Figure 3C, page 2, Sub-section 2.2, “displaying the frame-based dataset on a 60 Hz LCD monitor in front a DVS camera (DAVIS240 [7]).”
a set of sensors that is configured to capture sensor data of a subject positioned within the area; (Calabres, Figure 3e, page 4, left column, lines 1-7, “To map the camera space to 3D space, each DAVIS camera was individually calibrated using images acquired from the APS output. The position of 38 Vicon markers was acquired in 8 different position and the 2D marker positions were manually labelled on the APS frames. The camera projection matrix P and the camera position C were calculated for each camera. P maps 3D world coordinates to image coordinates. I”, and page 3, right column, lines 4-7, “The Vicon setup was composed by ten Bonita Motion Capture (BMC) infrared (IR) cameras surrounding a motorized treadmill where the subjects performed the different movements.”)
and
a controller coupled to the first set of display panels and to the set of sensors, (Calabres, right column, section: “Time synchronization. ”, “Camera1 (Fig. 3c) ) is the master for the other cameras and receives a trigger input from the Vicon controller at the start and end of recording.”,) the controller being configured to:
display media content to the subject in the area using the first set of display panels; (Calabres, Figure 3C, page 2, Sub-section 2.2, “displaying the frame-based dataset on a 60 Hz LCD monitor in front a DVS camera (DAVIS240 [7]).”
capture sensor data of the subject using the set of sensors while the media content is displayed on the first set of display panels; (Calabres, Page 3, right column, lines 4-7, ”The Vicon setup was composed by ten Bonita Motion Capture (BMC) infrared (IR) cameras surrounding a motorized treadmill where the subjects performed the different movements.”)
6. Calabres doesn’t expressly discloses:
insert the 3D representation of the subject into a target scene of a media item; and render the media item, which includes the 3D representation of the subject in the target scene.
7. West discloses:
insert the 3D representation of the subject into a target scene of a media item; and (West, [0008], “At least one, preferably both, of a ceiling-located screen and a bottom or floor located screen are provided to carry projected images (still and/or preferably moving pictures or video) always corresponding to or coordinated with the images projected onto the screens located to the front and the sides. Collectively, the viewer is provided with a fully immersive, multi-dimensional video experience.”)
render the media item, which includes the 3D representation of the subject in the target scene. (West, [0080], “Relatedly, the screens and the images projected thereon can be of 3-dimensional type and that, too, can be manipulated by the image creator to advantage, again, to enhance the immersive experience.”, and FIGS. 32 and 33)
8. West is analogous art with respect to Calabres because they are from the same field of endeavor, namely image processing. At the time the application was filed, it would have been obvious to a person of ordinary skill in the art to include the process of installing the processes of inserting the 3D representation of the subject into a target scene of a media item; and rendering the media item, which includes the 3D representation of the subject in the target scene, as taught by West into the teaching of Calabres. The suggestion for doing so would provide fully immersive, realistic yet virtual viewing. Therefore, it would have been obvious to combine West with Calabres.
9. Calabres in view of West doesn’t expressly disclose:
generate, based on the captured sensor data, a three-dimensional (3D) representation of the subject along with at least one texture image that provides texture for the 3D representation of the subject;
10. Koch discloses:
generate, based on the captured sensor data, a three-dimensional (3D) representation of the subject along with at least one texture image that provides texture for the 3D representation of the subject; (Koch, claim 1:” generating a 3-D representation of the subject in a virtual 3-D space, wherein a geometry and texture of the 3-D representation is generated based on the plurality of 2D video sequences.”)
11. Koch is analogous art with respect to Calabres in view of West because they are from the same field of endeavor, namely image processing. At the time the application was filed, it would have been obvious to a person of ordinary skill in the art to include the process of generating, based on the captured sensor data, a three-dimensional (3D) representation of the subject along with at least one texture image that provides texture for the 3D representation of the subject, as taught by Koch into the teaching of Calabres in view of West. The suggestion for doing so would depict the subject from a different perspective. Therefore, it would have been obvious to combine Koch with Calabres in view of West.
12. As per claim 2, Calabres in view of West, and in view of Koch discloses: Calabres in view of West discloses: The system of claim 1, wherein the media content comprises video data that is displayed on the first set of display panels. (Calabres, Figure 3C, page 2, Sub-section 2.2, “displaying the frame-based dataset on a 60 Hz LCD monitor in front a DVS camera (DAVIS240 [7])
13. As per claim 3, Calabres in view of West, and in view of Koch discloses: The system of claim 2, wherein the video data comprises video data captured using a 360-degree camera (Calabres, page 3, Right column , lines 15-17, “The DAVIS cameras were arranged to provide almost 360-degrees coverage of the scene around the subject.”) .
14. As per claim 4, Calabres in view of Wes, and in view of Koch t discloses: The system of claim 3, wherein the sensor data captured an acting performance of the subject using the set of sensors, ; (Calabres, Page 3, right column, lines 4-7, ”The Vicon setup was composed by ten Bonita Motion Capture (BMC) infrared (IR) cameras surrounding a motorized treadmill where the subjects performed the different movements.”) and wherein the 360-degree camera captures the target scene into which the subject’s acting performance is to be inserted. (Calabres, page 3, Right column, lines 15-17, “The DAVIS cameras were arranged to provide almost 360-degrees coverage of the scene around the subject.”, and lines 31-35, “The output of Vicon cameras was recorded and processed using Vicon proprietary software (Nexus 2.6), that we used to visualize the markers, generate the skeleton structure and label the joints, as shown in Fig. 3d).”) .
15. As per claim 5, Calabres in view of West, and in view of Koch discloses: The system of claim 4, wherein the target scene is captured by the 360-degree camera from the perspective of the subject. (Calabres, page 3, Right column, lines 15-17, “The DAVIS cameras were arranged to provide almost 360-degrees coverage of the scene around the subject.”) .
16. As per claim 6, Calabres in view of West, and in view of Koch discloses: The system of claim 1, wherein the target scene is generated using a 3D model of the target scene. (Calabres, Page 3, Right column, lines 3-7, “. Recordings were made in a therapy environment in are cording volume of 2x2x2m3. The Vicon setup was composed by ten Bonita Motion Capture (BMC) infrared (IR) cameras surrounding a motorized treadmill where the subjects performed the different movements.”)
17. As per claim 7, Calabres in view of West, and in view of Koch discloses: The system of claim 6, wherein the 3D representation of the subject is placed at a desired location within the 3D model. (Calabres, Section, 4. DVS 3D human pose estimation.)
18. As per claim 8, Calabres in view of West, and in view of Koch discloses: The system of claim 1, wherein the rendered media item is generated based on both the at least one texture image and the 3D representation of the subject. (Calabres, Page 5, right column, lines 4-11, “In the proposed method we decompose the 3D pose estimation problem in 2D pose estimation based on CNN, and 2D-to-3D reconstruction using geometric information about the position of the cameras. First, a single CNN is trained on the two camera views. Then, we project each of the 2D predictions from the pixel space to the physical space through triangulation, knowing the projection matrices P and the camera positions C.”)
19. As per claim 9, Calabres in view of West, and in view of Koch discloses: The system of claim 1, wherein video frames of the subject in the target scene correspond to times at which sensor data was captured and rendered, which correspondence allows the rendered media item to be rendered at specified frame rates. (Calabres, page 8, Section: 6. Discussion, “Presence of movement and its speed. The DVS microsecond time resolution provides a continuous temporal information not limited by a fixed frame rate, which can be advantageous for HPE by alleviating the motion blur present for fast movements (e.g. in MADS dataset [32]).
20. As per claim 10, Calabres in view of West, and in view of Koch discloses: The system of claim 1, further comprising an omnidirectional treadmill in the area, wherein the subject provides an acting performance for the target scene while positioned on the omnidirectional treadmill. (Calabres, Page 4, right column, lines 4-7, ”The Vicon setup was composed by ten Bonita Motion Capture (BMC) infrared (IR) cameras surrounding a motorized treadmill where the subjects performed the different movements.”)
21. As per claim 11, Calabres in view of West, and in view of Koch discloses: The system of claim 1, further comprising:
a second set of display panels positioned above the area; (West, figure 1, [0017], “In addition, and possibly quite importantly, one or more screens are placed both on the ceiling above the head(s) of the viewer(s) and on the floor under or adjacent yet forwardly of their feet so as to provide a comprehensive and substantially complete experience.”) and
a third set of display panels positioned opposite to the second set of display panels and coupled to the controller, the third set of display panels being configured to receive additional media content comprising a multidimensional scene from the controller and being configured to display the additional media content. (West, [0021], “So, for example, after takeoff, the bottom or floor screen would display the receding and quickly "shrinking" of the early below. The images will show the earth moving rapidly away, seemingly downwardly as the rocket moves relative upward (at increasing acceleration) from the launch pad. The below or floor located screen would show the floor of the capsule and/or the upward rapid movement of the rocket and capsule as it leaves the ground and the ceiling-located screen could show the images corresponding to that which a camera housed in the nose cone of the rocket would "see" as rocket and capsule are launched into orbit, with sky and stars (maybe moon) rapidly approaching from above.”)
22. As per claim 12, Calabres in view of West , and in view of Koch discloses: The system of claim 11, wherein the first set of display panels, the second set of display panels, and the third set of display panels are simultaneously displaying media content related to the target scene. (West, Figure 13, and [0081], “ FIG. 13 is an enlarged side elevational view of FIG. 11b depicting an alternate embodiment of the projector set-up for the viewing screens. This embodiment depicts multiple projectors 42, 44, 46, and 48 used for side screens 6, and multiple projectors 92 and 94 for ceiling screen 10. “)
23. As per claim 15, Calabres in view of West, and in view of Koch discloses: The computer-implemented method of claim 14, wherein the media content displayed by the first set of display panels is modified based on information from the omnidirectional treadmill describing movement of the subject. (Calabres, Page 4, Sub- section: 2.3. Human pose datasets.” All of these datasets include multi-view camera recordings of the whole body of subjects performing different movements, and include ground truth 3D pose recording from a motion capture system.”, and Page 4, Right column, lines 4-7:’The Vicon setup was composed by ten Bonita Motion Capture (BMC) infrared (IR) cameras surrounding a motorized treadmill where the subjects performed the different movements.”)
24. Claims 13, 16, and 20, which are similar in scope to claim 1, thus rejected under the same rationale.
25. Claims 14, 17, 18, which are similar in scope respectively to claims 10, 3, and 4, are thus rejected under the same rationale.
26. Claim 19, which is similar in scope to claims 6, and 7, thus rejected under the same rationale.
Conclusion
27. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDERRAHIM MEROUAN whose telephone number is (571)270-5254. The examiner can normally be reached 9 AM -- 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDERRAHIM MEROUAN/Supervisory Patent Examiner, Art Unit 2683
Prosecution Insights