DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1, 5-7 and 8-12 objected to because of the following informalities:
Claim 7, line 6-7 recites “MPEG lighting extension at a glTF file level”. Every abbreviation and acronym must be fully defined the first time they are referred to before using them. The terms “MPEG” and “glTF” although common in the art, must be fully defined the first time they are used for clarity of the claim. Claim 1 recites similar limitations to claim 7. Dependent claims of both Claims 1 and 7 are thus also objected to.
Claim 11 recites “wherein the one or more parameters”. However, the one or more parameters is not introduced in independent claim 1 which claim 5 is dependent on. Examine acknowledges that an introduction to “one or more parameters for light sources for a cube map” is introduced in the independent claim 1, but the claim was amended to remove this introduction. Claim 5 recites similar limitations to claim 11.
Claim 6 and 8-12 recites: “the WTRU”. Every abbreviation and acronym must be fully defined the first time they are referred to before using them. The term “WTRU” is also not introduced in an independent claim which causes a lack of antecedent basis that may confuse the interpretation of the claim. Examiner acknowledges that independent claims 1 and 7 initially defines and introduces a “wireless transmit receive unit (WTRU)” but was amended to no longer include it in the current claim. For the purposes of examination, Examiner will interpret “the WTRU of claim 7” in claims 8-12 to refer to “the device of claim 7”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 7 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kiyama et al (US 20230300374 A1) and Khronos (“glTF 2.0 Specification”), hereinafter Kiyama and Khronos.
Regarding claim 7, Kiyama teaches a device, device comprising: a processor ("The present disclosure relates to an information processing apparatus" - Abstract), the processor configured to obtain scene description data for a 3D scene ("A scene description file that describes a scene of 3D object content is generated, the scene description file including information associated with Media Presentation Description (MPD) that stores metadata of the 3D object content" - Abstract); a MPEG extension at a glTF file level
"As illustrated in FIG. 9, an MPEG media object (MPEG_media) that manages actual data such as video data is provided as an extension object (extensions) of a glTF object." - Par 81, Lines 2-5
and the processor configured to render the 3D scene based on the scene description data.
“For example, the client apparatus 400 decodes the encoded data of the 3D object content stored in the 3D object content file on the basis of the scene description file, performs rendering of the generated 3D object content, and generates and displays a display image.” – Par 226, Lines 10-15
NOTE: Kiyama discloses rendering the generated 3D object content which obtained by scene description file that describes a scene of 3D object content, see Abstract.
Kiyama does not teach that the processor is configured to obtain lighting information, wherein the lighting information is included in a MPEG lighting extension at a glTF file level; wherein the lighting information includes at least one texture to be applied to at least one face of a cube called a cube map, wherein the at least one texture includes a video that changes the lighting over time, wherein the at least one texture includes at least two different textures applied to two different faces of the cube, wherein the lighting information indicates which texture is used with which face; and the processor configured to render the 3D scene based on the lighting information and the scene description data such that lighting of the 3D scene changes over time according to the lighting information. However, Khronos teaches the processor is configured to obtain lighting information, wherein the lighting information is included in a MPEG lighting extension at a glTF file level;
“Lights are defined within a dictionary property in the glTF scene description file, by adding an extensions property to the top-level glTF 2.0 object and defining a KHR_lights_punctual property with a lights array inside it.” – KHR_light_punctual, Defining Lights, Par 1 and Fig. 3
PNG
media_image1.png
403
759
media_image1.png
Greyscale
NOTE: Khronos discloses a glTF file that includes various extensions for light such as KHR_lights_punctual and EXT_lights_image_based. Khronos does not teach that the lighting information is included in an MPEG lighting extension. After the combination, lighting information included in an extension at a glTF file level as taught by Khronos can modify the MPEG_media extension included in the glTF file level as taught by Kiyama to include lighting information. This then teaches lighting information within an MPEG extension at a glTF file level.
wherein the lighting information includes at least one texture to be applied to at least one face of a cube called a cube map,
“The cubemap used for specular radiance is defined along with its prefiltered mipmaps with each face of the cube defined as separate images.” - EXT_lights_image_based, Prefiltered Specular Radiance Cubemaps, par 1
NOTE: Khronos discloses using the EXT_lights_image_based extension for defining image based lights which consist of an environment map that represents specular radiance of the scene, see EXT_lights_image_based, Overview, par 1. A cubemap is used for specular radiance wherein each face of the cube is defined as separate images. The image-based lights used to define each face of the cubemap can be understood as textures from the light information applied to the cube map.
wherein the at least one texture includes a video that changes the lighting over time
“MPEG_texture_video extension provides the possibility to link a texture object defined in glTF 2.0 to media and its respective track, listed by an MPEG_media. MPEG_texture_video extension also provides a reference to the timed accessor, i.e. accessor with MPEG_accessor_timed extension, where the decoded timed texture will be made available.” - MPEG_texture_video, overview, Par 1.
NOTE: Khronos teaches an MPEG_texture_video extension that links a texture object to media such as a video. MPEG_texture_video also references the MPEG_accessor_timed extension which describes access to the dynamically changing data of a scene that contains timed media such as a video. One of ordinary skill in the art could then use the image-based lights from the light information that are mapped to each face of the cubemap and link those texture objects to media which would result in at least one texture that includes a video that dynamically changes the lighting over time. After the combination, the cubemap defined by image-based lights from the light information as taught by Khronos can be substituted with a video texture using the MPEG_texture_video extension to create a video texture based on the images from the light information. This modification can then be implemented into Kiyama’s MPEG_media extension located at the glTF file level.
wherein the at least one texture includes at least two different textures applied to two different faces of the cube, wherein the lighting information indicates which texture is used with which face;
“The cubemap used for specular radiance is defined along with its prefiltered mipmaps with each face of the cube defined as separate images.” - EXT_lights_image_based, Prefiltered Specular Radiance Cubemaps, Par 1.
PNG
media_image2.png
557
686
media_image2.png
Greyscale
NOTE: Khronos discloses that the cubemap comprises separate images for each face of the cube and further specifies the ordering of the cube faces (Positive X, Negative X, Positive Y, Negative Y, Positive Z, Negative Z). One of ordinary skill in the art would understand that the order identifies which texture is associated with each cube face since the cubemap necessarily associates each directional image with its corresponding directional face in order to correctly represent the surrounding environment for image-based lighting. Since each image depicts a different directional face of the cube map, the different faces of the cubes would have different images/textures.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Kiyama by incorporating the teachings of Khronos to obtain light information wherein the light information is included in a MPEG lighting extension at a glTF file level, wherein the lighting information includes at least one texture to be applied to at least one face of a cube called a cube map, wherein the at least one texture includes a video that changes the lighting over time, wherein the at least one texture includes at least two different textures applied to two different faces of the cube, and wherein the lighting information indicates which texture is used with which face. One would be motivated to make this combination since it is already well known that lighting information within an extension can exist inside a glTF file level (such as the Khronos extension) and that MPEG extensions can also exist in a glTF file level. One of ordinary skill could include the lighting information within an MPEG defined extension namespace instead of the Khronos namespace. The texture that includes video of light changing over time can then be applied to the cubemap so that the environment can accurately portray lighting reflections and illuminations when representing dynamic scenes. The different textures of each directional face of the cube face would help improve realism in order to produce dynamic lighting within the scene. NOTE: Kiyama discloses rendering the 3D scene based on the scene description data as explained above. After the combination, the time-varying lighting video textures created by the image-based lights in the EXT_lights_image_based extension modified by the MPEG_video_texture extension as taught by Khronos can be implemented by Kiyama’s MPEG extension located at the glTF file level. Kiyama can then use the obtained 3D scene description and the light information to render the 3D scene. Since the texture from the lighting information changes the lighting over time through the video texture, the lighting of the 3D rendered scene would also change over time. This would then teach that the processor renders the 3D scene based on the lighting information and the scene description data such that the lighting of the 3D scene changes over time according to the lighting information.
Regarding claim 1, the claim recites similar limitations to claim 7. Therefore, method claim 1 corresponds to the device disclosed in claim 7 and is rejected for the same reasons obviousness as used above.
Regarding claim 9, Kiyama in view of Khronos teaches the device of claim 7. Kiyama as modified by Khronos further teaches wherein the lighting information is applicable to one or more light sources of the 3D scene.
“This extension defines a set of lights for use with glTF 2.0. Lights define light sources within a scene. Many 3D tools and engines support built-in implementations of light types. Using this extension, tools can export and engines can import these lights. This extension defines three "punctual" light types: directional, point and spot. Punctual lights are defined as parameterized, infinitely small points that emit light in well-defined directions and intensities. These lights are referenced by nodes and inherit the transform of that node. A conforming implementation of this extension must be able to load light data defined in the asset and has to render the asset using those lights.” – Khronos, KHR_lights_punctual, Overview
NOTE: Khronos teaches that the KHR_lights_punctual extension defines the light sources within a scene as nodes. Each defined light has corresponding lighting information properties such as color, intensity, type, and range, see Khronos, KHR_lights_punctual, Light Types. Since the nodes are part of the scene, this shows that the lighting information is applicable to light sources of the 3D scene
Regarding claim 3, the claim recites similar limitations to claim 9. Therefore, method claim 3 corresponds to the device disclosed in claim 9 and is rejected for the same reasons obviousness as used above.
Claim(s) 2 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kiyama and Khronos, and Yip et al (US 12382150 B2), hereinafter Yip.
Regarding claim 8, Kiyama in view of Khronos teaches the device of claim 7. Kiyama does not teach wherein the 3D scene changes as a result of a scheduled event. However, Yip teaches wherein the 3D scene changes as a result of a scheduled event
“ In accordance with an aspect of the disclosure, a method for timed and event triggered updates in a scene description for extended reality (XR) multimedia is provided. The method includes parsing a scene description and at least one scene update track sample in a timed scene description update track or at least one scene update item to update the scene description and rendering the updated scene description on a display.” – Col 3, lines 10-17
NOTE: Yip discloses updating and displaying a scene based on a timed and event triggered updates to the scene description for extended reality XR media. This functionally corresponds to a 3D scene change based on the result of a scheduled event.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Kiyama by incorporating the teachings of Yip to have the 3D scene change as a result of a scheduled event. One would be motivated to make this combination to accurately represent environments that change over time. Updating the scene can be used for ensuring the proper lighting effects in the environment to occur at the right time based on the lighting information.
Regarding claim 2, the claim recites similar limitations to claim 8. Therefore, method claim 2 corresponds to the device disclosed in claim 8 and is rejected for the same reasons obviousness as used above.
Claim(s) 4-5 and 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kiyama and Khronos, and Arch Viz Artist (“Animating and Looping Lights in 3ds Max”), hereinafter Arch Viz Artist.
Regarding claim 10, Kiyama in view of Khronos teaches the device of claim 7. Kiyama does not teach wherein the 3D scene changes are part of an animation involving several lights of the 3D scene over time. However, Arch Viz Artist teaches wherein the 3D scene changes are part of an animation involving several lights of the 3D scene over time.
PNG
media_image3.png
847
1549
media_image3.png
Greyscale
"Turn on the Autokey option and change the intensity to 0 so the lights are turned off. Then go to frame 12 and change the intensity back to 300. Lastly, go to frame 25 so the distances between frames is the same and set the intensity to 0" - Timestamp 1:57-2:22
NOTE: Arch Viz Artist teaches creating an animation using the autokey feature in Autodesk 3DS Max for multiple light sources in the scene. The animation comprises of varying intensity so that the lights appear dim and then bright over time. After the combination, the animation of lights over time as demonstrated by Arch Viz Artist can modify the 3D scene rendered by Kiyama which would naturally change the 3D scene to include the animation of the lights changing intensity.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Kiyama by incorporating the teachings of Arch Viz Artist to have the 3D scene changes as part of an animation involving several lights of the 3D scene over time. One would be motivated to make this combination to add realism to the rendered scene so that the viewer can observe the changes to the 3D scene caused by the changes in light intensity.
Regarding claim 4, the claim recites similar limitations to claim 10. Therefore, method claim 4 corresponds to the device disclosed in claim 10 and is rejected for the same reasons obviousness as used above.
Regarding claim 11, Kiyama in view of Khronos teaches the device of claim 7. Kiyama as modified by Khronos further teaches wherein the one or more parameters further includes intensity or color,
“Light properties are unaffected by node transforms — for example, range and intensity do not change with scale. The intensity represents the luminous intensity that the light would emit if it were colored pure white ([1.0, 1.0, 1.0]). The color property acts as a wavelength-specific multiplier.” - Khronos, KHR_lights_punctual, Light Types, Lines 3-4
PNG
media_image4.png
298
827
media_image4.png
Greyscale
NOTE: Khronos teaches that the light types defined in the scene have various adjustable parameters such as intensity and color, see KHR_lights_punctual Table 1.
Kiyama in view of Khronos does not teach wherein the rendering of the 3D scene includes changing the intensity of a light of the 3D scene, or a color of a light of the 3D scene. However, Arch Viz Artist teaches wherein the rendering of the 3D scene includes changing the intensity of a light of the 3D scene, or a color of a light of the 3D scene
"Turn on the Autokey option and change the intensity to 0 so the lights are turned off. Then go to frame 12 and change the intensity back to 300. Lastly, go to frame 25 so the distances between frames is the same and set the intensity to 0" - Timestamp 1:57:2:22
NOTE: Arch Viz Artist discloses modifying the light intensity of the 3D scene over time by creating an animation for the light sources. After the combination, changing the intensity of lights in a scene as demonstrated by Arch Viz Artist can modify the 3D scene rendered by Kiyama which would naturally change the 3D scene to include the animation of the lights changing intensity.
It would have been obvious to one of ordinary skill before the effective filing date of the present invention to modify Kiyama by incorporating the teachings of Arch Viz Artist to have the rendered 3D scene include changing the intensity of the light in the 3D scene. One would be motivated to make this combination to add realism to the rendered scene so that the viewer can observe the changes to the 3D scene caused by the changes in light intensity.
Regarding claim 5, the claim recites similar limitations to claim 11. Therefore, method claim 5 corresponds to the device disclosed in claim 11 and is rejected for the same reasons obviousness as used above.
Claim(s) 6 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kiyama and Khronos, and Hong et al (US 20200326774 A1), hereinafter Hong.
Regarding claim 12, Kiyama in view of Khronos teaches the device of claim 7. Kiyama does not teach wherein the rendered 3D scene is displayed through a head mounted display operatively connected to the WTRU or a part of the WTRU. However Hong teaches wherein the rendered 3D scene is displayed through a head mounted display operatively connected to the WTRU or a part of the WTRU
“As shown in FIG. 5, the user is viewing a 3D scene 502 though the HMD 301. In some embodiments, the 3D scene 502 is an XR environment, which can be partially or completely virtual. As particular examples, the 3D scene 502 could be a real-world scene viewed in AR or MR or a virtual scene viewed in VR, AR, or MR.” – Par 57, Lines 1-7
NOTE: Hong teaches displaying the 3D scene through a head-mounted display (HMD). This would be obvious for one of ordinary skill in the art since the combination of Kiyama and Khronos obtains 3D scene data and renders the 3D scene with lighting effects as explained in the rejection of claim 7. This would naturally be displayed in an HMD as that is an extremely common technique for displaying 3D scene content. After the combination, the HMD used to display a 3D scene as taught by Hong can display the rendered 3D scene as taught by Kiyama. Note that examiner has chosen to interpret the WTRU as the device related to independent claim 7, please see claim objection of claim 12.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Kiyama by incorporating the teachings of Hong to display the rendered 3D scene through a head mounted display. One would be motivated to make this combination to provide the user with an immersive 3D scene environment where they may observe the timed lighting experience in the 3D scene.
Regarding claim 6, the claim recites similar limitations to claim 12. Therefore, method claim 6 corresponds to the device disclosed in claim 12 and is rejected for the same reasons obviousness as used above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID V. NGUYEN whose telephone number is (571)272-6111. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID VAN NGUYEN/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights