DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Amendment
The amendment filed 08/18/2026 has been entered.
Response to Arguments
Applicant's arguments filed 08/18/2026 have been fully considered but they are not persuasive.
Applicant argues that Danieau (US 20230418381 A1) fails to teach or suggest at least the amended limitations directed to generation of the surface roughness texture based on the normal texture as recited as recited claims 1 and 12.
Examiner replies that Applicant’s argument with respect to claims 1 and 12 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Instead, Boubekeur (US 20220292762 A1) is used to teach generation of the surface roughness texture based on the normal texture.
Regarding the remaining arguments: Applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth
below.
Conclusion: The rejections set forth in the previous Office Action are shown to
have been proper, and the claims are rejected below. New citations and parenthetical
remarks can be considered new grounds of rejection, such new grounds of rejection are
necessitated by Applicant’s amendments to the claims. Therefore, the present Office Action is made final.
Claim Objections
Claim 1 objected to because of the following informalities:
Claim 1, line 19 “representing a distribution of surface roughness” should be representing distribution of surface roughness.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12 and 19-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “basic” in claim 1 and 12 is a relative term which renders the claim indefinite. The term “basic” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “basic” has a plain meaning of forming a base, foundation, or essential starting point of something as defined by dictionary. However, the claim is indefinite since one of ordinary skill in the art cannot translate the definition into a meaningful precise claim scope to interpret the metes and bounds of the term “basic”. For examination purposes, the term “basic” is being interpreted as a roughness parameter that indicates the degree of roughness of a surface.
Claim 8 line 3 recites “sensory representation metadata”, which should indicate the first or the second.
Claim 10 line 4 recites “first sensory...”, which should be “second sensory…”
The term “preferentially” in claim 20 is a relative term which renders the claim indefinite. The term “preferentially” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “preferentially” has a plain meaning of in a way that shows someone or something is preferred over others, or in a manner that gives an advantage, priority, or superior treatment to a specific person, group, or outcome. However, the claim is indefinite since one of ordinary skill in the art cannot translate the definition into a meaningful precise claim scope to interpret the metes and bounds of the term “preferentially”. For examination purposes, the term “preferentially” is being interpreted as applying a property value according to a defined priority.
Claims 1-12 and 19-21 will be interpreted and rejected as best understood by the examiner.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 2 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 1 recites “scene description information that defines a configuration of the three-dimensional space and three- dimensional object data including video object data that defines a three-dimensional video object in the three-dimensional space, wherein the scene description information includes first sensory representation metadata indicating at least a basic surface roughness of a scene configured by the three-dimensional space,… second sensory representation metadata..” Claim 2 recites the substantially same limitations. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12, 19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Danieau et al. (US 20230418381 A1) (hereinafter Danieau) in view Boubekeur et al. (US 20220292762 A1) (hereinafter Boubekeur).
Regarding claim 1, Danieau discloses A generation apparatus, comprising: (Abstract, "A haptic rendering device and corresponding rendering method allow to render a haptic effect defined in a haptic signal comprising information representative of an immersive scene description")
circuitry configured to generate three-dimensional spatial data used in rendering processing executed to represent a three-dimensional space,
wherein the three-dimensional spatial data includes scene description information that defines a configuration of the three-dimensional space (processor 101 in Figure 1; Figure 3; para. [0047], “FIG. 3 illustrates an example of data structure of an immersive scene according to at least one embodiment. This embodiment is based on the glTF™ file format. The core of glTF™ is a JSON file that describes the structure and composition of a scene containing 3D models. The figure shows the relationship between the elements composing this structure. In this context, a scene 300 is the top level element gathering all the other elements. Most notably, it comprises an array of nodes. Each node 301 can contain child nodes allowing to create a hierarchy. A node may refer to a mesh or camera or skin and a local geometrical transform may be associated with the node. A mesh 310 corresponds to the geometry data required to render the mesh. A skin 320 is used to perform vertex skinning to let vertices of a mesh be influenced by the bones of a skeleton based on its pose. A camera 325 determines a projection matrix. An animation 340 may be applied to the properties of a node. Buffers 355 contain the data used for the geometry of 3D models, animations and skinning. BufferViews 350 add structural information to the buffer data, while accessor 345 define the exact type and layout of BufferViews. Material 360 determines how an object should be rendered based on physical material properties. Texture 365 allows to define the appearance of an object. Images 370 define the image data used for a texture while a sampler 380 describes the wrapping and scaling of textures. All these elements of a glTF™ file allow to define a conventional immersive scene with any haptic feedback."; Figure 4; Figure 5)
and three-dimensional object data including video object data that defines a three-dimensional video object in the three-dimensional space, ([0030] FIG. 1 illustrates a block diagram of an example of system in which various aspects and embodiments are implemented. In the depicted immersive system, the user Alice uses the haptic rendering device 100 to interact with a server 180 hosting an immersive scene 190 through a communication network 170. This immersive scene 190 may comprise various data and/or files representing different elements (scene description 191, audio data, video data, 3D models, and haptic object 192) required for its rendering)
wherein the scene description information includes first sensory representation metadata indicating at least a basic surface roughness of a scene configured by the three-dimensional space, ([0004] An example of a haptic object is an explosion. An explosion can be rendered though vibrations and heat, thus combining different haptic effects on the user to improve the realism. An immersive scene typically comprises multiple haptic objects, for example using a first haptic object related to a global effect and a second haptic object related to a local effect.; [0029] Haptic objects may be related to the global environment, such as a breeze of wind,; [0052] . In one variant embodiment, the haptic volume is the whole immersive scene so that the haptic effect is global and independent of the position of the user.; [0098] Table 16 illustrates an example of syntax for the first level of the extension describing the global haptic experience according to at least one embodiment)
wherein the three-dimensional object data includes a normal texture used to visually represent a surface of the three- dimensional video object, and ([0009] In addition, modern 3D engines are able to map textures to 3D objects. These textures contain information related to various appearance-related parameters such as the color of the object but also information on the geometry such as the normal maps or the bump maps that help modern visual rendering algorithms in the rendering process as well as more complex parameters such as diffuse, emissive, glossiness that also determine how to render the object.; [0077] FIG. 6A illustrates an example of 3D object according to an embodiment using haptic texture maps. The 3D object 1700 represents a metallic bottle 1710 with a black soft rubber holder 1720 to isolate the user's hand from the temperature of the bottle. Conventionally, texture files may be used to describe the color, diffuse, emissive, normal, occlusion, roughness metallic, specular glossiness of the object materials and allow an appropriate (physical-based) rendering by the rendering engine based on the texture files.)
second sensory representation metadata indicating at least a basic surface roughness of the three-dimensional video object, and ([0004] An example of a haptic object is an explosion. An explosion can be rendered though vibrations and heat, thus combining different haptic effects on the user to improve the realism. An immersive scene typically comprises multiple haptic objects, for example using a first haptic object related to a global effect and a second haptic object related to a local effect.; Figure 3; [0048] Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365. Such haptic objects are described herein.; According to an embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow to simulate different harshness and temperature of the bottle of FIG. 6A by defining different parameters for different haptic properties of specific areas of the 3D object.; [0079] According to an embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow to simulate different harshness and temperature of the bottle of FIG. 6A by defining different parameters for different haptic properties of specific areas of the 3D object.)
wherein the circuitry is further configured to generate, ([0079] According to an embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow to simulate different harshness and temperature of the bottle of FIG. 6A by defining different parameters for different haptic properties of specific areas of the 3D object. For example, the metallic bottle 1710 is rendered as a hard and cold metallic bottle with a softer and warmer rubber holder 1720 by using additional textures information, encoded using similar principles based on texture maps. A temperature texture map illustrated in FIG. 6B determines the temperature on the surface of the object (the metallic part being colder than the plastic part) and a rate harness texture map illustrated in FIG. 6C indicates that the metallic part is rigid while the rubber holder is soft.; Figure 6A; Figure 6B; Figure 6C)
PNG
media_image1.png
478
321
media_image1.png
Greyscale
Danieau fails to disclose generate, based on the normal texture, a surface roughness texture representing a distribution of surface roughness on the surface of the three-dimensional video object.
Boubekeur taches generate, based on the normal texture, a surface roughness texture representing a distribution of surface roughness on the surface of the three-dimensional video object. ([0081] The output of the AO map 712, the normal map 706, and the albedo map 704 is concatenated or otherwise used, as the roughness computation 714, to derive the roughness map 712.; [0083] n some embodiments, the Ambient Occlusion map 710 is computed using the Substance Designer filter HBAO 709 (Image-Space Horizon-Based Ambient occlusion). The displacement factor to consider for this computation is exposed in order to be used. As illustrated in FIG. 7, the roughness map 712 is calculated (714) using the albedo map 704, the normal map 706, the ambient occlusion map 710, using the following formula:
Roughness—max_(1-normalZ)+(1-ambient occlusion), 1-luminance); Figure 7)
PNG
media_image2.png
508
914
media_image2.png
Greyscale
As both Danieau and Boubekeur are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Danieau to include generate, based on the normal texture, a surface roughness texture representing a distribution of surface roughness on the surface of the three-dimensional video object, in the context of generating surface in image processing, according to the teaching of Boubekeur, in order to provide image-to-material translation, especially when input images include heterogeneous materials, complex shading, and/or complex surface texture characteristics (para [0002] of Boubekeur).
Regarding claim 2, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 1, wherein the circuitry generates the scene description information including the first sensory representation metadata and the three-dimensional object data including the secondary sensory representation metadata. (Danieau, processor 101 in Figure 1; para. [0004] An example of a haptic object is an explosion. An explosion can be rendered though vibrations and heat, thus combining different haptic effects on the user to improve the realism. An immersive scene typically comprises multiple haptic objects, for example using a first haptic object related to a global effect and a second haptic object related to a local effect.; Figure 3; [0048] Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365. Such haptic objects are described herein.; According to an embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow to simulate different harshness and temperature of the bottle of FIG. 6A by defining different parameters for different haptic properties of specific areas of the 3D object.; [0079] According to an embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow to simulate different harshness and temperature of the bottle of FIG. 6A by defining different parameters for different haptic properties of specific areas of the 3D object.; Figure 3; para. [0047], " FIG. 3 illustrates an example of data structure of an immersive scene according to at least one embodiment. This embodiment is based on the glTF™ file format. The core of glTF™ is a JSON file that describes the structure and composition of a scene containing 3D models. The figure shows the relationship between the elements composing this structure. In this context, a scene 300 is the top level element gathering all the other elements. Most notably, it comprises an array of nodes. Each node 301 can contain child nodes allowing to create a hierarchy. A node may refer to a mesh or camera or skin and a local geometrical transform may be associated with the node. A mesh 310 corresponds to the geometry data required to render the mesh. A skin 320 is used to perform vertex skinning to let vertices of a mesh be influenced by the bones of a skeleton based on its pose. A camera 325 determines a projection matrix. An animation 340 may be applied to the properties of a node. Buffers 355 contain the data used for the geometry of 3D models, animations and skinning. BufferViews 350 add structural information to the buffer data, while accessor 345 define the exact type and layout of BufferViews. Material 360 determines how an object should be rendered based on physical material properties. Texture 365 allows to define the appearance of an object. Images 370 define the image data used for a texture while a sampler 380 describes the wrapping and scaling of textures. All these elements of a glTF™ file allow to define a conventional immersive scene with any haptic feedback."; para. [0070], "This allows to define a more sophisticated haptic effect than simply using a sine signal at fixed frequency. In addition, the intensity parameter may also apply to the haptic signal. This allows to share a unique haptic signal file and apply it at different levels of intensity in different haptic objects. In an example using haptic objects with concentric volumes, the haptic objects could share the same haptic signal file and have increasing intensity to provide a progressive effect. The file format is adapted for storing a haptic signal. Example of formats for such use are Waveform audio (WAV), Object Haptic Metadata (OHM), Apple Haptic Audio Pattern (AHAP) or Immersion Corporation's HAPT format.")
Regarding claim 3, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 2, wherein the circuitry generates the scene description information further including a basic temperature of the scene configured by the three-dimensional space as the first sensory representation metadata. (Danieau, processor 101 in Figure 1; para. [0076], " Table 12 illustrates an example of a syntax based on glTF™ for the temperature haptic effect according to at least one embodiment. A temperature haptic effect can be defined simply by a numeric temperature value expressed in Celsius degrees in a “value” syntax element. A default value of “20.0” corresponding to no pressure should be used if the “value” syntax element is not present. All embodiments presented above in the context of the vibration haptic effect apply similarly to the temperature haptic effect.”; para. [0048], " Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365.")
Regarding claim 4, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 2, wherein the circuitry generates the three-dimensional object data further including basic temperature of the three-dimensional video object as the second sensory representation metadata. (Danieau, processor 101 in Figure 1; para. [0076], "Table 12 illustrates an example of a syntax based on glTF™ for the temperature haptic effect according to at least one embodiment. A temperature haptic effect can be defined simply by a numeric temperature value expressed in Celsius degrees in a “value” syntax element. A default value of “20.0” corresponding to no pressure should be used if the “value” syntax element is not present. All embodiments presented above in the context of the vibration haptic effect apply similarly to the temperature haptic effect.”; para. [0048], "Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365."; para. [0078], "Conventionally, texture files may be used to describe the color, diffuse, emissive, normal, occlusion, roughness metallic, specular glossiness of the object materials and allow an appropriate (physical-based) rendering by the rendering engine based on the texture files."; para. [0081], "According to an embodiment, the glTF™-based syntax for defining a haptic effect comprises a haptic texture map to define the haptic effect. Different haptic features may be considered and need to be differentiated. In an embodiment, the haptic features listed in table 13 are considered."; TABLE 13; para. [0045, “In a first example, the immersive scene 191 may comprise a virtual environment of an outdoor camp site where the user can move an avatar representing him. A first haptic feedback could be a breeze of wind that would be present anywhere in the virtual environment and generated by a fan. A second haptic feedback could be a temperature of 30° C. when the avatar is in proximity of a campfire.”)
PNG
media_image3.png
410
475
media_image3.png
Greyscale
Regarding claim 5, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 2, wherein the circuitry is further configured to generate a temperature texture representing a temperature as the second sensory representation metadata with respect to a surface of the three-dimensional video object. (para. [0076], "Table 12 illustrates an example of a syntax based on glTF™ for the temperature haptic effect according to at least one embodiment. A temperature haptic effect can be defined simply by a numeric temperature value expressed in Celsius degrees in a “value” syntax element. A default value of “20.0” corresponding to no pressure should be used if the “value” syntax element is not present. All embodiments presented above in the context of the vibration haptic effect apply similarly to the temperature haptic effect.”; para. [0078], "Conventionally, texture files may be used to describe the color, diffuse, emissive, normal, occlusion, roughness metallic, specular glossiness of the object materials and allow an appropriate (physical-based) rendering by the rendering engine based on the texture files."; para. [0048], "Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365."; Table 13)
Regarding claim 6, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 5, wherein the circuitry generates the surface roughness texture with a variation range determined based on the normal texture. (Danieau, "pbrMetallicRoughness" in TABLE 2;
PNG
media_image4.png
127
439
media_image4.png
Greyscale
;
TABLE 13; para. [0083], “kinetic-friction” allows to determine the coefficient of the kinetic friction, i.e. the force due to the friction between each object. The value is stored in an 8-bit texture and covers values from −5 to +5 with a resolution of 0.04, “static-friction” allows to determine the coefficient of the static friction, i.e. the force necessary to make to object slide over each other. The value is stored in an 8-bit texture and covers values from −5 to +5 with a resolution of 0.04,"; para. [0077] Conventionally, texture files may be used to describe the color, diffuse, emissive, normal, occlusion, roughness metallic, specular glossiness of the object materials and allow an appropriate (physical-based) rendering by the rendering engine based on the texture files.)
PNG
media_image3.png
410
475
media_image3.png
Greyscale
Regarding claim 7, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 2, wherein a data format of the scene description information is a glTF (GL Transmission Format). (Danieau, para. [0047], "FIG. 3 illustrates an example of data structure of an immersive scene according to at least one embodiment. This embodiment is based on the glTF™ file format. The core of glTF™ is a JSON file that describes the structure and composition of a scene containing 3D models. The figure shows the relationship between the elements composing this structure.”)
Regarding claim 8, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 7, wherein the sensory representation metadata is stored in at least one of an extension area of a node corresponding to the scene configured by the three- dimensional space, an extension area of a node corresponding to the three-dimensional video object, or an extension area of a node corresponding to a surface state of the three-dimensional video object. (Danieau, para. [0054], "Table 1 illustrates an example of a syntax for defining a haptic object according to at least one embodiment. More particularly, the figure shows the JSON syntax based on the extension mechanism of glTF™. The extension for haptic objects designed according at least one embodiment is identified as “IDCC_Haptics” in this example of syntax."; para. [0062], "The scene comprises a single node named “haptic_example”. It indicates the use of the “IDCC_haptics” extension, the version (2.0) of the glTF™ Specification and no buffer is used for loading resources.")
Regarding claim 9, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 8, wherein in the scene description information, at least the basic surface roughness of the scene is stored as the first sensory representation metadata in the extension area of the node corresponding to the scene. (Danieau, Figure 3;
PNG
media_image5.png
553
621
media_image5.png
Greyscale
para. [0047], "FIG. 3 illustrates an example of data structure of an immersive scene according to at least one embodiment. This embodiment is based on the glTF™ file format. The core of glTF™ is a JSON file that describes the structure and composition of a scene containing 3D models. The figure shows the relationship between the elements composing this structure. In this context, a scene 300 is the top level element gathering all the other elements. Most notably, it comprises an array of nodes. Each node 301 can contain child nodes allowing to create a hierarchy. A node may refer to a mesh or camera or skin and a local geometrical transform may be associated with the node."; para. [0048], "Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365. Such haptic objects are described herein."; para. [0054], "Table 1 illustrates an example of a syntax for defining a haptic object according to at least one embodiment. More particularly, the figure shows the JSON syntax based on the extension mechanism of glTF™. The extension for haptic objects designed according at least one embodiment is identified as “IDCC_Haptics” in this example of syntax."; para. [0006], "Haptics refers to sense of touch and includes two dimensions, tactile and kinesthetic. The first one relates to tactile sensations such as friction, roughness, hardness, temperature and is felt through the mechanoreceptors of the skin (Merkel cell, Ruffini ending, Meissner corpuscle, Pacinian corpuscle).”; para. [0081], "According to an embodiment, the glTF™-based syntax for defining a haptic effect comprises a haptic texture map to define the haptic effect. Different haptic features may be considered and need to be differentiated. In an embodiment, the haptic features listed in table 13 are considered."; TABLE 13)
PNG
media_image3.png
410
475
media_image3.png
Greyscale
Regarding claim 10, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 8, wherein in the scene description information, at least the basic surface roughness of the three-dimensional video object is stored as the first sensory representation metadata in the extension area of the node corresponding to the three-dimensional video object. (Danieau, para. [0030], "This immersive scene 190 may comprise various data and/or files representing different elements (scene description 191, audio data, video data, 3D models, and haptic object 192) required for its rendering."; claim 8, " The method according to claim 1, wherein the element of the scene is selected among a set comprising 3D objects, 2D or 3D video, and omnidirectional video."; para. [0005], "The principles described herein apply to any immersive environment using haptics such as augmented reality, virtual reality, mixed reality or haptics-enhanced video (or omnidirectional/360° video) rendering, for example, and more generally apply to any haptics-based user experience."; para. [0047], "FIG. 3 illustrates an example of data structure of an immersive scene according to at least one embodiment. This embodiment is based on the glTF™ file format. The core of glTF™ is a JSON file that describes the structure and composition of a scene containing 3D models. The figure shows the relationship between the elements composing this structure. In this context, a scene 300 is the top level element gathering all the other elements. Most notably, it comprises an array of nodes. Each node 301 can contain child nodes allowing to create a hierarchy. A node may refer to a mesh or camera or skin and a local geometrical transform may be associated with the node."; para. [0048], "Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365. Such haptic objects are described herein."; para. [0054], "Table 1 illustrates an example of a syntax for defining a haptic object according to at least one embodiment. More particularly, the figure shows the JSON syntax based on the extension mechanism of glTF™. The extension for haptic objects designed according at least one embodiment is identified as “IDCC_Haptics” in this example of syntax."; para. [0006], "Haptics refers to sense of touch and includes two dimensions, tactile and kinesthetic. The first one relates to tactile sensations such as friction, roughness, hardness, temperature and is felt through the mechanoreceptors of the skin (Merkel cell, Ruffini ending, Meissner corpuscle, Pacinian corpuscle).”; para. [0081], "According to an embodiment, the glTF™-based syntax for defining a haptic effect comprises a haptic texture map to define the haptic effect. Different haptic features may be considered and need to be differentiated. In an embodiment, the haptic features listed in table 13 are considered."
Regarding claim 11, the combination of Danieau and Boubekeur discloses The generation apparatus according to 8, wherein in the scene description information, at least link information to the surface roughness texture representing the surface roughness is stored in the extension area of the node corresponding to the surface state of the three-dimensional video object. (Danieau, para. [0030], "This immersive scene 190 may comprise various data and/or files representing different elements (scene description 191, audio data, video data, 3D models, and haptic object 192) required for its rendering."; claim 8, " The method according to claim 1, wherein the element of the scene is selected among a set comprising 3D objects, 2D or 3D video, and omnidirectional video."; para. [0005], "The principles described herein apply to any immersive environment using haptics such as augmented reality, virtual reality, mixed reality or haptics-enhanced video (or omnidirectional/360° video) rendering, for example, and more generally apply to any haptics-based user experience."; para. [0047], "FIG. 3 illustrates an example of data structure of an immersive scene according to at least one embodiment. This embodiment is based on the glTF™ file format. The core of glTF™ is a JSON file that describes the structure and composition of a scene containing 3D models. The figure shows the relationship between the elements composing this structure. In this context, a scene 300 is the top level element gathering all the other elements. Most notably, it comprises an array of nodes. Each node 301 can contain child nodes allowing to create a hierarchy. A node may refer to a mesh or camera or skin and a local geometrical transform may be associated with the node."; para. [0048], "Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365. Such haptic objects are described herein."; para. [0054], "Table 1 illustrates an example of a syntax for defining a haptic object according to at least one embodiment. More particularly, the figure shows the JSON syntax based on the extension mechanism of glTF™. The extension for haptic objects designed according at least one embodiment is identified as “IDCC_Haptics” in this example of syntax."; para. [0006], "Haptics refers to sense of touch and includes two dimensions, tactile and kinesthetic. The first one relates to tactile sensations such as friction, roughness, hardness, temperature and is felt through the mechanoreceptors of the skin (Merkel cell, Ruffini ending, Meissner corpuscle, Pacinian corpuscle).”; para. [0081], "According to an embodiment, the glTF™-based syntax for defining a haptic effect comprises a haptic texture map to define the haptic effect. Different haptic features may be considered and need to be differentiated. In an embodiment, the haptic features listed in table 13 are considered."; TABLE 13; Figure 5; Figure 8A-8B; para. [0096]; TABLE 15; para. [0083])
Regarding claim 12, claim 12 is the method claim of claim 1 and is accordingly rejected under same rationale.
Regarding claim 19, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 1, wherein a variation range of surface irregularity represented by the surface roughness texture is determined based on at least one of the basic surface roughness of the scene or the basic surface roughness of the three-dimensional video object. (Danieau, para. [0081], "According to an embodiment, the glTF™-based syntax for defining a haptic effect comprises a haptic texture map to define the haptic effect. Different haptic features may be considered and need to be differentiated. In an embodiment, the haptic features listed in table 13 are considered."; TABLE 13; ([0004] An example of a haptic object is an explosion. An explosion can be rendered though vibrations and heat, thus combining different haptic effects on the user to improve the realism. An immersive scene typically comprises multiple haptic objects, for example using a first haptic object related to a global effect and a second haptic object related to a local effect.; Figure 3; [0048] Therefore, in at least one embodiment, the glTF™ file further comprises a haptic object 330 that describes a haptic feedback to be rendered. In a variant embodiment, the haptic object comprises a haptic texture map 335 whose data may be stored along with the other textures 365. Such haptic objects are described herein.; According to an embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow to simulate different harshness and temperature of the bottle of FIG. 6A by defining different parameters for different haptic properties of specific areas of the 3D object.; [0079] According to an embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow to simulate different harshness and temperature of the bottle of FIG. 6A by defining different parameters for different haptic properties of specific areas of the 3D object.)
Regarding claim 21, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 5, wherein link information to the temperature texture and link information to the surface roughness texture are stored in the scene description information. (Danieau, [0096] FIGS. 8A and 8B illustrate an example of scene description representing a haptic object comprising haptic texture maps according to at least one embodiment. The scene corresponds to the teapot with a temperature related haptic map as described in FIGS. 7A and 7B. The scene description syntax spans over FIGS. 8A and 8B. Starting from the end of the glTF™ description file, in FIG. 8B, the scene description 2001 comprises a single node named teapot. The set of nodes 2010 thus a single child node named teapot. The geometry is defined in 2020 as the first mesh with a translation to position the object within the scene. The single child node also comprises a haptic object 2030 that comprises two effects: a vibration effect 2031 and temperature haptic map 2032. The haptic map 2032 is defined as providing a haptic effect related to the “temperature” and uses the texture image of index ‘0’, thus the first one, in the list of texture files 2040. The vibration effect is directly defined by its parameters: a vibration frequency of 250 Hz and an intensity of 0.7 to be applied on the mesh of the object since the ShapeType parameter is equal to 2. The other sections of the scene description file are related to the meshes defining the geometry of the object 2050, and in FIG. 8A, the materials 2060 defining the visual aspect of the surface through the defaultMat material, the buffer views 2070, the buffers storing the data 2075, the version number 2080, the description of the buffers 2085 and the list of extensions used 2090.; Table 13; Table 15; [0083] Table 15 illustrates an example of syntax for defining haptic texture map properties for haptic objects according to at least one embodiment. This syntax allows to determine the parameters for different types of haptic effect: Examiner’s note: It’s implicit that link information to the surface roughness based on glTF, haptic features of Table 13, and haptic effect parameter of Table 15).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Danieau et al. (US 20230418381 A1) (hereinafter Danieau) in view Boubekeur et al. (US 20220292762 A1) (hereinafter Boubekeur) and further in view of Lebaredian et al. (US 20220134222 A1) (hereinafter Lebaredian).
Regarding claim 20, the combination of Danieau and Boubekeur discloses The generation apparatus according to claim 1, wherein, when both the basic surface roughness of the scene and the basic surface roughness of the three-dimensional video object are set for a same area, (Danieau, [0064] In one embodiment, a scene comprises multiple overlapping haptic objects having concentric volumes with different haptic effects. For example, a set of haptic objects may use concentric spheres located around the fireplace, the volumes having decreasing sizes with increasing temperature values. Using this technique, the user would feel a progressive heat increase when approaching the fireplace. Since the user is going to collide with multiple spheres, the smallest (i.e. the closest one to the fire) will be selected.
The combination of Danieau and Boubekeur fails to disclose the basic surface roughness of the three-dimensional video object indicated by the second sensory representation metadata is used preferentially.
Lebaredian teaches the basic surface roughness of the three-dimensional video object indicated by the second sensory representation metadata is used preferentially. ([0061]-[0062]; [0066] Thus, the asset 230 may inherit a property-value pair 228 from the asset 222 assigning the color property to blue. The layer 202 may be ranked as a stronger layer than the layer 204. Thus, the property-value pair 228 may override the property-value pair 236 for the asset 230. As such, the assets 216, 218, and 220 may each also inherit the property-value pair 228 from the asset 230. However, the scene description for the asset 220 may include a property-value pair 226 which may override the property-value pair 228. As such, the data store manager 108 may resolve the asset 216 as having the property-value pair 228, the asset 218 as having the property-value pair 228, and the asset 220 as having the property-value pair 226, as shown in the resolved view 206.)
As Danieau, Boubekeur, and Lebaredian are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Danieau and Boubekeur to include wherein, when both the basic surface roughness of the scene and the basic surface roughness of the three-dimensional video object are set for a same area, the basic surface roughness of the three-dimensional video object indicated by the second sensory representation metadata is used preferentially, in the context of generating three-dimensional world/space, according to the teaching of Lebaredian, in order to allow for a rich description of a 3D world using complex hierarchical relationships between elements or objects and scenes (para [0003] of Lebaredian).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hyorim Park/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights