Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Response to Arguments
Regarding 35 USC § 102/103.
Applicant argues:
Therefore, it is respectfully submitted that Lee does not disclose "Smoother process before encoding," as alleged by the Office Action, and Lee fails to teach or suggest "simplifying the geometry information; and encoding a stream including the color information, the alpha information and the simplified geometry information and sending the encoded stream from the server to the client for decoding and reconstructing," as recited in amended claim 1. For at least the same reasons that claim 1 is patentable over the cited references, it is respectfully submitted that claim 10 is also patentable, to the extent that claim 10 recites similar features as claim 1. Applicant, therefore, respectfully requests that the corresponding rejection under 35 U.S.C. §103 be reconsidered and withdrawn.
[…] Accordingly, Lee fails to teach or suggest "projecting information on the reproduced shape of the 3D object to reconstruct the 3D object, the information resulting from combining the color information and the alpha information," as recited in amended claim 7. For at least the same reasons that claim 7 is patentable over the cited references, it is respectfully submitted that claim 11 is also patentable, to the extent that claim 11 recites similar features as claim 7. Applicant, therefore, respectfully requests that the corresponding rejection under 35 U.S.C. §103 be reconsidered and withdrawn.
Examiner replies that:
Applicant has amended the claims to change the scope since the previous action. The amendment(s) necessitate new ground(s) of rejection and are rejected in detail under the § 102/103 headings below.
Double Patenting
Claims 1-13 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 of copending Application No. 17/413135 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because the current claims are strictly broader than 17/413135. It is well settled that "anticipation is the epitome of obviousness," in re McDaniel, 293 F3d. 1379,1385 (Fed. Cir. 2002}{quoting Connell v. Sears Roebuck & Co,, 722 F.2d 1542, 1.548 (Fed. Cir. 1.983}}; In re Fracalossi, 681 F.2d 792, 794 (CCPA 1982). This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claims 1, 7-8, 10-15 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim1, 7-11 of U.S. Patent No. 12254566. Although the claims at issue are not identical, they are not patentably distinct from each other because the current claims are strictly broader than 12254566. It is well settled that "anticipation is the epitome of obviousness," in re McDaniel, 293 F3d. 1379,1385 (Fed. Cir. 2002}{quoting Connell v. Sears Roebuck & Co,, 722 F.2d 1542, 1.548 (Fed. Cir. 1.983}}; In re Fracalossi, 681 F.2d 792, 794 (CCPA 1982).
18/549458
17/413135
12254566
1.A method for sending at least one 3D object generated based on a position information of a client from a server to the client, comprising:
1.A method for sending at least one 3D object from a server to a client, the server comprising at least one processor and a memory, the method comprising executing, by the at least one processor, instructions stored in the memory, for:
1. A method for analyzing an input by a user received by a client on a server; the method comprising; sending a content to be displayed by the client from the server to the client; receiving from the client, an input information by the user to the client during the content is displayed; analyzing the input information by the user received from the client; changing the content to be displayed by the client based on the result of the analysis, wherein the changed content includes 3D objects;
extracting color information, alpha information and geometry information from the 3D object on the server;
extracting color information-and alpha information on the server using a render camera, wherein the alpha information represents transparency to the color information of conventional RGB; and extracting geometry information from the 3D object on the server using a depth camera;
extracting by the server color information, alpha information, and geometry information from the 3D objects;
simplifying the geometry information;
simplifying the geometry information, wherein the simplifying the geometry information includes converting a cloud of points extracted from the 3D object to information of vertices of triangles polygons representing a shape of the 3D object
simplifying by the server the geometry information by converting the geometry information to information of vertex of triangles;
and encoding a stream including the color information, the alpha information and the simplified geometry information and sending the encoded stream from the server to the client for decoding and reconstructing.
and encoding and compressing a 3D stream including the color information, the alpha information, and the simplified geometry information; and sending a container stream of the encoded 3D stream from the server to the client.
encoding by the server the color information, the alpha information, and the simplified geometry information as a stream; and sending the changed content as the stream to the client.
3
3
4
4
5
5
6
6
7
7
9
8
8
9
9
9
10
10
8
11
11
11
12
12
10
13
12+7
10+7
14-15
1
Allowable Subject Matter
Claim 3 overcomes the prior art but is rejected under Double Patenting above. The claim recites “wherein the stream further includes at least one of position information of the client, metadata, sound data, and a command” which requires at least one position, at least one metadata, at least one sound data and at least one command (SuperGuide, the Federal Circuit held that the plain meaning of “at least one of A, B, and C” means: at least one A, at least one of B and at least one of C.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-8, 10-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Van Den Berghe U.S. Patent/PG Publication 20130321417 in view of Macq U.S. Patent/PG Publication 20130321576.
Regarding claim 1 (independent):
A method for sending at least one 3D object generated based on a position information of a client from a server to the client the method, comprising: (Van Den Berghe [0024] According to a second aspect of the present invention, a server is disclosed for transmitting a 3D representation of a 3D scene model to a client device, the 3D scene model being information defining geometry and material information for a set of 3D objects and light sources, over a data communication network, wherein the 3D representation corresponds to a virtual viewing point for a user associated with the client device,).
extracting color information, (Van Den Berghe [0012] Material information is information defining the optical characteristics of the (virtual) materials which are defining the 3D objects. Material information may comprise information which describes the interaction of the respective 3D objects with light. Material information may comprise at least one of colour, light absorption, diffusion, refraction and reflection parameters of objects or parts of objects. Material information may comprise a plurality of or all of these parameters.) and geometry (Van Den Berghe [0011] Geometry information is information relating to the (possibly changing) geometry of objects within the 3D scene model.) information from the 3D object on the server (Van Den Berghe [0024] a server is disclosed for transmitting a 3D representation of a 3D scene model to a client device, the 3D scene model being information defining geometry and material information for a set of 3D objects
simplifying the geometry information (Van Den Berghe [0026] performing compression of the representation information;) where the compression is a form of simplifying, since it reduces information for more efficient transmission.
and encoding a stream including the color information, the (Van Den Berghe [0056] The structure of the scene is than encoded using "traditional" mesh encoding techniques. According to embodiments of the present invention, the illumination information is encoded and combined with the mesh into the output stream.)
sending the encoded stream from the server to the client for decoding and reconstructing (Van Den Berghe [0056] For both cases we assume the encoding is based on a in-process encode/decode loop to create temporal references.)(Van Den Berghe [0027] forwarding the compressed representation information towards the client device for display).
Van Den Berghe does not teach alpha information. In a related field of endeavor, Macq teaches:
extracting color information, alpha information (Macq [0010] [0011] extracting color information, depth information, and transparency information pertaining to groups of objects in respective object layers from respective ones of the first plurality of 2D video streams)
and encoding a stream including the color information, the alpha information and the (Macq [0004] encoding a plurality of 2D video streams, respective ones of the plurality of 2D video streams including color information, depth information, and transparency information pertaining to objects in respective object layers)
sending the encoded stream from the server to the client (Macq [0010] According to an aspect of the invention, there is provided a method for decoding a multiview video stream representing a plurality of viewpoints of a 3D scenery comprising objects, the method comprising for a first one of the viewpoints: extracting a first combined 2D video stream from the multiview video stream; deserializing the first combined 2D video stream into a first plurality of 2D video streams; [0011] extracting color information, depth information, and transparency information pertaining to groups of objects in respective object layers from respective ones of the first plurality of 2D video streams; using the respective color information, depth information, and transparency information pertaining to the groups of objects in the respective object layers to generate a first superimposed stream, the first superimposed stream being consistent with the first one of the viewpoints.) for decoding and reconstructing (Macq [0021] a view synthesizer, operatively connected to the decoder, the visualizing agent being configured to use the respective color information, depth information, and transparency information pertaining to the objects in the respective object layers to generate a first superimposed stream, the first superimposed stream being consistent with the first one of the viewpoints.)
Therefore, it would have been obvious before the effective filing date of the claimed invention to use alpha as taught by Macq. The rationale for doing so would have been that it combines prior art elements according to known methods to yield predictable results where Van Den Berghe encodes, transmits and decodes scene information that includes lighting information (which likely includes alpha/transparency information), and Macq encodes, transmits and decodes scene information including alpha, where alpha is merely adding additional information in the same process. Therefore it would have been obvious to combine Macq with Van Den Berghe to obtain the invention.
Regarding claim 4:
The method according to claim 1, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe further teaches wherein the server receives a command from the client to redraw the 3D object on the server and the command includes a position information of the client (Van Den Berghe [0052] When two users request a view on the same scene, re-use of the illumination information can avoid double processing and thus reduce the processing requirements.).
Regarding claim 5:
The method according to claim 1, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe further teaches wherein when the server receives a command from the client to redraw the 3D object, the server redraws the 3D object on the server based on a position information of the client included in the command, (Van Den Berghe [0052] When two users request a view on the same scene, re-use of the illumination information can avoid double processing and thus reduce the processing requirements.).
extracts the color information, the (Van Den Berghe [0012] Material information is information defining the optical characteristics of the (virtual) materials which are defining the 3D objects. Material information may comprise information which describes the interaction of the respective 3D objects with light. Material information may comprise at least one of colour, light absorption, diffusion, refraction and reflection parameters of objects or parts of objects. Material information may comprise a plurality of or all of these parameters.) and the geometry information from the redrawn 3D object, (Van Den Berghe [0011] Geometry information is information relating to the (possibly changing) geometry of objects within the 3D scene model.) simplifies the geometry information, (Van Den Berghe [0026] performing compression of the representation information;) and encodes a stream including the color information, the (Van Den Berghe [0056] The structure of the scene is than encoded using "traditional" mesh encoding techniques. According to embodiments of the present invention, the illumination information is encoded and combined with the mesh into the output stream.)
Van Den Berghe does not teach alpha information. In a related field of endeavor, Macq teaches:
extracting color information, alpha information (Macq [0010] [0011] extracting color information, depth information, and transparency information pertaining to groups of objects in respective object layers from respective ones of the first plurality of 2D video streams)
Therefore, it would have been obvious before the effective filing date of the claimed invention to use alpha as taught by Macq. The rationale for doing so would have been that it combines prior art elements according to known methods to yield predictable results where Van Den Berghe encodes, transmits and decodes scene information that includes lighting information (which likely includes alpha/transparency information), and Macq encodes, transmits and decodes scene information including alpha, where alpha is merely adding additional information in the same process. Therefore it would have been obvious to combine Macq with Van Den Berghe to obtain the invention.
Regarding claim 6:
The method according to claim 1, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe further teaches the color information, the alpha information, and the geometry information are generated based on an image obtained by an RGB camera (Van Den Berghe [0058] Ray tracing (see FIG. 3) is a technique for creating photo-realistic 2D views from a 3D scene representation. The basic principle is that traversal of light rays through the scene, and their physical properties (reflection, diffusion, . . . ) is simulated. The most used form is to simulate a ray coming from a camera, for each point in a viewport.) where it is a simulated rgb camera
Regarding claim 7 (independent):
A method for reproducing a 3D object generated based on a position information of a client on the client, the 3D object being present on a server, the method comprising: (Van Den Berghe [0024] According to a second aspect of the present invention, a server is disclosed for transmitting a 3D representation of a 3D scene model to a client device, the 3D scene model being information defining geometry and material information for a set of 3D objects and light sources, over a data communication network, wherein the 3D representation corresponds to a virtual viewing point for a user associated with the client device,).
receiving from the server, an encoded stream including color information, (Van Den Berghe [0012] Material information is information defining the optical characteristics of the (virtual) materials which are defining the 3D objects. Material information may comprise information which describes the interaction of the respective 3D objects with light. Material information may comprise at least one of colour, light absorption, diffusion, refraction and reflection parameters of objects or parts of objects. Material information may comprise a plurality of or all of these parameters.) and simplified geometry information (Van Den Berghe [0011] Geometry information is information relating to the (possibly changing) geometry of objects within the 3D scene model.) (Van Den Berghe [0026] performing compression of the representation information;) where the compression is a form of simplifying, since it reduces information for more efficient transmission.
of the 3D object (Van Den Berghe [0024] a server is disclosed for transmitting a 3D representation of a 3D scene model to a client device, the 3D scene model being information defining geometry and material information for a set of 3D objects)(Van Den Berghe [0056] The structure of the scene is than encoded using "traditional" mesh encoding techniques. According to embodiments of the present invention, the illumination information is encoded and combined with the mesh into the output stream.)
decoding the encoded stream and extracting the color information, the alpha information and the simplified geometry information from the decoded stream (Van Den Berghe [0056] For both cases we assume the encoding is based on a in-process encode/decode loop to create temporal references.)(Van Den Berghe [0027] forwarding the compressed representation information towards the client device for display).
reproducing a shape of the 3D object based on the simplified geometry information and projecting information on the reproduced shape of the 3D object to reconstruct the 3D object, the information resulting from combining the color information and the alpha information (Van Den Berghe [0023] Each client device can comprise or can be associated to a single display means, as for instance a screen for rendering 2D images/video.).
Van Den Berghe does not teach alpha information. In a related field of endeavor, Macq teaches:
receiving from the server, an encoded stream including color information, alpha information and geometry information of the 3D object decoding the encoded stream and extracting the color information, the alpha information and the geometry information from the decoded stream (Macq [0010] According to an aspect of the invention, there is provided a method for decoding a multiview video stream representing a plurality of viewpoints of a 3D scenery comprising objects, the method comprising for a first one of the viewpoints: extracting a first combined 2D video stream from the multiview video stream; deserializing the first combined 2D video stream into a first plurality of 2D video streams; [0011] extracting color information, depth information, and transparency information pertaining to groups of objects in respective object layers from respective ones of the first plurality of 2D video streams; using the respective color information, depth information, and transparency information pertaining to the groups of objects in the respective object layers to generate a first superimposed stream, the first superimposed stream being consistent with the first one of the viewpoints.)
reproducing a shape of the 3D object based on the geometry information and projecting information on the reproduced shape of the 3D object to reconstruct the 3D object, the information resulting from combining the color information and the alpha information (Macq [0021] a view synthesizer, operatively connected to the decoder, the visualizing agent being configured to use the respective color information, depth information, and transparency information pertaining to the objects in the respective object layers to generate a first superimposed stream, the first superimposed stream being consistent with the first one of the viewpoints.)(Macq [0054] The decoded camera view of the scene and the view containing the objects can then be easily blended by looking at the depth and alpha values pixel per pixel.)
Regarding claim 8:
The method according to claim 7, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe in view of Macq further teaches further including displaying the reconstructed 3D object on a display device (Van Den Berghe [0027] forwarding the compressed representation information towards the client device for display;).
Regarding claim 10 (independent):
The claim is a parallel version of claim 1. As such it is rejected under the same teachings.
Regarding claim 11 (independent):
The claim is a parallel version of claim 7. As such it is rejected under the same teachings.
Regarding claim 12 (independent):
The claim is a parallel version of claim 1. As such it is rejected under the same teachings.
Regarding claim 13 (independent):
The claim is a parallel version of claim 7. As such it is rejected under the same teachings.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Van Den Berghe U.S. Patent/PG Publication 20130321417 in view of Macq U.S. Patent/PG Publication 20130321576 and Taubin U.S. Patent/PG Publication 5506947.
Regarding claim 2:
The method according to claim 1, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe in view of Macq does not teach removing high frequency components. In a related field of endeavor, Taubin teaches:
wherein the simplifying the geometry information is to remove high frequency component from the geometry information (Taubin C5 L25-30 The invention applies to general and arbitrary polygonal curves, polyhedral surfaces, and more generally polytopes of arbitrary dimension. It produces a low-pass filter effect as a function of curvature.).
Therefore, it would have been obvious before the effective filing date of the claimed invention to remove high frequency components as taught by Taubin. The motivation for doing so would have been further reduction in the amount of data sent, where Van Den Berghe is already compressing and encrypting to reduce data. Therefore it would have been obvious to combine Taubin with Van Den Berghe in view of Macq to obtain the invention.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Van Den Berghe U.S. Patent/PG Publication 20130321417 in view of Macq U.S. Patent/PG Publication 20130321576 and Lee U.S. Patent/PG Publication 20200153885.
Regarding claim 9:
The method according to claim 8, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe in view of Macq does not teach glasses. In a related field of endeavor, Lee teaches:
wherein the display device is a smart glass or smart glasses (Lee [0558] The viewport information may be information about an area in 3D space currently viewed by the user through a device or an HMD.).
Therefore, it would have been obvious before the effective filing date of the claimed invention to have smart glasses as taught by Lee. The rationale for doing so would have been that it is a simple substitution of one known element for another to obtain predictable results where Van Den Berghe and Macq output to a display, and Lee outputs to a display which includes an HMD, where there are predictable results since they are merely a different type of display, but the output information is the same. Therefore it would have been obvious to combine Lee with Van Den Berghe in view of Macq to obtain the invention.
Claim(s) 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Van Den Berghe U.S. Patent/PG Publication 20130321417 in view of Macq U.S. Patent/PG Publication 20130321576 and Perry U.S. Patent/PG Publication 6954203.
Regarding claim 14:
(New) The method according to claim 1, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe in view of Macq does not teach converting. In a related field of endeavor, Perry teaches:
wherein the simplifying the geometry information comprises converting the geometry information to information of vertex of polygons (Perry C1 L20-30 Prior art hardware and software for graphic modeling systems are predominantly designed to store, manipulate, and render models that are represented by polygons, e.g., triangles.)(Perry C1 L35-50 A number of methods are known for transforming complex objects into level-of-detail (LOD) models having various resolutions. Lower resolution models are simpler expressions of the input model. LOD models represent the input object with varying accuracy and geometrical complexity. LOD methods typically process a subset of the available geometrical representations, e.g., points, polygons, Bezier patches, non-uniform rational B-splines (NURBS), volumes, implicit surfaces, and CSG models, and usually generate a triangle mesh that can be rendered by standard rendering engines.)(Perry C2 L5-15 Because arbitrary polygons can always be decomposed into triangles, a simple mesh consists only of triangles.)(Perry C2 L25-40 Vertex clustering, as described by Rossignac et al. in "Multi-resolution 3D approximations for rendering complex scenes," Modeling in Computer Graphics: Methods and Applications, pp. 455-465, 1993, is one decimation method that quickly simplifies mesh models. In its simplest form, vertex clustering partitions the initial geometry into cells for a given level of detail, and clusters all the vertices that lie within each cell together. New triangles are formed by replacing each vertex in the original mesh with its cluster in the new mesh. Degenerate triangles are removed where two or more of the vertices of an input triangle map to the same cluster.).
Therefore, it would have been obvious before the effective filing date of the claimed invention to convert as taught by Perry. The motivation for doing so would have been further reduction in the amount of data sent, where Van Den Berghe is already compressing and encrypting to reduce data. Therefore it would have been obvious to combine Perry with Van Den Berghe in view of Macq to obtain the invention.
Regarding claim 15:
(New) The method according to claim 14, has all of its limitations taught by Van Den Berghe in view of Macq. Van Den Berghe in view of Macq does not teach converting. In a related field of endeavor, Perry teaches:
wherein the converting the geometry information to the information of vertex of polygons comprises generating a group of triangles representing an actual geometry of the 3D object (Perry C1 L20-30 Prior art hardware and software for graphic modeling systems are predominantly designed to store, manipulate, and render models that are represented by polygons, e.g., triangles.)(Perry C1 L35-50 A number of methods are known for transforming complex objects into level-of-detail (LOD) models having various resolutions. Lower resolution models are simpler expressions of the input model. LOD models represent the input object with varying accuracy and geometrical complexity. LOD methods typically process a subset of the available geometrical representations, e.g., points, polygons, Bezier patches, non-uniform rational B-splines (NURBS), volumes, implicit surfaces, and CSG models, and usually generate a triangle mesh that can be rendered by standard rendering engines.)(Perry C2 L5-15 Because arbitrary polygons can always be decomposed into triangles, a simple mesh consists only of triangles.)(Perry C2 L25-40 Vertex clustering, as described by Rossignac et al. in "Multi-resolution 3D approximations for rendering complex scenes," Modeling in Computer Graphics: Methods and Applications, pp. 455-465, 1993, is one decimation method that quickly simplifies mesh models. In its simplest form, vertex clustering partitions the initial geometry into cells for a given level of detail, and clusters all the vertices that lie within each cell together. New triangles are formed by replacing each vertex in the original mesh with its cluster in the new mesh. Degenerate triangles are removed where two or more of the vertices of an input triangle map to the same cluster.).
Therefore, it would have been obvious before the effective filing date of the claimed invention to convert as taught by Perry. The motivation for doing so would have been further reduction in the amount of data sent, where Van Den Berghe is already compressing and encrypting to reduce data. Therefore it would have been obvious to combine Perry with Van Den Berghe in view of Macq to obtain the invention.
Conclusion
For the prior art referenced and the prior art considered pertinent to Applicant’s disclosure but not relied upon, see PTO-892 “Notice of References Cited”.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
/JASON A PRINGLE-PARKER/
Primary Examiner, Art Unit 2617
Prosecution Insights