DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Amendment filed on 7/17/2026.
Claims 1-16 are pending. Claims 1, 8, 15, 16 have been amended.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 4, 5, 6, 8, 11, 12, 13, 15, 16 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 3, 4, 6, 7, 8, 9, 11, 12 of 17/386,194 (now is US patent US 12051169 B2). Although the claims at issue are not identical, they are not patentably distinct from each other because.
Claim 1 is determined to be obvious in light of claim 1 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application claim 1
17/386,194 claim 1
1. A three-dimensional data processing method comprising: multiplexing pieces of data of a plurality of types including three-dimensional data to generate an output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data of the plurality of types and metadata;
1. A three-dimensional data multiplexing method, comprising: multiplexing pieces of data of a plurality of types including point cloud data to generate an output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data and metadata;
the three dimensional data including a plurality of three-dimensional points
and storing, in the metadata, type
information indicating a type of each of the pieces of data
of the plurality of types and information associating the pieces of data of the plurality of types, wherein the type information includes
position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types.
and storing, in the metadata included in the file configuration,
information indicating a type of each of the pieces of data
included in the output signal,
wherein the information includes, for each of the pieces of data,
position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating the piece of data.
Although the claims at issue are not identical, they are not patentably distinct from each other. The Claim 1 of the instant application and parent case 17/386,194 (now is US patent US 12051169 B2) discloses almost every single limitations the same, but the instant (child) applicant has broader scope, although the parent case has additional limitations “multiplexing,”, “storing, in the metadata included in the file configuration,
” but those can be fully anticipated by the instant application based on the similarity in between the Claim 1 from both applications. Regarding the instant application also include additional limitation like “type information indicating a type of each of the pieces of data of the plurality of types and information associating the pieces of data of the plurality of types,” which are not identical to Claim limitation of 17/386,194 (now is US patent US 12051169 B2). However, prior art Wang et al. (US 10,819,907 B2) teaches in Column 10, Line 7-15 of those limitations. Wang and Claim 1 of 17/386,194 are analogous since both of them are dealing with handling displaying three-dimensional object data in the augmented/virtual reality environment. Claim 1 of 17/386,194 provided a way of dealing with systems that generate and associate structured metadata with captured multi-type data so that a downstream device can correctly identify, process, and use each data type without having to re-query the originating device or data source. Wang provided a way of encapsulating multiple types of visual data (sphere visual tracks representing VR 3D content and region visual tracks) into an ISOBMFF file along with timed metadata tracks that carry type identification and inter-track association information, enabling downstream decoders to select and decode each visual data type appropriately. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate data of the plurality of types taught by Wang into modified invention of Claim 1 of 17/386,194 such that when dealing with the shared AR/VR environment, system will be able to allow user to use additional data with plurality of types which enhance the functionality and to provide more powerful data processing ability in AR/VR environment. Therefore, Claim 1 of 17/386,194 with Wang et al. (US 10,819,907 B2) discloses all limitations of instant application’s claim 1.
Claim 4 is determined to be obvious in light of Claim 2 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 4
17/386,194 Claim 2
4. The three-dimensional data processing method according to claim 1, wherein the type information further indicates, for each of the pieces of data of the plurality of types, (1) an encoding scheme applied to the piece of data, (2) a configuration of the piece of data, (3) a type of a sensor that generated the piece of data, or (4) a data format of the piece of data.
2. The three-dimensional data multiplexing method according to claim 1, wherein the information, for each of the pieces of data, indicates (1) an encoding scheme applied to the piece of data, (2) a configuration of the piece of data, (3) a type of a sensor that generated the piece of data, or (4) a data format of the piece of data.
Claim 5 is determined to be obvious in light of Claim 3 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 5
17/386,194 Claim 3
5. The three-dimensional data processing method according to claim 1, wherein the metadata includes synchronization information for synchronizing times of the pieces of data of the plurality of types included in the output signal.
3. The three-dimensional data multiplexing method according to claim 1, wherein the metadata includes synchronization information for synchronizing times of the pieces of data included in the output signal.
Claim 6 is determined to be obvious in light of Claim 4 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 6
17/386,194 Claim 4
6. The three-dimensional data multiplexing method according to claim 5, wherein the synchronization information indicates a difference in timestamps between the pieces of data of the plurality of types.
4. The three-dimensional data multiplexing method according to claim 3, wherein the synchronization information indicates a difference in timestamps between the pieces of data.
Claim 8 is determined to be obvious in light of claim 6 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application claim 8
17/386,194 claim 6
8.A three-dimensional data processing method comprising:
obtaining, from an output signal generated by multiplexing pieces of data of a plurality of types including three-dimensional data, type information indicating a type of each of the pieces of data of the plurality of types and information associating the pieces of data of the plurality of types,
the output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data of the plurality of types and metadata, the type information and the information being stored in the metadata;
6. A three-dimensional data demultiplexing method, comprising: obtaining, from an output signal generated by multiplexing pieces of data of a plurality of types including point cloud data, information indicating a type of each of the pieces of data included in the output signal,
the output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data and metadata, the information being stored in the metadata included in the file configuration; and
the three dimensional data including a lurality of three-dimensional points;
and obtaining the pieces of data of the plurality of types from the output signal, using the type information, wherein the type information includes position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types.
obtaining the pieces of data from the output signal, using the information, wherein the information indicates includes, for each of the pieces of data, position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating the piece of data.
Although the claims at issue are not identical, they are not patentably distinct from each other. The Claim 8 of the instant application and Claim 6 of parent case 17/386,194 (now is US patent US 12051169 B2) discloses almost every single limitations with the same reason set forth in the previous section for Claim 1 of instant application verse Claim 1 of parent application.
Claim 11 is determined to be obvious in light of Claim 7 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 11
17/386,194 Claim 7
11. The three-dimensional data processing method according to claim 8, wherein the type information further indicates, for each of the pieces of data of the plurality of types, (1) an encoding scheme applied to the piece of data, (2) a configuration of the piece of data, (3) a type of a sensor that generated the piece of data, or (4) a data format of the piece of data.
7. The three-dimensional data demultiplexing method according to claim 6, wherein the information, for each of the pieces of data, indicates (1) an encoding scheme applied to the piece of data, (2) a configuration of the piece of data, (3) a type of a sensor that generated the piece of data, or (4) a data format of the piece of data.
Claim 12 is determined to be obvious in light of Claim 8 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 12
17/386,194 Claim 7
12. The three-dimensional data processing method according to claim 8, wherein the metadata includes synchronization information for synchronizing times of the pieces of data of the plurality of types included in the output signal.
8. The three-dimensional data demultiplexing method according to claim 6, wherein the metadata includes synchronization information for synchronizing times of the pieces of data included in the output signal.
Claim 13 is determined to be obvious in light of Claim 9 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 13
17/386,194 Claim 9
13. The three-dimensional data multiplexing method according to claim 12, wherein the synchronization information indicates a difference in timestamps between the pieces of data of the plurality of types.
9. The three-dimensional data demultiplexing method according to claim 8, wherein the synchronization information indicates a difference in timestamps between the pieces of data.
Claim 15 is determined to be obvious in light of Claim 11 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 15
17/386,194 Claim 11
15. A three-dimensional data processing device comprising: a processor; and memory, wherein using the memory, the processor: multiplexes pieces of data of a plurality of types including three-dimensional data to generate an output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data of the plurality of types and metadata; and
11. A three-dimensional data multiplexing device, comprising: a processor; and memory, wherein using the memory, the processor: multiplexes pieces of data of a plurality of types including point cloud data to generate an output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data and metadata;
the three-dimensional data including a plurality of three-dimensional points;
stores, in the metadata, type information indicating a type of each of the pieces of data of the plurality of types and information associating the pieces of data of the plurality of types, wherein the type information includes position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types.
and stores, in the metadata included in the file configuration, information indicating a type of each of the pieces of data included in the output signal, and wherein the information indicates includes, for each of the pieces of data, position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating the piece of data.
Although the claims at issue are not identical, they are not patentably distinct from each other. The Claim 15 of the instant application and Claim 11 of parent case 17/386,194 (now is US patent US 12051169 B2) discloses almost every single limitations with the same reason set forth in the previous section for Claim 1 of instant application verse Claim 1 of parent application.
Claim 16 is determined to be obvious in light of Claim 11 of 17/386,194 (now is US patent US 12051169 B2) based on reasons below for having similar limitations.
Instant application Claim 16
17/386,194 Claim 12
16. A three-dimensional data processing device comprising: a processor; and memory, wherein using the memory, the processor: obtains, from an output signal generated by multiplexing pieces of data of a plurality of types including three-dimensional data, type information indicating a type of each of the pieces of data of the plurality of types and information associating the pieces of data of the plurality of types, the output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data of the plurality of types and metadata, the type information and the information being stored in the meta data; and obtains the pieces of data of the plurality of types from the output signal, using the type information, wherein the type information includes position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types.
12. A three-dimensional data demultiplexing device, comprising: a processor; and memory, wherein using the memory, the processor: obtains, from an output signal generated by multiplexing pieces of data of a plurality of types including point cloud data, information indicating a type of each of the pieces of data included in the output signal, the output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data and metadata, the information being stored in the metadata included in the file configuration; and obtains the pieces of data from the output signal, using the information, and wherein the information indicates includes, for each of the pieces of data, position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating the piece of data.
Although the claims at issue are not identical, they are not patentably distinct from each other. The Claim 16 of the instant application and Claim 12 of parent case 17/386,194 (now is US patent US 12051169 B2) discloses almost every single limitations with the same reason set forth in the previous section for Claim 1 of instant application verse Claim 1 of parent application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-12, 14-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 10,819,907 B2, hereinafter Wang) in view of Naito (US 20200166388 A1, hereinafter Naito) and further in view of Budagavi et al. (US 20180268570 A1, hereinafter "Budagavi").
Regarding Claim 15, Wang teaches a three-dimensional data processing device (Wang, Column 3, Line 47-49, "The VR system 100 includes two video processing apparatuses (e.g., a source electronic device 102 and a destination electronic device 104)"), comprising: a processor; and memory (Wang, Column 3, Line 49-51, "The source electronic device 102 includes a video capture device 112, a conversion circuit 114, a video encoder 116 <read on processor>, and a file encapsulation circuit 118 <read on memory>"), wherein using the memory, the processor: multiplexes pieces of data of a plurality of types including three-dimensional data to generate an output signal having a file configuration that is predetermined (Wang, Column 5, Line 7-11, "the file encapsulation circuit 118 encapsulates the part of the coded bitstream into one or more ISOBMFF <read on output signal having a file configuration that is predetermined> files F together with additional metadata Om, where a sphere visual track (i.e., VR 30 video track) <read on three-dimensional data, first type> is included in the ISOBMFF files F, and spherical region visual tracks (i.e., region 30 video track) <read on a second type of data> are included in the ISOBMFF files F <read on multiplexing pieces of data of a plurality of types>"), the file configuration including the multiplexed pieces of data
of the plurality of types and metadata (Wang, Column 4, Line 63-66, "The metadata Dm may include metadata directly placed in a visual track by using a track metadata box, and/or may include metadata placed in a metadata track (e.g., timed metadata track) associated with the visual track"), [[the three-dimensional data including a plurality of three-dimensional points;]] and stores, in the metadata, type information indicating a type of each of the pieces of data of the plurality of types (Wang, Column 10, Line 7-15, "the timed metadata track m signals that the referenced (main) sphere visual track v contains a spherical region defined by its spherical coordinates (e.g., center_yaw and center pitch) and sizes (e.g., hor_range and ver_range), identified by a unique spherical region identifier, and contributed by ( or carried in) the region visual tracks v1 , . .. , vn identified or referenced by track identifiers <read on type information indicating a type of each of the pieces of data of the plurality of types>"), and information associating the pieces of data of the plurality of types (Wang, Column 5, Line 63-67, "the latest OMAF standard draft provides a spherical region definition in the ISO Base Media File Format (ISOBMFF), which can be carried in a timed metadata track, and associated with the sphere visual track, by means of a 'cdsc' (content describes) track reference <read on information associating the pieces of data of the plurality of types>"), [[ wherein the type information includes position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types]].
But Wang does not explicitly disclose the type information includes position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types.
However, Naito teaches the type information includes position information of a
sensor (Naito, Paragraph [0020], "the sub-sensor may be a temperature sensor, a GPS (Global Positioning System) sensor <read on position information of a sensor>, an acceleration sensor, an air pressure sensor, a magnetic field sensor, a gyro sensor, or the like"), angle information of the sensor (Naito, Paragraph [0014], "the attribute relating to the situation in which the sensor is installed is, for example, the angle at which the sensor is installed <read on angle information of the sensor>, the temperature around the sensor, the distance between the sensor and an observation target, or the like"), or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types (Naito, Paragraph [0012], "a metadata generating portion configured to generate dynamic metadata indicating an attribute of the sensor at a point in time when the sensing data is obtained <read on time information of the sensor at a time of generating at least a piece of data>, the attribute relating to a form of use of the sensor that may dynamically change over time"), and further that all such metadata is managed in association with the sensing data (Naito, Paragraph, "a metadata managing portion configured to manage the generated dynamic metadata in association with the sensing data").
Naito and Wang are analogous since both are dealing with systems that generate and associate structured metadata with captured multi-type data so that a downstream device can correctly identify, process, and use each data type without having to re-query the originating device or data source. Wang provided a way of encapsulating multiple types of visual data (sphere visual tracks representing VR 3D content and region visual tracks) into an ISOBMFF file along with timed metadata tracks that carry type identification and inter-track association information, enabling downstream decoders to select and decode each visual data type appropriately. Naito provided a way of generating dynamic metadata that captures the sensor's operational state at the exact point in time the sensing data is acquired - including the installation angle of the sensor, the GPS-derived position of the sensor, and the timestamp of acquisition - and associating that metadata with the sensing data so that users can check the form of use without accessing the sensor again. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the dynamic sensor-state metadata
taught by Naito into modified invention of Wang such that in addition to signaling the visual track type and inter-track associations, the metadata would also carry position, angle, and time information of the sensor that generated each data type, enabling downstream devices to assess data reliability and context without requiring additional sensor queries. The motivation is to reduce the time and effort required to check the form of use of a sensor when using sensing data.
The combination does not explicitly disclose the three-dimensional data including a plurality of three-dimensional points.
However, Budagavi teaches the three-dimensional data including a plurality of three-dimensional points (Budagavi, Paragraph [0004], "Point clouds and meshes are a set of three-dimensional (3-D) points <read on a plurality of three-dimensional points> that represent a model of a surface of an object or a scene"; Paragraph [0035], "a point cloud is a collection of data points defined by a coordinate system. For example, in a 3-D Cartesian coordinate system, each point of a point cloud is identified by three coordinates, that of X, Y, and Z <read on three-dimensional points>. When each point is identified by the three coordinates, a precise location in 3-D space is identified, relative to an origin point where the X, Y, and Z axes intersect").
Budagavi and Wang/Naito are analogous since all are dealing with systems that decompose captured three-dimensional content into a plurality of typed data pieces, package those pieces into a multiplexed output signal, and signal descriptive metadata that allows a downstream device to identify and reconstruct the content. Wang (as modified by Naito) provided a way of encapsulating a plurality of typed visual tracks — with sensor position/angle/time metadata. Budagavi provided a way of representing three-dimensional data as a set of three-dimensional points (a point cloud), decomposing that point cloud into a geometry frame and one or more attribute frames, and multiplexing those typed frames together into an output bitstream. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the point-cloud three-dimensional data representation taught by Budagavi into the modified invention of Wang, Naito and Budagavi such that the three-dimensional data being multiplexed into the file configuration comprises a plurality of three-dimensional points, so that the same file-level multiplexing framework and sensor-state metadata scheme of Wang/Naito can be used to package and deliver point-cloud content. The motivation is that point clouds are common representations of three-dimensional data used in a wide range of applications, as discussed by Budagavi in Paragraph [0004].
Regarding Claim 16, Wang teaches a three-dimensional data processing device (Wang, Column 3, Line 47-49, "The VR system 100 includes two video processing apparatuses (e.g., a source electronic device 102 and a destination electronic device 104)"), comprising: a processor; and (Wang, Column 3, Line 49-51 , "The source electronic device 102 includes a video capture device 112, a conversion circuit 114, a video encoder 116 <read on processor>, and a file encapsulation circuit 118 <read on memory>"), wherein using the memory, the processor: obtains, from an output signal generated by multiplexing pieces of data of a plurality of types including three-dimensional data, type information indicating a type of each of the pieces of data of the plurality of types and information associating the pieces of data of the plurality of types, the output signal having a file configuration that is predetermined, the file configuration including the multiplexed pieces of data of the plurality of types and metadata, the type information and the information being stored in the meta data (Wang, Column 5, Line 38-42, "After receiving the ISOBMFF files F', the file decapsulation circuit 120 decapsulates the ISOBMFF files F' into a part of a coded bitstream (which includes an encoded data E'v of a picture to be reconstructed), and extracts/parses the accompanying metadata D'm <read on type information indicating a type of each of the pieces of data of the plurality of types and information associating the pieces of data of the plurality of types> from the ISOBMFF files F"') and (Wang, Column 9, Line. 51-56, "the timed metadata track is decapsulated from the ISOBMFF files F' to obtain extracted/parsed metadata indicating that the associated region visual track <read on pieces of data of one type> contributes to only a single spherical region carried in one sphere visual track <read on information associating the pieces of data of the plurality of types, the type information and the information being stored in the meta data>"), [[the three-dimensional data including a plurality of three-dimensional points;]] and obtains the pieces of data of the plurality of types from the output signal, using the type information (Wang, Column 13, Line 17-20, "the video decoder 122 accesses and decodes encoded data of visual track(s) of a selected type <read on obtains the pieces of data of the plurality of types from the output signal> according to at
least the metadata signaled by the timed metadata track <read on using the type
information>"), [[the type information includes position information of a sensor, angle
information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types]].
But Wang does not explicitly disclose the type information includes position information of a sensor, angle information of the sensor, or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types.
However, Naito teaches the type information includes position information of a sensor (Naito, Paragraph [0020], "the sub-sensor may be a temperature sensor, a GPS (Global Positioning System) sensor <read on position information of a sensor>, an acceleration sensor, an air pressure sensor, a magnetic field sensor, a gyro sensor, or the like"), angle information of the sensor (Naito, Paragraph [0014], "the attribute relating to the situation in which the sensor is installed is, for example, the angle at which the sensor is installed <read on angle information of the sensor>, the temperature around the sensor, the distance between the sensor and an observation target, or the like"), or time information of the sensor at a time of generating at least a piece of data of one type among the pieces of data of the plurality of types (Naito, Paragraph [0011], "a metadata generating portion configured to generate dynamic metadata indicating an attribute of the sensor at a point in time
when the sensing data is obtained <read on time information of the sensor at a time of generating at least a piece of data>"), and further that the receiving device (user terminal) obtains such sensor-state type information from the metadata associated with the sensing data (Naito, Paragraph [0049], "the user terminal 3 can check, when using sensing data, the form of use of the first sensor 101 at the point in time when the sensing data to be used is obtained, by referencing the associated dynamic metadata. That is to say, the user terminal 3 can omit processing to access the first sensor 101 or make an inquiry to the sensor management unit 1").
Naito and Wang are analogous since both are dealing with end-to-end systems in which metadata describing the type and context of captured data is generated and stored at the source side and subsequently retrieved and consumed at the destination side to guide correct data decoding and processing. Wang provided a way of decapsulating ISOBMFF files at the destination electronic device to extract timed metadata that signals the type of each visual track (sphere vs. region) and their
inter-track associations, which the video decoder then uses to access and decode the correct data types. Naito provided a way of supplying the receiving user terminal, together with sensing data, dynamic metadata that records the sensor's position, angle, and time of data acquisition, allowing the user terminal to retrieve sensor context information from the stored metadata without re-querying the original sensor.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the sensor-state type metadata taught by Naito into modified invention of Wang such that when the destination electronic device decapsulates the file and obtains the pieces of data of the plurality of types using the type information, it also retrieves the position, angle, and time information of the sensor that generated each data type from the same metadata, providing richer context for data decoding and rendering without additional sensor queries. The motivation is to reduce the time and effort required to check the form of use of a sensor when using sensing data.
The combination of Wang and Naito does not explicitly disclose But Budagavi teaches the three-dimensional data including a plurality of three-dimensional points (Budagavi, Paragraph [0004], "Point clouds and meshes are a set of three-dimensional (3-D) points <read on a plurality of three-dimensional points> that represent a model of a surface of an object or a scene"; Paragraph [0035], "a point cloud is a collection of data points defined by a coordinate system. For example, in a 3-D Cartesian coordinate system, each point of a point cloud is identified by three coordinates, that of X, Y, and Z <read on three-dimensional points>").
Budagavi and Wang/Naito are analogous since all are dealing with end-to-end systems in which three-dimensional content is decomposed into a plurality of typed pieces at a source side, packaged with descriptive metadata into a multiplexed output signal, and subsequently retrieved and reconstructed at a destination side using that metadata. Wang (as modified by Naito) provided a way for the destination electronic device to decapsulate an ISOBMFF file and use the type/association metadata (including sensor position/angle/time) to obtain the correct pieces of visual data. Budagavi provided a way of representing three-dimensional data at the source as a set of three-dimensional points (a point cloud) that is decomposed and multiplexed into a bitstream, and correspondingly demultiplexing and reconstructing the point cloud at the destination decoder. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the point-cloud three-dimensional data representation taught by Budagavi into the modified invention of Wang, Naito and Budagavi such that the three-dimensional data carried by the output signal that the destination device obtains and demultiplexes comprises a plurality of three-dimensional points, so that Wang's file-level decapsulation and Naito's sensor-state metadata retrieval scheme can be used to obtain and reconstruct point-cloud content at the destination. The motivation is that point clouds are common representations of three-dimensional data used in a wide range of applications, as discussed by Budagavi in Paragraph [0004].
Regarding Claim 1, it recites limitations similar in scope to the limitations of Claim 15 but as a method and the combination of Wang, Naito and Budagavi teaches all the limitations as of Claim 15. Therefore is rejected under the same rationale.
Regarding Claim 2, The combination of Wang, Naito and Budagavi teaches the invention in Claim 1.
The combination further teaches presenting the three-dimensional data and any of the pieces of data of the plurality of types, based on the metadata (Wang, Column 9, Line 63-Column 10, Line 1, "the video decoder 122 decodes the part of the coded bitstream according to at least the extracted/parsed metadata <read on based on the metadata>. Specifically, the video decoder 122 accesses and decodes encoded data of visual track(s) of a selected type according to at least the metadata signaled by the timed metadata track"; Column 13, Line 20-22, "the graphic rendering circuit 124 drives the display screen 126 according to a decoding result of the part of the coded bitstream; Column 5, Line 48-50, “a decoding circuit arranged to decode the part of the coded bitstream according to the extracted/parsed metadata D'm”).
Regarding Claim 3, The combination of Wang, Naito and Budagavi teaches the invention in Claim 1 .
The combination further teaches wherein one of the pieces of data of the plurality of types is two-dimensional image data obtained based on the three-dimensional data (Wang, Column 4, Line 5-8, "the conversion circuit 114 generates a picture Dv on a two-dimensional (2D) <read on two-dimensional image data> plane by applying visual pre-processing to the VR content Bv in a 3D space <read on obtained based on the three-dimensional data>"; Column 7, Line 58-61, "the present invention provides methods for signaling associations of sphere visual tracks and their spherical region carrying or contributing (spherical or 2D projected) region visual tracks in ISOBMFF").
Regarding Claim 4, The combination of Wang, Naito and Budagavi teaches the invention in Claim 1.
The combination further teaches wherein the type information further indicates, for each of the pieces of data of the plurality of types, (1) an encoding scheme applied to the piece of data (Wang, Column 4, Line 54-57, "the content of the same tile can be coded in different qualities and bit rates to generate variants of the encoded tile, or can be coded in different codecs and protection schemes <read on encoding scheme applied to the piece of data> to generate variants of the encoded tile"), (2) a configuration of the piece of data (Wang, Column 4, Line 67-Coumn 5, Line 1-3, "the signaled metadata Dm may include projection/orientation information, packing information <read on configuration of the piece of data>, sub-picture composition information, region of interest (ROI) information, viewport information"), [[ (3) a type of a sensor that generated the piece of data, ]] or (4) a data format of the piece of data (Wang, Column 4, Line 18-20, "The 360° VR projection format may be an equirectangular projection (ERP) format or a cubemap projection (CMP) format <read on data format of the piece of data>").
Wang does not explicitly disclose (3) a type of a sensor that generated the piece of data.
However, Naito teaches (3) a type of a sensor that generated the piece of data (Naito, Paragraph [0056], "the type of the first sensor 101 may be, for example, an image sensor, an infrared sensor <read on a type of a sensor that generated the piece of data>, a sound sensor, a light sensor, a pressure sensor, an air pressure sensor, a temperature sensor, or the like"; [0018], "the attribute relating to device information regarding the sensor may include, for example, the name of the sensor, a description of the sensor, or the like <read on a type of a sensor that generated the piece of data>").
Naito and Wang are analogous since both are dealing with end-to-end systems in which metadata describing the type and context of captured data is generated and stored at the source side and subsequently retrieved and consumed at the destination side to guide correct data decoding and processing. Wang provided a way of decapsulating ISOBMFF files at the destination electronic device to extract timed metadata that signals the type of each visual track (sphere vs. region) and their
inter-track associations, which the video decoder then uses to access and decode the correct data types. Naito provided a way of supplying the receiving user terminal, together with sensing data by using type of a sensor that generated the piece of data. Therefore, It would have been obvious to one of ordinary skill in the art at the time of the invention to incorporate the sensor type information taught by Naito into Wang's ISOBMFF metadata framework, as both references are directed to managing metadata that characterizes the origin and properties of data types captured by sensing devices. Incorporating Naito's sensor type information into Wang's timed metadata tracks would allow downstream decoders to efficiently identify and distinguish data types by their originating sensor (e.g., image sensor vs. infrared sensor), thereby enhancing metadata utility and reducing the time and effort required to check the form of use of the sensor when using sensing data .
Regarding Claim 5, The combination of Wang, Naito and Budagavi teaches the invention in Claim 1.
The combination further teaches wherein the metadata includes synchronization information for synchronizing times of the pieces of data of the plurality of types included in the output signal (Wang, Column 4, Line 63-66, "the metadata Om may include metadata directly placed in a visual track by using a track metadata box, and/or may include metadata placed in a metadata track (e.g., timed metadata track) <read on synchronization information for synchronizing times> associated with the visual track"; Column 5, Line 42-45, "If there are no transmission errors and decapsulation errors, the metadata D'm extracted/parsed from the ISOBMFF files F' should be the same as the metadata Dm added to the ISOBMFF files F"; it is noted ISOBMFF structure, a "timed" metadata track inherently carries per-sample timestamps that temporally align the metadata track with the corresponding sphere visual track and region visual tracks within the shared movie timeline, thereby providing synchronization of the times of the pieces of data of the plurality of types included in the output file).
Regarding Claim 7, The combination of Wang, Naito and Budagavi teaches the invention in Claim 1.
The combination further teaches the type information further including time information of each of the pieces of data of the plurality of types at a time of generating the pieces of data of the plurality of types (Naito, Paragraph [0012], "a metadata generating portion configured to generate dynamic metadata indicating an attribute of the sensor at a point in time when the sensing data is obtained, the attribute relating to a form of use of the sensor that may dynamically change over time") and (Naito, [0043], "the sensor management unit 1 generates, as the dynamic metadata, information regarding the form of use of the first sensor 101 at the point in time when the sensing data is obtained <read on time information at a time of generating the pieces of data>", [0048], "and manages the generated dynamic metadata in association with this sensing data").
Naito and Wang are analogous since both are dealing with end-to-end systems in which metadata describing the type and context of captured data is generated and stored at the source side and subsequently retrieved and consumed at the destination side to guide correct data decoding and processing. Wang provided a way of decapsulating ISOBMFF files at the destination electronic device to extract timed metadata that signals the type of each visual track (sphere vs. region) and their inter-track associations, which the video decoder then uses to access and decode the correct data types. Naito provided a way of supplying the receiving user terminal, together with sensing data by using time information of each of the pieces of data. Therefore, It would have been obvious to one of ordinary skill in the art to incorporate Naito's teaching of time-stamped sensor state metadata into Wang's metadata signaling framework, as both references are directed to managing metadata that characterizes data captured by sensors/capture devices and stored/transmitted within a structured file format. A person of ordinary skill would recognize that incorporating time-of-generation information into each data type's metadata, as taught by Naito, would enable receiving devices in Wang's system to assess the temporal context of each captured data type and judge its reliability, thereby further reducing the time and effort required to check the form of use of the sensor when using sensing data, as discussed by Naito in ,r.
Regarding Claim 8, it recites limitations similar in scope to the limitations of Claim 16 but as a method and the combination of Wang, Naito and Budagavi teaches all the limitations as of Claim 16. Therefore is rejected under the same rationale.
Regarding Claim 9, it recites limitations similar in scope to the limitations of Claim 2 and therefore is rejected under the same rationale.
Regarding Claim 10, it recites limitations similar in scope to the limitations of Claim 3 and therefore is rejected under the same rationale.
Regarding Claim 11, it recites limitations similar in scope to the limitations of Claim 4 and therefore is rejected under the same rationale.
Regarding Claim 12, it recites limitations similar in scope to the limitations of Claim 5 and therefore is rejected under the same rationale.
Regarding Claim 14, it recites limitations similar in scope to the limitations of Claim 7 and therefore is rejected under the same rationale.
Claim(s) 6, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 10,819,907 82, hereinafter Wang) in view of Naito (US 20200166388 A1, hereinafter Naito) as applied to Claims 5, 12 above respectively and further in view of Kim et al. (US 9900577 B2, hereinafter "Kim")
Regarding Claim 6, the combination of Wang, Naito and Budagavi teaches the invention in Claim 5.
The combination does not explicitly disclose but Kim teaches wherein the synchronization information indicates a difference in timestamps between the pieces of data of the plurality of types (Kim, Column 8, Fig. 5, Timestamp Offset Mode, Lines 48-51, "The synchronization information generation unit 203 may generate timestamps of AUs of left and right images to be output at the same particular time for a 3DTV service as synchronization information"; Column 21, Lines 22-25, "the synchronization information is a timestamp offset related to a difference
between a PTS of the AU of the left image and an RTP timestamp of the AU of the additional view content stream at a synchronization time"; Column 22, Lines 63-Column 23, Line 1-3, "an absolute value and a sign of a timestamp offset, which is a difference between 65 PTS_Sync of the AU as a synchronization reference in the main content stream in accordance with the ATSC main broadcast standard and RTP Sync of the AU as a synchronization reference in the additional view content stream in accordance with the MDTV broadcast standard, are used as
synchronization information"; Column 8, Lines 60-63, "The synchronization information multiplexing unit 204 may insert the generated synchronization information into a main content stream or a additional view content stream as a descriptor transmitted periodically")
Kim and Wang/Naito are analogous since both of them are dealing with multiplexing and synchronizing multiple streams of heterogeneous data in a file/stream format where synchronization information is stored in the metadata or signaling information of the multiplexed output. Wang provided a way of multiplexing pieces of three-dimensional data together with other data types in a predetermined file configuration, and storing synchronization information in the metadata of the multiplexed output signal to enable temporal alignment of the different data types. Kim provided a way of expressing the synchronization information as a timestamp offset - specifically a difference between the timestamps of two different data streams - rather than storing the absolute timestamps of both streams as a pair, explicitly teaching that "an amount of data to be transmitted may be less than in the timestamp pairing mode." Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention was made to incorporate the timestamp difference representation for synchronization information taught by Kim into the modified invention of Wang such that the synchronization information stored in the metadata of the multiplexed output signal would indicate the difference in timestamps between the pieces of data of the plurality of, rather than storing their absolute timestamps independently, in order to reduce the data amount of the output signal while maintaining accurate temporal synchronization between the multiplexed data streams.
Regarding Claim 13, it recites limitations similar in scope to the limitations of Claim 6 and therefore is rejected under the same rationale.
Response to Arguments
The rejection of Claims 8-14, 16 under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph are withdrawn in view of Applicant’s amendment to the independent Claims 8, 16.
Applicant’s arguments with respect to claim 1, 8, 15, 16, filed on 7/17/2026, with respect to rejection under 35 USC § 103 have been considered but are moot in view of the new ground(s) of rejection. It has now been taught by the combination of prior arts Wang, Naito and Budagavi.
In regard to Claims 2-7, 9-14, they directly/indirectly depends on independent Claim 1, 8,respectively. Applicant does not argue anything other than the independent claim 1, 8,. The limitations in those claims in conjunction with combination previously established as explained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20190108655 A1 METHOD AND APPARATUS FOR ENCODING A POINT CLOUD REPRESENTING THREE-DIMENSIONAL OBJECTS
US 20190080483 A1 Point Cloud Compression
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights