DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: an extraction unit that extracts a sample, a decoding unit that decodes the extracted sample and a construction unit that constructs a point cloud in claim 1; an encoding unit that encodes a sample to generate encoded data, a bit stream generation unit that generates a bit stream including the encoded data, and a file generation unit that generates a file for storing the bit stream in claim 11.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Denoual et al. (WO 2023/111214, hereinafter Denoual).
Regarding claim 1, Denoual teaches an information processing device (media player 130, fig. 1) comprising:
an extraction unit (this element is interpreted under 35 USC 112(f) as a processor; De-encapsulation module (parsing module) 132, fig. 1; page 5 lines 9-13: At least parts of the methods according to the disclosure may be computer implemented. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system") that extracts, based on first information on a display timing of a sub-frame stored as metadata in a file (page 20 lines 19-24: Figure 6 illustrates an example of steps of a parsing process according to some embodiments of the invention, making it possible to identify metadata structures providing ATI in G-PCC tracks and to extract a subset of data corresponding to a G-PCC sub-frame or a set of G-PCC sub-frames. The ATI or sub-frame description also allows extraction of a sample containing specific sub-frames, or frame(s) that have been combined or aggregated (as illustrated with reference 950 in Figure 9); page 36 line 34 - page 37 line 2: In a variant of the described embodiments, the metadata structure dedicated to sub-frame configuration information may further describe the characteristics of the timing associated with the points of a sample. It can be for example a number of different timing values within a sample. For instance, each timing value may correspond to a subframe within the sample), a sample including the sub-frame from a bit stream stored in the file in time for the display timing of the sub-frame (page 6 paragraph 3: Figure 6 illustrates an example of steps of a parsing process according to some embodiments of the invention, making it possible to identify metadata structures providing ATI in G-PCC tracks and to extract a subset of data corresponding to a G-PCC sub-frame or a set of G-PCC sub-frames; page 6 last two lines – page 7 line 9: These additional metadata may comprise additional timing information (ATI) such as acquisition time or rendering time information in the description of PCC tracks. It is an additional timing information to classical sample timing information like decoding time or composition time contained in the sample description boxes. This ATI may be associated with subsamples to provide PCC sub-frames (e.g. from MPEG-I Part-9) description in G-PCC tracks. ATI may also be called “sub-frame descriptor”, “sub-frame description”, or “sub-frame description structure”. Such additional information allows identifying or accessing or extracting data units corresponding to a sub-frame, i.e. data units within a sample corresponding to a given acquisition or rendering time (or consecutive range of acquisition or rendering time) or to a given shot direction of the sensor producing the point cloud; page 9 lines 6-10: De-encapsulation module (or parsing module) 132 then extracts the, or a subset of, encapsulated point cloud data … The extracted point cloud data may result in a bit-stream such as a bit-stream complying with MPEG-I Part-9; page 21 lines 1-7: Next, the parser (or de-encapsulation module, such as de-encapsulation module 132 in Figure 1) is initialized (step 605). The initialization may be carried out by parsing the top-level boxes of the media file, for example the ‘moov’ box, the ‘trak’ boxes, and sample description boxes … the parser (or the deencapsulation module) reads sample description from the metadata part of the received media file to locate corresponding media data in the media data box of the received media file. After having read the sample, the parser (or de-encapsulation module) looks for additional timing information or sub-frame description (test 615). Such a test may consist in looking for a metadata structure providing ATI; page 21 lines 9-31: If additional timing information is identified, the additional timing information is read (step 620). The parser is then able to determine the presence of a (one or more) partial representation of a point cloud for the frame. Depending on the user settings or applications needs, it may select a partial representation of the point cloud frame if it contains a point cloud representation with appropriate timing precision. In a variant, the parser may select one (or more) partial representation of a point cloud frame to read or to extract (step 625). For G-PCC, it may correspond to one or more G-PCC sub-frames within a G-PCC frame. Based on the ATI, the parser can retrieve the data units corresponding to the selected partial representation of a point cloud frame (step 630), the retrieval may be based on timing information, on spatial information corresponding to a partial representation, or a laser shot direction, any parameter conveyed in the ATI);
a decoding unit (this element is interpreted under 35 USC 112(f) as a processor; decompression module 136, fig. 1; page 5 lines 9-13: At least parts of the methods according to the disclosure may be computer implemented. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system") that decodes the extracted sample (decompression unit takes a compressed point cloud data being encapsulated as one or more samples as input and generates point cloud data, and therefore, it inherently decompresses (decodes) the input point cloud data; page 8 line 34 – page 9 line2: Media player 130 may also contain a decompression module 136 taking as input a bit-stream representing compressed point cloud data, for example a bit-stream complying with MPEG-I Part-9, and generating point cloud data (or volumetric data) for rendering or analysis; page 20 lines 31-34: When the media player contains a decoder (or a decompression module, (e.g. decompression module 136 in Figure 1), the decoder may also be initialized during this step, for example using decoder configuration information from the sample description (e.g. G-PCC configuration box ‘gpcC’); page 21 line 32 – page 22 line 3: Next, the retrieved data units are transmitted to a decoder, when they are compressed, to a display, or to an application using the point cloud data (step 635). If no additional timing information is identified, the sample data are read (step 640) and transmitted to a decoder, when they are compressed, to a display, or to an application using the point cloud data (step 635); page 37 lines 8-18: In yet another embodiment, the metadata structure dedicated to sub-frame configuration information may indicate whether the encoding of the samples allows accessing to a partial representation of the point cloud frame that is associated with a particular value of timing. Typically, it can be specified using an additional parameter, for example named subframe_accessible_type parameter. When set to 0, no access to a partial representation is possible. When set to 1 , access to a partial representation is possible (but may require to decode more units than the units associated with this partial representation. This may require parser or reader to check possible dependencies between sub-frames or partial representations, for example indicated as ATI. When set to 2, access to and extraction of a partial representation are possible (the partial representation is decodable without data units from other partial representations); page 67 lines 10-18: decoding a sample); and
a construction unit (this element is interpreted under 35 USC 112(f) as a processor; decompression module 136, fig. 1; page 5 lines 9-13: At least parts of the methods according to the disclosure may be computer implemented. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system") that constructs a point cloud for the sub-frame based on the first information in time for the display timing of the sub-frame included in the decoded sample (page 8 line 34 – page 9 line2: Media player 130 may also contain a decompression module 136 taking as input a bit-stream representing compressed point cloud data, for example a bit-stream complying with MPEG-I Part-9, and generating point cloud data (or volumetric data) for rendering or analysis; page 9 lines 10-13: In such a case, the bit-stream is provided to a decompression module (an external decompression module or an internal decompression module, e.g. internal decompression module 136) for the reconstruction of the point cloud data 154 for usage by a user or application (for example visualization or analysis)).
Denoual teaches a decompression module that performs functions similar to that of a decoding unit and a construction unit, although the decoding unit and the construction unit are claimed as two separated units of the information processing device.
However, in Nerwin v. Erlichman, 168 USPQ 177, 179 (PTO Bd. Of Int. 1969), it has been noted “the mere fact that a given structure is integral does not preclude its consisting of various elements”. Therefore, it would have been obvious to one of ordinary skilled in the art at the time of present invention to separate decompression module of Denoual into two separate units to function as a decoding unit and a construction unit because using separate dedicated hardware units for specific functions will result in higher efficiency by freeing up resources. Also, See In re Dulberg, 289 F.2d 522, 523, 129 USPQ 348, 349 (CCPA 1961).
Regarding claim 11, Denoual teaches an information processing device (file writer 100, fig. 1) comprising:
an encoding unit (this element is interpreted under 35 USC 112(f) as a processor; compression module 106, fig. 1; page 5 lines 9-13: At least parts of the methods according to the disclosure may be computer implemented. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system") that encodes as a sample a combined frame in which a plurality of frames are combined as sub-frames to generate encoded data (page 7 line 34 – page 8 line 2: Media file writer 100 may also contain a compression module such as compression module 106 to compress the input point cloud data into a compressed bit-stream, for example using a point cloud compression algorithm like the one described in MPEG-I Part-9; page 10 lines 33-34: It is also to be noted that for compressed point cloud data, several frames may be packed or combined into one combined or aggregated frame; page 11 lines 21-27: It is to be noted that when combined or aggregated frames are encoded as a single frame, each frame that is part of the combination or aggregation may be considered as a sub-frame or as a G-PCC sub-frame when compressed with MPEG-I Part-9. Several points may share the same timing information or each point may have its own timing information. Timing information may be incremental from one point to another, following a scanning path of the 3D sensor. The timing increments may be fixed or variable depending on the 3D sensor; page 11 line 31 - page 12 line 12: In case the media file writer also contains a compression module, for example compression module 106 in Figure 1 , the configuration step may comprise setting parameters for the encoder: for example setting the display frame rate, setting an acquisition frame rate, setting a maximum number of sub-frames allowed in a frame, setting the number of sub-frames per frame, setting whether the number of sub-frames per frame is fixed or not, setting whether the sampling rate or time difference between sub-frames is constant or variable or setting parameters of the 3D sensor like field of view, angular resolution, description of the scanning path, etc. The compression module may encode these configuration parameters, for example as additional syntax elements of a G-PCC bit-stream, for example in the Sequence Parameter Set, in a volumetric usage information unit with a dedicated TLV type, or as supplemental enhancement information message, also with a dedicated TLV type. When the point cloud data are received by the encapsulation module as a G-PCC bit-stream (e.g. a bit-stream generated by the compression module 106 in Figure 1 , or by an external compression module), the configuration of the encapsulation module may use information from the parameter sets of the bit-stream or supplemental information associated with the bitstream (sometimes called SEI (Supplemental Enhancement Information) messages); page 43 lines 26-29: Figure 12 illustrates an example of steps to be carried out during data compression for generating description information to be encapsulated and/or to be used during encapsulation. For the sake of illustration, such steps may be carried out in compression module 106 in Figure 1; page 45 lines 10-12: the compressed data and the generated items of information describing the attribute values are encoded (step 1225); page 61 lines 23-28: One example of use is the case of combined frames, as illustrated with reference 950 in Figure 9: several frames (e.g., frame 1 and frame 2), captured by a 3D sensor, are aggregated into a combined frame for (geometry) compression efficiency. The compression may be done with MPEG-I Part-9 to produce a G-PCC bitstream; page 61 line 32 – page 32 line 2: The encoder, when using a frame number attribute, may change the order between captured frames (denoted sub-frames) before aggregating them into combined frames (denoted frame or G-PCC frame). For example, a sub-frame captured at time 40ms may be encoded in a first frame with decoding time t while a sub- frame captured earlier may be encoded in a second frame with decoding time t+delta (delta being a positive time period));
a bit stream generation unit (this element is interpreted under 35 USC 112(f) as a processor; compression module 106, fig. 1; page 5 lines 9-13: At least parts of the methods according to the disclosure may be computer implemented. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system") that generates a bit stream including the encoded data and generates first information on display timings of the sub-frames based on the bit stream (page 12 lines 3-12: The compression module may encode these configuration parameters, for example as additional syntax elements of a G-PCC bit-stream, for example in the Sequence Parameter Set, in a volumetric usage information unit with a dedicated TLV type, or as supplemental enhancement information message, also with a dedicated TLV type. When the point cloud data are received by the encapsulation module as a G-PCC bit-stream (e.g. a bit-stream generated by the compression module 106 in Figure 1 , or by an external compression module), the configuration of the encapsulation module may use information from the parameter sets of the bit-stream or supplemental information associated with the bitstream (sometimes called SEI (Supplemental Enhancement Information) messages); page 12 lines 16-23: Further to the configuration of the encapsulation module, metadata structures of a media file such as top-level boxes (e.g., ftyp or styp, moov, trak, mdat and boxes for sample description like stbl, stsd, etc) are created during an initialization step (step 310). Such an initialisation step may comprise reading parameter sets (e.g. geometry and attribute parameter sets) from an encoded bit-stream of point cloud data or may comprise obtaining information about the sensor (for uncompressed data) like a number of points, the types of attributes associated with the points (e.g., color, reflectance, timestamp, areas of interests, etc.); page 61 lines 23-28: The compression may be done with MPEG-I Part-9 to produce a G-PCC bitstream); and
a file generation unit (this element is interpreted under 35 USC 112(f) as a processor; encapsulation module 102, fig. 1; page 5 lines 9-13: At least parts of the methods according to the disclosure may be computer implemented. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system") that generates a file (media file 152) for storing the bit stream and stores the first information as metadata in the file (page 8 lines 3-7: Encapsulation module 102 may encapsulate received point cloud data according to an ISOBMFF-based format like MPEG-I Part-18, for interoperability purposes, in order to generate a media file like media file 152 that may be stored for later use by a player or by an image analysis tool or that may be transmitted to a media player or streaming client; page 12 lines 7-12: When the point cloud data are received by the encapsulation module as a G-PCC bit-stream (e.g. a bit-stream generated by the compression module 106 in Figure 1 , or by an external compression module), the configuration of the encapsulation module may use information from the parameter sets of the bit-stream or supplemental information associated with the bitstream (sometimes called SEI (Supplemental Enhancement Information) messages); page 17 lines 17-19: The example illustrated in Figure 5a is directed to a configuration corresponding to a single track encapsulation scheme according to which media file 500 comprises a metadata part (‘moov.sup.1 box 502) and a media data part (‘mdaf box 504); page 31 lines 16-19: Figure 8 illustrates a media file 800 (e.g. a media file 152 in Figure 1) having a G-PCC track described in ‘trak’ box 810 of ‘moov’ box 805, containing an additional timing information (ATI) or sub-frame description defined in a sample group description box 815 that is used with a G-PCC unit mapping sample group (references 820 and 825); page 75 lines 26-31: it possible to obtain a description of attribute data from a bit-stream containing encoded point cloud data, making it possible, in particular, to access and to extract specific encoded point cloud data and/or to describe point cloud data to be encapsulated, and to generate a media file comprising encapsulated point cloud data and a description of encapsulated attribute data).
Denoual teaches a compression module that performs functions similar to that of an encoding unit and a bit stream generation unit, although the encoding unit and the bit stream generation unit are claimed as two separated units of the information processing device.
However, in Nerwin v. Erlichman, 168 USPQ 177, 179 (PTO Bd. Of Int. 1969), it has been noted “the mere fact that a given structure is integral does not preclude its consisting of various elements”. Therefore, it would have been obvious to one of ordinary skilled in the art at the time of present invention to separate compression module of Denoual into two separate units to function as a encoding unit and a bit stream generation unit because using separate dedicated hardware units for specific functions will result in higher efficiency by freeing up resources. Also, See In re Dulberg, 289 F.2d 522, 523, 129 USPQ 348, 349 (CCPA 1961).
Regarding claim 2, Denoual teaches the information processing device according to claim 1, wherein the first information includes subframe_time_offset indicating an offset from a display timing of the sample (page 59 lines 17-30; page 60 lines 10-13: subframe_time_offset is an integer that gives the offset between Composition Timestamp CT(n) of the n-th sample and the Composition Timestamp CTs of the i-th subframe such that CTs[i] = CT[n]+ subframe_time_offset [n], with n equal to the current sample index for which SubframeCompositionOffset applies; page 69 line 20 – page 71 line 22 describes representation of sub-frame time offsets … “The sub-frame timing information of a sub-frame may be given by a time offset relative to the composition time of the sample containing this sub-frame or relative to a previous sub-frame in the same sample. In the latter case, time offsets may be smaller and could be encoded on a smaller number of bits (by default 32 bits could be used, as for sample timing)”).
Regarding claim 3, Denoual teaches the information processing device according to claim 2, wherein subframe_time_offset includes a sign (page 71 line 33 – page 72 line 9: The time offsets in the sub-frame timing information may be expressed as unsigned (positive) integers or as signed (positive or negative) integers ... when the composition time for the earliest sub-frame of a sample and the composition time for this sample are not equal (or not aligned), the time offsets for the sub-frames should be expressed as signed integers).
Regarding claim 4, Denoual teaches the information processing device according to claim 2, wherein the file further stores, as the metadata, second information on a display order of the sub-frame (rendering order of the sub-frame; page 61 line 23 – page 62 line10: The sub-frame configuration information may be dedicated to sub-frame timing information. One example of use is the case of combined frames, as illustrated with reference 950 in Figure 9: several frames (e.g., frame 1 and frame 2), captured by a 3D sensor, are aggregated into a combined frame for (geometry) compression efficiency. The compression may be done with MPEG-I Part-9 to produce a G-PCC bitstream. After aggregation, the frames that are aggregated are then denoted “sub-frames” or “G-PCC sub-frames” and the combined frame is denoted “G-PCC frame” or “frame”. The resulting bit-stream may then contain less encoded G-PCC frames than captured frames. The sub-frames may be indicated in the G-PCC bit-stream using frame index or frame number attributes. The encoder, when using a frame number attribute, may change the order between captured frames (denoted sub-frames) before aggregating them into combined frames (denoted frame or G-PCC frame). For example, a sub-frame captured at time 40ms may be encoded in a first frame with decoding time t while a sub- frame captured earlier may be encoded in a second frame with decoding time t+delta (delta being a positive time period). When encapsulating a G-PCC bit-stream with subframes into G-PCC tracks according to MPEG-I Part-18 (e.g., document w20565, dated October 2021), the resulting tracks do not provide timing information allowing the reconstruction of captured frames in their original order. Indeed, the composition time or decoding time of the samples cannot help since it is associated with encoded G-PCC frames and not with the captured frames. In order to help media readers or players to compute the composition timestamp of the sub-frames corresponding to their capture time, it is proposed to add sub-frame timing information as part of sub-frame configuration information in G-PCC tracks; page 66 lines 15-32: On implicit or explicit order of sub-frames in the sub-frame timing information, when the sub-frame timing information describes the timing of sub-frames present in the samples, an implicit order can be used. For example, in entries of a sample group description for sub-frame timing information, the items, or parameters, of sub-frame timing information making it possible to compute the composition time of a sub-frame (e.g., time offsets, time durations, or time offset deltas) are implicitly ordered in the order of sub-frames present in the samples, mapped to this sample group description entry, with increasing frame index or frame number attribute values. In a variant, the implicit order may follow a decreasing frame index or frame number attribute values. This requires the encapsulation module to obtain the information of the number of sub-frames per sample and their frame index or frame number values. This can be provided through application-specific means or by decoding the G-PCC bit-stream before encapsulation. The number of sub-frames per sample may be parameterized in the application or as input of a G-PCC encoder or from ATI inserted in the bit-stream. In a variant, providing the explicit order of sub-frames, the timing information describing each sub-frame may comprise an identifier of the sub-frame, e.g. the frame index or frame number attribute value), and the extraction unit extracts the sample according to the display order of the sub-frame (rendering order of the sub-frame) indicated in the second information (parsers reorder decoded subframes to render captured frames in a correct processing (e.g. rendering) order based on the sub-frame order; page 20 lines 19-24: Figure 6 illustrates an example of steps of a parsing process according to some embodiments of the invention, making it possible to identify metadata structures providing ATI in G-PCC tracks and to extract a subset of data corresponding to a G-PCC sub-frame or a set of G-PCC sub-frames. The ATI or sub-frame description also allows extraction of a sample containing specific sub-frames, or frame(s) that have been combined or aggregated (as illustrated with reference 950 in Figure 9; page 66 lines 3-28: More variants for signalling timing information, especially sub-frame timing information, may be considered, depending on whether: the declaration of sub-frame time offsets assumes an implicit order of sub-frames or not, the sub-frame time offset is declared as a function of the sample composition time or as a function of the previous sub-frame in the same sample, the sub-frame timing information distinguishes cases between subframes with constant duration and sub-frames with varying durations, and/or the sub-frame time offsets are expressed as unsigned (positive) integers or as signed (positive or negative) integers. On implicit or explicit order of sub-frames in the sub-frame timing information, when the sub-frame timing information describes the timing of sub-frames present in the samples, an implicit order can be used. For example, in entries of a sample group description for sub-frame timing information, the items, or parameters, of sub-frame timing information making it possible to compute the composition time of a sub-frame (e.g., time offsets, time durations, or time offset deltas) are implicitly ordered in the order of sub-frames present in the samples, mapped to this sample group description entry, with increasing frame index or frame number attribute values. In a variant, the implicit order may follow a decreasing frame index or frame number attribute values. This requires the encapsulation module to obtain the information of the number of sub-frames per sample and their frame index or frame number values. This can be provided through application-specific means or by decoding the G-PCC bit-stream before encapsulation. The number of sub-frames per sample may be parameterized in the application or as input of a G-PCC encoder or from ATI inserted in the bit-stream; page 55 lines 9-10: reorder the decoded subframes in a correct processing (e.g. rendering) order; page 67 lines 10-18: Indicating an explicit index allows readers or parsers to directly associate a time offset with a given sub-frame index or sub-frame number before decoding any data unit. In the case of implicit ordering, parsers or readers, when parsing the sub-frame timing information, may only associate the timing information (e.g., a time offset, time duration, or time offset delta) with the first sub-frame, the second sub-frame, etc., but the actual mapping between a frame index or frame number and its corresponding timing information, particularly when sub-frames are reordered in a sample or across samples, may only be determined when decoding a sample, in particular the geometry and framejndex or frame_number attribute data; page 67 lines 10-18: Indicating an explicit index allows readers or parsers to directly associate a time offset with a given sub-frame index or sub-frame number before decoding any data unit. In the case of implicit ordering, parsers or readers, when parsing the sub-frame timing information, may only associate the timing information (e.g., a time offset, time duration, or time offset delta) with the first sub-frame, the second sub-frame, etc., but the actual mapping between a frame index or frame number and its corresponding timing information, particularly when sub-frames are reordered in a sample or across samples, may only be determined when decoding a sample, in particular the geometry and framejndex or frame_number attribute data; page 72 lines 10-17: a parameter indicating whether sub-frame reordering needs to be done is also included in the SubFrameTimingGroupEntry. This parameter, set by the encapsulation module when it has the knowledge about sub-frame order in the G-PCC bit-stream, may be useful for readers or parsers. When an indication of reordering is set, parsers or readers know that they may need buffering to render captured frames (represented by sub-frames in encoded frames) in their acquisition order. Conversely, when the parameter indicates that no reordering is expected, parsers or readers do not need to allocate memory for reordering decoded frames or sub-frames).
Regarding claim 5, Denoual teaches the information processing device according to claim 1, wherein the first information includes second information indicating a display order of the sub-frame (rendering order of the sub-frame; page 61 line 23 – page 62 line10: The sub-frame configuration information may be dedicated to sub-frame timing information. One example of use is the case of combined frames, as illustrated with reference 950 in Figure 9: several frames (e.g., frame 1 and frame 2), captured by a 3D sensor, are aggregated into a combined frame for (geometry) compression efficiency. The compression may be done with MPEG-I Part-9 to produce a G-PCC bitstream. After aggregation, the frames that are aggregated are then denoted “sub-frames” or “G-PCC sub-frames” and the combined frame is denoted “G-PCC frame” or “frame”. The resulting bit-stream may then contain less encoded G-PCC frames than captured frames. The sub-frames may be indicated in the G-PCC bit-stream using frame index or frame number attributes. The encoder, when using a frame number attribute, may change the order between captured frames (denoted sub-frames) before aggregating them into combined frames (denoted frame or G-PCC frame). For example, a sub-frame captured at time 40ms may be encoded in a first frame with decoding time t while a sub- frame captured earlier may be encoded in a second frame with decoding time t+delta (delta being a positive time period). When encapsulating a G-PCC bit-stream with subframes into G-PCC tracks according to MPEG-I Part-18 (e.g., document w20565, dated October 2021), the resulting tracks do not provide timing information allowing the reconstruction of captured frames in their original order. Indeed, the composition time or decoding time of the samples cannot help since it is associated with encoded G-PCC frames and not with the captured frames. In order to help media readers or players to compute the composition timestamp of the sub-frames corresponding to their capture time, it is proposed to add sub-frame timing information as part of sub-frame configuration information in G-PCC tracks; page 66 lines 15-32: On implicit or explicit order of sub-frames in the sub-frame timing information, when the sub-frame timing information describes the timing of sub-frames present in the samples, an implicit order can be used. For example, in entries of a sample group description for sub-frame timing information, the items, or parameters, of sub-frame timing information making it possible to compute the composition time of a sub-frame (e.g., time offsets, time durations, or time offset deltas) are implicitly ordered in the order of sub-frames present in the samples, mapped to this sample group description entry, with increasing frame index or frame number attribute values. In a variant, the implicit order may follow a decreasing frame index or frame number attribute values. This requires the encapsulation module to obtain the information of the number of sub-frames per sample and their frame index or frame number values. This can be provided through application-specific means or by decoding the G-PCC bit-stream before encapsulation. The number of sub-frames per sample may be parameterized in the application or as input of a G-PCC encoder or from ATI inserted in the bit-stream. In a variant, providing the explicit order of sub-frames, the timing information describing each sub-frame may comprise an identifier of the sub-frame, e.g. the frame index or frame number attribute value) and a duration indicating a length of a display period of the sub-frame (page 59 lines 29-30: the timing information may also signal the duration of the sub-frames as an optional parameter; page 60 lines 14-15: subframe_duration, when present, is an unsigned integer indicating the duration of the sub-frame; page 66 lines 15-28: in entries of a sample group description for sub-frame timing information, the items, or parameters, of sub-frame timing information making it possible to compute the composition time of a sub-frame (e.g., time offsets, time durations, or time offset deltas) are implicitly ordered in the order of sub-frames present in the samples, mapped to this sample group description entry, with increasing frame index or frame number attribute values).
Regarding claim 6, Denoual teaches the information processing device according to claim 5, wherein the second information includes a frame number indicating a display order in entire sequence (page 14 line 34 – page 15 line 1: it may come as a G-PCC attribute data unit of type frame index (attrjabel equal to 3) or of type frame number (attrjabel equal to 4); page 15 lines 14-17: The parameters are, for example, an index (e.g., within a frame, for example the value of a frame index attribute for a G-PCC bit-stream or within a sequence, for example the value of a frame number attribute for a G-PCC bitstream); page 26 lines 16-17: subframe_number is the value of the frame number attribute of the sub-frame in the sample sequence; page 34 lines 23-27: instead of indicating a frame index of the sub-frame within a frame, a frame number of the sub-frame may be indicated within the sequence or track. It may correspond to a value indicated in a frame number attribute in a G-PCC data unit or to a range of values of frame number attributes in G-PCC data units after conversion to an index value within a frame; page 39 lines 29-32: For example, frame 1015-1 contains sub-frames 1020-10 to 1020-13. Each sub-frame has a sub-frame index or frame number, for example starting at O, incremented by 1 from one sub-frame to another within the PCC frame for frame index and within the sequence for frame number).
Regarding claim 7, Denoual teaches the information processing device according to claim 5, wherein the second information includes a frame index indicating a display order in the sample (page 14 line 34 – page 15 line 1: it may come as a G-PCC attribute data unit of type frame index (attrjabel equal to 3) or of type frame number (attrjabel equal to 4); page 15 lines 14-17: The parameters are, for example, an index (e.g., within a frame, for example the value of a frame index attribute for a G-PCC bit-stream or within a sequence, for example the value of a frame number attribute for a G-PCC bitstream); page 53 lines 15-23: Since the frame number attribute value is an index relative to the point cloud sequence (i.e. not relatively to the point cloud frame containing the sub-frames as the frame index), the order of the subframes may be different from the decoding order of the point cloud frames. For this reason, the specialized AVDI data structure may indicate whether the frame number attribute values are strictly increasing (or decreasing) between consecutive point cloud frames. In such a case, the encapsulation may indicate if buffering the decoded subframes is necessary to process (e.g. render) the sub-frames in a correct order; page 66 lines 15-32: On implicit or explicit order of sub-frames in the sub-frame timing information, when the sub-frame timing information describes the timing of sub-frames present in the samples, an implicit order can be used. For example, in entries of a sample group description for sub-frame timing information, the items, or parameters, of sub-frame timing information making it possible to compute the composition time of a sub-frame (e.g., time offsets, time durations, or time offset deltas) are implicitly ordered in the order of sub-frames present in the samples, mapped to this sample group description entry, with increasing frame index or frame number attribute values. In a variant, the implicit order may follow a decreasing frame index or frame number attribute values. This requires the encapsulation module to obtain the information of the number of sub-frames per sample and their frame index or frame number values. This can be provided through application-specific means or by decoding the G-PCC bit-stream before encapsulation. The number of sub-frames per sample may be parameterized in the application or as input of a G-PCC encoder or from ATI inserted in the bit-stream. In a variant, providing the explicit order of sub-frames, the timing information describing each sub-frame may comprise an identifier of the sub-frame, e.g. the frame index or frame number attribute value; page 68 lines 2-4: A second loop provides time offsets for the present sub-frames with an implicit order of sub-frames (e.g., increasing frame index or frame number values)).
Regarding claim 8, Denoual teaches the information processing device according to claim 1, wherein the file is International Organization for Standardization Base Media File Format (ISOBMFF) (page 2 lines 24-25: According to a first aspect of the disclosure there is provided a method for encapsulating point cloud data in a file compliant with an ISOBMFF based standard; page 8 lines 3-7: Encapsulation module 102 may encapsulate received point cloud data according to an ISOBMFF-based format like MPEG-I Part-18, for interoperability purposes, in order to generate a media file like media file 152 that may be stored for later use by a player or by an image analysis tool or that may be transmitted to a media player or streaming client; page 8 lines 25-30: Media file 152 may consist in a single media file or in a set of media segment files, for example as ISOBMFF segments (ISO base media file containing one or more segment(s)). The media file may be a fragmented file, for example for live acquisition or capture and encapsulation or live (or low-latency) streaming. It may comply with the ISOBMFF standard or to standard specifications derived from ISOBMFF (e.g. MPEG-I Part- 18)), and the first information is stored in SubSampleInformationBox (page 17 lines 28-34: For single track encapsulation of G-PCC data in ISOBMFF, the SubSamplelnformationBox with flags equal to 0 in SampleTableBox, or in the TrackFragmentBox of each of its MovieFragmentBoxes is present. SubSamplelnformationBox with flags equal to 2 (frame index) or 3 (frame_number) in SampleTableBox, or in the TrackFragmentBox of each of its MovieFragmentBoxes may also be present; page 18 line 31 – page 19 line 5: For this multi-track encapsulation of G-PCC data in ISOBMFF, when sub-samples are present, SubSamplelnformationBox with flags equal to 1 in SampleTableBox, or in the TrackFragmentBox of each of its MovieFragmentBoxes should be present. SubSamplelnformationBox with flags equal to 2 or 3 in SampleTableBox, or in the TrackFragmentBox of each of its MovieFragmentBoxes may also be present. SubSamplelnformationBox with flags equal to 0 in SampleTableBox, or in the TrackFragmentBox of each of its MovieFragmentBoxes should not be present, because useless (each track conveying its own type of data). The syntax and semantics of the sub-sample with flags equal to 1 , 2, or 3 as defined in reference to Figure 7 applies here; page 42 line 28-33: MPEG-I Part-18 defines G-PCC items and sub-sample item property under the ‘meta’ box of a media file. Similarly, to G-PCC tracks, a G-PCC item, corresponding to one Point Cloud frame may provide description of sub-frames within this image. The sub-sample item property may then be extended to support the subsample information box as described in reference to Figure 7 defining new flags value(s) or reusing existing flags value).
Regarding claim 9, Denoual teaches the information processing device according to claim 1, wherein the file is International Organization for Standardization Base Media File Format (ISOBMFF) (page 2 lines 24-25: According to a first aspect of the disclosure there is provided a method for encapsulating point cloud data in a file compliant with an ISOBMFF based standard; page 8 lines 3-7: Encapsulation module 102 may encapsulate received point cloud data according to an ISOBMFF-based format like MPEG-I Part-18, for interoperability purposes, in order to generate a media file like media file 152 that may be stored for later use by a player or by an image analysis tool or that may be transmitted to a media player or streaming client; page 8 lines 25-30: Media file 152 may consist in a single media file or in a set of media segment files, for example as ISOBMFF segments (ISO base media file containing one or more segment(s)). The media file may be a fragmented file, for example for live acquisition or capture and encapsulation or live (or low-latency) streaming. It may comply with the ISOBMFF standard or to standard specifications derived from ISOBMFF (e.g. MPEG-I Part- 18)), and the first information is stored in SampleGroupBox (SampleGroupDescriptionBox; page 13 lines 2-5: For dynamic configuration, a metadata structure dedicated to sub-frame configuration information may be provided as entries in a SampleGroupDescriptionBox with a specific grouping type so that group of samples sharing the same configuration can be mapped to one of these entries; page 31 lines 22-34: According to this embodiment, the file writer (e.g. file writer 100 in Figure 1) generates a first sample grouping to map samples onto entries with different TLV unit patterns. This first sample grouping results in a TLV mapping sample group indicated, for example, by a grouping_type = ‘tlvm’ (here reference 830) in a SampleToGroupBox box (here ‘sbgp’ box 820) and in an associated SampleGroupDescriptionBox box (here ‘sgpd’ box 825). As illustrated, SampleToGroupBox 820 defines a group of samples and for each group, indicates an entry (reference 835) in the SampleToGroupDescriptionBox 830. Each entry 835 is a specific VolumetricVisualSampleGroupEntry, called TLVMapGroupEntry. A TLVMapGroupEntry can be used to map a range of contiguous TLV units into an entry in a second ‘sgpd’ box. By default, this second ‘sgpd’ box may be a SampleGroupDescriptionBox with grouping type indicating that the box contains SubFramelnformationGroupEntry (reference 840), for example the grouping type ‘sfif (sub-frame information) as illustrated with reference 845; page 62 lines 2-20: When encapsulating a G-PCC bit-stream with subframes into G-PCC tracks according to MPEG-I Part-18 (e.g., document w20565, dated October 2021), the resulting tracks do not provide timing information allowing the reconstruction of captured frames in their original order. Indeed, the composition time or decoding time of the samples cannot help since it is associated with encoded G-PCC frames and not with the captured frames. In order to help media readers or players to compute the composition timestamp of the sub-frames corresponding to their capture time, it is proposed to add sub-frame timing information as part of sub-frame configuration information in G-PCC tracks. Since MPEG-I Part-18 defines several kinds of track, it is described hereafter in which tracks it is recommended to add the sub-frame timing information. Preferably, it is embedded in the media file as a sample group because it allows handling both static or dynamic configurations, through appropriate flags or version values in the SampleGroupDescriptionBox (‘sgpd’). A sub-frame configuration timing sample group may provide the sub-frame timing information, e.g. the timing information on G-PCC sub-frames. The sub-frame timing information is indicated by encapsulation module as a time offset relative to the composition time of the sample containing the G-PCC sub-frame, e.g., as a time offset to the composition time of the sample containing the sub-frame or as a time offset to the previous sub-frame in the same sample).
Claims 10, 12-19 and 20 are similar in scope to claims 1, 2-9 and 11, and therefore the examiner provides similar rationale to reject these claims.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JWALANT B AMIN whose telephone number is (571)272-2455. The examiner can normally be reached Monday-Friday 10am - 630pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JWALANT AMIN/Primary Examiner, Art Unit 2612