DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the response to this Office Action, the Examiner respectfully requests that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line numbers in the specification and/or drawing figure(s). This will assist the Examiner in prosecuting this application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-16 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2021/0397260 A1 to Birnbaum et al. (hereinafter "Birnbaum") in view of “Text for Working Draft of ISO/IEC 23090-31: Haptics Coding” (hereinafter "Haptics Coding").
Regarding Claims 1 and 5, Birnbaum teaches a method and a device for encoding haptic data of a haptic sequence (Claim 1; Figs. 1A-3; Para. 31-40 of Birnbaum; computing device 1100 may be configured to facilitate the providing of a haptic effect for experiencing the 3D environment by generating a drive signal 1400 for a haptic output device 1210), the device comprising a processor configured for: obtaining binary haptic data representative of haptic effects and metadata describing the haptic effects (Claim 1; Figs. 1A-3; Para. 31-40, 113 of Birnbaum; computing device 1100 may include one or more processors 1110 that are configured to receive media data 1300 that describes aspects of the 3D environment, and may generate the drive signal 1400 based on the media data 1300. The media data 1300 may have, e.g., an omnidirectional media format (OMAF) for allowing the 3D environment to be viewed in multiple directions, or may have some other format… media data may be generated according to the MPEG-I standard… haptic effect may be defined in metadata at a source and rendered at a device on the client side, such as the computing device 1100 or user peripheral device 1200); encoding tracks and the metadata in access units (Claim 1; Figs. 1A-3; Para. 31-40, 113 of Birnbaum; media data may have been created by encoding various sources of data, such as video data, audio data, and haptic data… media data may be generated according to the MPEG-I standard… a device (e.g., computing device 1500) may encapsulate and/or store the media data, which may include the encoded haptic data, audio data, video data, and/or image data, in a file format such as the International Standards Organization Base Media File Format (ISOBMFF)… haptic effect may be defined in metadata at a source and rendered at a device on the client side, such as the computing device 1100 or user peripheral device 1200).
Birnbaum does not explicitly disclose decomposing the haptic effects in temporal events and frequency bands and grouping them in tracks; and encoding the tracks and the metadata, the metadata comprising perception data pointing to a sub-set of tracks and experience data pointing to a sub-set of perception data.
However, Haptics Coding teaches decomposing the haptic effects in temporal events and frequency bands and grouping them in tracks (Figures 1-3; Section 3.1 to 3.2, 4.1 to 4.7 of Haptics Coding; haptic data of a track is contained in a set of haptic bands defined by their frequency range); and encoding the tracks and the metadata, the metadata comprising perception data pointing to a sub-set of tracks and experience data pointing to a sub-set of perception data (Figures 1-3, 8-9; Section 3.1 to 3.2, 4.1 to 4.8, 5.1, 6.1 to 6.2 of Haptics Coding; In addition to specific metadata, a perception contains a list of tracks where the data is decomposed in frequency bands. Each band defines part of the signal in a given frequency range. The bands are described with a list of Haptic effects each containing a list of keyframes. The haptic signal in a track can then be reconstructed by combining the data in the different bands as illustrated in Figure 3… haptic signals can be encoded on multiple tracks… a haptic track defines a signal to be rendered at a specific body location. Metadata stored at the track level includes information such as the gain associated to the track, the mixing weight, the desired body location of the haptic feedback and optionally the reference device and/or a direction. Additional information such as the desired sampling frequency or sample count can also be provided).
Therefore, at the time when the invention was filed, it would have been obvious to a person of ordinary skill in the art to include decomposing the haptic effects in temporal events and frequency bands and grouping them in tracks; and encoding the tracks and the metadata, the metadata comprising perception data pointing to a sub-set of tracks and experience data pointing to a sub-set of perception data using the teachings of Haptics Coding in order to modify the device taught by Birnbaum. The motivation to combine these analogous arts would have been to encode both descriptive and quantized data in a human readable JSON format used for exchange purposes and a compressed bitstream version, optimized for memory usage, for distribution purposes (Section 1 of Haptics Coding).
Regarding Claims 9 and 13, Birnbaum teaches a method and a device for decoding haptic data of a haptic sequence (Claim 1; Figs. 1A-4; Para. 31-45 of Birnbaum; computing device 1100 may be configured to facilitate the providing of a haptic effect for experiencing the 3D environment by generating a drive signal 1400 for a haptic output device 1210… decoding and/or rendering of media data may be performed by the computing device 1100 of FIGS. 1A-1C. FIG. 4 illustrates a computing device 4100, which may be an embodiment of the computing device 1100. In this embodiment, the computing device 4100 may include at least one processor 4110 and a memory 4120… modules 4120 a, 4120 b may include instructions which may be executed by the processor 4110 to execute a haptic decoding operation), the device comprising a processor configured for: obtaining a set of tracks and metadata encoded in access units (Claim 1; Figs. 1A-3; Para. 31-42, 113 of Birnbaum; media data may have been created by encoding various sources of data, such as video data, audio data, and haptic data… media data may be generated according to the MPEG-I standard… a device (e.g., computing device 1500) may encapsulate and/or store the media data, which may include the encoded haptic data, audio data, video data, and/or image data, in a file format such as the International Standards Organization Base Media File Format (ISOBMFF)… computing device 1100 may receive and process (e.g., decapsulate) the file F/Fs, such as the ISOBMFF file, which may include file data that encodes the media data. More particularly, the processed file data may include encoded haptic data E′h, encoded audio data E′a, and/or encoded video data E′v. In some implementations, the OMAF player may be configured to decode various encoded data using a codec… haptic effect may be defined in metadata at a source and rendered at a device on the client side, such as the computing device 1100 or user peripheral device 1200).
Birnbaum does not explicitly disclose the metadata comprising perception data pointing to a sub-set of tracks and experience data pointing to a sub-set of perception data; and accessing the experience data in tracks pointed by perception data associated with the experience data.
However, Haptics Coding teaches metadata comprising perception data pointing to a sub-set of tracks and experience data pointing to a sub-set of perception data; and accessing the experience data in tracks pointed by perception data associated with the experience data (Figures 1-3, 8-9; Section 3.1 to 3.2, 4.1 to 4.8, 5.1, 6.1 to 6.2, 7.1 to 7.2 of Haptics Coding; haptic data of a track is contained in a set of haptic bands defined by their frequency range… In addition to specific metadata, a perception contains a list of tracks where the data is decomposed in frequency bands. Each band defines part of the signal in a given frequency range. The bands are described with a list of Haptic effects each containing a list of keyframes. The haptic signal in a track can then be reconstructed by combining the data in the different bands as illustrated in Figure 3… haptic signals can be encoded on multiple tracks… a haptic track defines a signal to be rendered at a specific body location. Metadata stored at the track level includes information such as the gain associated to the track, the mixing weight, the desired body location of the haptic feedback and optionally the reference device and/or a direction. Additional information such as the desired sampling frequency or sample count can also be provided).
Therefore, at the time when the invention was filed, it would have been obvious to a person of ordinary skill in the art to include the metadata comprising perception data pointing to a sub-set of tracks and experience data pointing to a sub-set of perception data; and accessing the experience data in tracks pointed by perception data associated with the experience data using the teachings of Haptics Coding in order to modify the device taught by Birnbaum. The motivation to combine these analogous arts would have been to encode both descriptive and quantized data in a human readable JSON format used for exchange purposes and a compressed bitstream version, optimized for memory usage, for distribution purposes (Section 1 of Haptics Coding).
Regarding Claims 6, 10, 14, and 18, the combination of Birnbaum and Haptics Coding teaches that the access units are Network Abstraction Layer units (Figs. 1A-3; Para. 31-40 of Birnbaum; media data may be generated according to the MPEG-I standard).
Regarding Claims 7, 11, 15, and 19, the combination of Birnbaum and Haptics Coding teaches that the access units are structured depending on whether they comprise metadata or band-data (Figs. 1A-3; Para. 31-40 of Birnbaum; media data may be generated according to the MPEG-I standard… Section 6.1 to 6.2 of Haptics Coding;
PNG
media_image1.png
270
802
media_image1.png
Greyscale
).
Regarding Claims 8, 12, 16, and 20, the combination of Birnbaum and Haptics Coding teaches that the metadata information describing a haptic effect library, avatars or devices (Section 3.2 to 4.6, 6.1 of Haptics Coding; For each haptic perception, metadata information is provided on the modality, the corresponding avatar representation, and technical characteristics of compatible haptic devices… Metadata stored at the track level includes information such as the gain associated to the track, the mixing weight, the desired body location of the haptic feedback and optionally the reference device and/or a direction).
Response to Arguments
Applicant's arguments filed 05/04/2026 have been fully considered but they are not persuasive.
Examiner respectfully disagrees with applicant representative’s arguments on the grounds that no prima facie evidence of obviousness was established for all limitations that over the combination of Birnbaum and Haptics Coding. Applicant argues that Birnbaum fails to disclose the claimed “access units” because Applicant’s Specification describes an access unit as the “lowest, independently decodable information.” Applicant further argues that Birnbaum merely discloses a high-level MPEG-I media framework in which haptic data is encapsulated within a larger media stream and does not disclose “slicing haptic signals into the lowest independently decodable blocks to facilitate parallel processing.”
This argument is not persuasive because it relies upon limitations that are not recited in the claims.
The claims recite, for example, “encoding the tracks and the metadata in access units.” The claims do not recite that each access unit must constitute the lowest independently decodable information, nor do they require that the access units be mapped to network packets, processed in parallel, or possess the particular streaming implementation discussed by Applicant.
Although Applicant relies upon paragraphs [0035] and [0039] of the Specification, limitations appearing in the Specification are not ordinarily read into the claims absent a claim limitation requiring such features or a sufficiently clear definition requiring the asserted construction. Moreover, Applicant's reliance on advantages allegedly resulting from the disclosed implementation such as streaming, network distribution, and parallel processing does not distinguish the claimed subject matter where those particular implementation details are not recited by the claims. As explained in the Office Action, Birnbaum teaches encoding media data including haptic data and associated metadata and encapsulating and/or storing such encoded media data in an appropriate media format. Birnbaum therefore provides evidence of the claimed encoding and organization of the tracks and metadata. Applicant has not identified language in the claims requiring the narrower packetization and parallel-processing implementation upon which Applicant's argument relies.
Accordingly, Applicant's argument does not establish that Birnbaum fails to teach or suggest the claimed encoding of tracks and metadata in access units.
Applicant additionally argues that Haptics Coding merely discloses a “nested file hierarchy,” whereas the claims allegedly require a fundamentally different “pointer-based metadata hierarchy.” This argument is also not persuasive. The independent claims recite metadata comprising: “perception data pointing to a sub-set of tracks and experience data pointing to a sub-set of perception data.” However, Applicant's argument effectively construes “pointing to” as requiring a particular pointer-based organizational layer that permits a decoder to skip irrelevant data, selectively navigate a stream, and independently process selected chunks in parallel. Those additional structural and functional requirements are not apparent from the quoted claim language. In particular, the claims do not expressly require a computer-memory pointer, direct address, random-access pointer, or any particular implementation by which the claimed data “points to” the corresponding subset. Nor do the quoted limitations require that the decoder access the referenced subset without parsing other portions of a file.
Haptics Coding teaches relationships between its metadata, perceptions, tracks, frequency bands, and haptic effects. As acknowledged in Applicant's own characterization of the reference, a perception contains a list of tracks, tracks contain or are associated with corresponding haptic information, and track-level metadata identifies characteristics associated with particular tracks, including body location, gain, mixing weight, reference device, direction, sampling frequency, and sample count. Thus, the fact that Haptics Coding may implement these relationships using a nested or hierarchical file organization does not establish that the disclosed metadata fails to identify, reference, or otherwise point to the corresponding tracks or subsets of data within the meaning of the claims.
Applicant's distinction between a “nested file hierarchy” and a “pointer-based metadata hierarchy” therefore relies upon an implementation distinction that is not commensurate in scope with the claims.
Applicant further argues that the claimed organization enables a decoder to “skip irrelevant data and process only the required chunks in parallel,” whereas Haptics Coding allegedly requires an entire file structure to be parsed. This argument is not persuasive because Applicant has not established that the relied-upon advantages constitute limitations of the claims. The patentability inquiry is directed to the subject matter actually recited in the claims rather than every advantage associated with an embodiment described in the Specification. To the extent Applicant contends that the invention requires selective random access, processing without parsing intervening data, parallel decoding, or mapping independently decodable access units to network packets, Applicant may amend the claims to expressly recite such limitations. The presently recited limitations, however, are broader.
Applicant's arguments additionally address Birnbaum and Haptics Coding largely in isolation. The rejection under 35 U.S.C. §103, however, is based upon the combined teachings of Birnbaum and Haptics Coding. Birnbaum is relied upon for, inter alia, the encoding and encapsulation of media including haptic information and metadata, while Haptics Coding is relied upon for the organization and relationships among perceptions, tracks, frequency bands, haptic effects, and associated metadata. It is not necessary that either reference independently disclose every limitation of the claims where the rejection is based upon what the combined teachings would have suggested to a person of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). Accordingly, Applicant's showing that Birnbaum alone allegedly lacks the particular metadata hierarchy of Haptics Coding, or that Haptics Coding alone allegedly lacks Applicant's asserted access-unit implementation, does not rebut the rejection as articulated based upon the combination.
For at least the reasons discussed above, Applicant's arguments have been fully considered but are not persuasive. Applicant's arguments rely upon limitations including lowest-level independent decodability, network-packet mapping, parallel decoding, selective navigation without parsing, and a particular pointer implementation that are not expressly required by the claims.
The teachings of Birnbaum and Haptics Coding, considered together for the reasons previously set forth, continue to provide evidence that the subject matter of claims 1–16 would have been obvious to one of ordinary skill in the art.
Accordingly, the rejection of claims 1–16 under 35 U.S.C. §103 is maintained.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABHISHEK SARMA whose telephone number is (571)272-9887. The examiner can normally be reached on Mon - Fri 8:00-5:00.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amr Awad can be reached on 571-272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABHISHEK SARMA/
Primary Examiner, Art Unit 2621
Prosecution Insights