MetaTF: a Language for Specifying Trace Formats

Richard Jones, 30 March 2001

The files in this directory represent the current definition of Meta-TF, version 1.2.1. This has been developed from Meta-TF 1.0 after discussion at ISMM 2000 in Minneapolis, October 15-16, 2000. (Meta-TF 1.1 was a simplification of Meta-TF 1.0, designed to simplify tool construction).

Program traces

Dynamic storage allocation continues to play an important role in the performance and correctness of systems ranging from user productivity software to high-performance servers. While algorithms for dynamic storage allocation have been studied for decades, much of the literature is based on measuring the performance of benchmark programs unrepresentative of many important allocation-intensive workloads.

To overcome the difficulties of acquiring benchmarks and of generating program traces of realistic scale, Trishul Chilimbi, Ben Zorn and Richard Jones proposed in our ISMM 2000 paper, Designing a Trace Format for Heap Allocation Events that large traces from realistic benchmarks be collected and shared . This site provides a portable mechanism for sharing traces. We hope that others will take up these tools and share their traces with the community.

A specification language

MetaTF is proposed as a standard for publishing and exchanging representative allocation workloads. Although our interest lies in heap allocation events, MetaTF is a general-purpose specification language for program traces. A trace is specified by a DTD, a document type definition (but not to be confused with SGML or XML DTDs). The DTD defines sections of the trace and records within each section. Record formats allow trace events to be encoded in a number of ways: for example, defaults may be used, or an allocation address may be given as a delta from the previous address, or an event may refer to a record in another section (for example, that section may contain a list of fully qualified type names). A novel feature of MetaTF is that a trace may contain metadata to vary the encoding of the trace on the fly.

DTDs, traces and tools

We provide a full, albeit terse, specification of MetaTF and a tool for automatically constructing reader and writer routines from a particular MetaTF DTD. The description of MetaTF also includes a tutorial on the use of the tool. Currently MetaTFtool generates only Java. Offers to extend it to generate readers/writers in other languages (particularly C or C++) will be gratefully received.

We also provide a directory of DTDs (currently rather thin :-) and a trace directory. Space limitations mean that the trace directory will provide links to traces rather than holding them here.

MetaTFtool

MetaTFtool is a compiler for MetaTF. It takes a MetaTF 1.2 DTD and produces a set of Java classes representing a reader/writer for traces that conform to the DTD. MetaTFtool is described in the full report along with a brief tutorial. Provided here is

Changes from MetaTF 1.2

The record field interpretation property is renamed encoding.

Changes from Meta-TF 1.0

The original definition of Meta-TF 1.0 can be found in Designing a Trace Format for Heap Allocation Events, Trishul Chilimbi, Richard Jones and Benjamin Zorn, ISMM 2000, Minneapolis, October 2000, ACM. Following discussion at ISMM~2000, Meta-TF 1.2 makes the following changes: