1 of 10

Novadrop

Home

Novadrop provides a collection of libraries, tools, and documentation for modifying the TERA game client.

Development of TERA stopped on April 20, 2022. Official game servers shut down on June 30 in the same year. Some of the knowledge and functionality provided by Novadrop was previously held in relative secrecy by a small portion of the game's third-party modding community for the sake of preserving game integrity. With the game effectively defunct, Novadrop now enables the game's community to freely analyze and modify the client, e.g. for the purposes of creating unofficial server emulators.

Game

Data Center Format

This page describes the encrypted and compressed data center format used by TERA.

C/C++-like primitive types, enums, structs, and unions will be used.
bool is equivalent to uint8_t but only allows the values true (1) and false (0).
Integers (uint8_t, int8_t, uint16_t, int16_t, etc) are little endian.
float and double are IEEE 754 binary32 and binary64, respectively.
Characters (i.e. char16_t) are UTF-16 and little endian.
Fields are laid out in the declared order with no implied padding anywhere.

Note that the format uses both zero-based and one-based array indexes in various, seemingly random places.

Encryption

Data center files are encrypted with the AES algorithm in CFB mode and with block, key, and feedback sizes all set to 128 bits. No padding is done for the final block.

The encryption key and initialization vector can both be extracted from a TERA client. This can be done with a running TERA process or an unpacked executable. These values are usually freshly generated for each client build.

Physical Structure

The overall structure can be described like this:

struct DataCenterFile
{
    DataCenterCompressionHeader compression_header;
    DataCenterHeader header;
    DataCenterSimpleRegion<DataCenterKey, false> keys;
    DataCenterSegmentedRegion<DataCenterAttribute> attributes;
    DataCenterSegmentedRegion<DataCenterNode> nodes;
    DataCenterStringTable<1024> values;
    DataCenterStringTable<512> names;
    DataCenterFooter footer;
};

Compression Header

After decryption, there is a small header of the form:

struct DataCenterCompressionHeader
{
    uint32_t uncompressed_size;
    uint16_t zlib_header;
};

All data immediately following this header is compressed with the Deflate algorithm.

uncompressed_size is the size of the data center file once inflated.

zlib_header is a zlib (§2.2) header. In official data center files, it is usually of the form 0x9c78, but it can be any valid zlib header.

File Header

After decompression, a data center file starts with this header:

struct DataCenterHeader
{
    uint32_t version;
    double timestamp;
    uint32_t revision;
    int16_t unknown_1;
    int16_t unknown_2;
    int32_t unknown_3;
    int32_t unknown_4;
    int32_t unknown_5;
};

version is currently 6. A past version 3 also existed. Note that this field has no distinguishing value for the 32-bit and 64-bit formats, but version 3 was only 32-bit.

timestamp is a Unix timestamp indicating when the file was produced.

unknown_1, unknown_2, unknown_3, unknown_4, and unknown5 are all always 0. They are actually part of a tree structure describing the XSD schema of the data graph, but official data centers never include this information.

revision indicates the version of the data graph contained within the file. It is sometimes (but not always) equal to the value sent by the client in the C_CHECK_VERSION packet. This field is not present if version is 3.

File Footer

A data center file ends with this footer:

struct DataCenterFooter
{
    int32_t marker;
}

marker is always 0 and has no known purpose in the client.

Regions

Most of the content in data center files is arranged into regions, which may be segmented. The region structures used throughout the format are described here:

struct DataCenterSimpleRegion<typename T, bool off_by_one>
{
    uint32_t count;
    T elements[off_by_one ? count - 1 : count];
};

struct DataCenterSegmentedSimpleRegion<typename T, uint32_t count>
{
    DataCenterSimpleRegion<T, false> segments[count];
};

struct DataCenterRegion<typename T>
{
    uint32_t full_count;
    uint32_t used_count;
    T elements[full_count];
};

struct DataCenterSegmentedRegion<typename T>
{
    uint32_t count;
    DataCenterRegion<T> segments[count];
};

In a DataCenterRegion, the used_count can be less than the full_count. full_count is usually 65535, even when used_count is much smaller. All data in the region that goes beyond used_count can be arbitrary and should be considered undefined.

A DataCenterSegmentedSimpleRegion is mostly the same as a DataCenterSegmentedRegion, with the main difference being that it has a static amount of segments.

Addresses are frequently used to refer to elements within both types of segmented regions:

struct DataCenterAddress
{
    uint16_t segment_index;
    uint16_t element_index;
};

Here, segment_index is a zero-based index into the segments array of the segmented region, while element_index is a zero-based index into the elements array of the segment.

String Tables

All strings, whether they are names or values, are arranged into string tables, which are effectively used as hash tables by the client. A string table has the form:

struct DataCenterStringTable<uint32_t count>
{
    DataCenterSegmentedRegion<char16_t> data;
    DataCenterSegmentedSimpleRegion<DataCenterString, count> table;
    DataCenterSimpleRegion<DataCenterAddress, true> addresses;
};

A string entry in the table region looks like this:

struct DataCenterString
{
    uint32_t hash;
    uint32_t length;
    uint32_t index;
    DataCenterAddress address;
};

hash is a hash code for the string, given by the expression data_center_string_hash(value) where value is the string value. In a typical data center file, there is only a very tiny amount of hash collisions.

length is the length of the string in terms of characters, including the NUL character.

index is a one-based index into the string table's addresses region. The address at this index must match the address field exactly.

address is an address into the string table's data region. This address points to the actual string data. The string read from this address must have the same length as the length field. Notably, if the string data straddles the end of its segment, the NUL character may be omitted. Readers should therefore not rely exclusively on the presence of a NUL character, but also check the segment bounds.

A string entry must be placed in the correct table segment based on its hash field. The segment index is given by the expression (hash ^ hash >> 16) % count where count is the static size of the table region. Further, entries in a segment must be sorted by their hash code in ascending order.

For the names table, the special names __root__ and __value__ must be added to the table last, so that their index values are greater than all other entries. Also, they must be present even if they are not used.

Finally, it is worth noting that the names table is always referred to by index, whereas the values table is always referred to by address. The reason for this is that the names table is small enough that all entries can be accessed by a uint16_t index value into its addresses region, whereas that is not the case for the values table. In spite of this difference, names and values must both have valid addresses regions.

String Hash

The data_center_string_hash function is a bizarre variant of CRC32. It is defined as follows:

const uint32_t string_hash_table[256] =
{
    0x00000000, 0x04c11db7, 0x09823b6e, 0x0d4326d9, 0x130476dc, 0x17c56b6b, 0x1a864db2, 0x1e475005,
    0x2608edb8, 0x22c9f00f, 0x2f8ad6d6, 0x2b4bcb61, 0x350c9b64, 0x31cd86d3, 0x3c8ea00a, 0x384fbdbd,
    0x4c11db70, 0x48d0c6c7, 0x4593e01e, 0x4152fda9, 0x5f15adac, 0x5bd4b01b, 0x569796c2, 0x52568b75,
    0x6a1936c8, 0x6ed82b7f, 0x639b0da6, 0x675a1011, 0x791d4014, 0x7ddc5da3, 0x709f7b7a, 0x745e66cd,
    0x9823b6e0, 0x9ce2ab57, 0x91a18d8e, 0x95609039, 0x8b27c03c, 0x8fe6dd8b, 0x82a5fb52, 0x8664e6e5,
    0xbe2b5b58, 0xbaea46ef, 0xb7a96036, 0xb3687d81, 0xad2f2d84, 0xa9ee3033, 0xa4ad16ea, 0xa06c0b5d,
    0xd4326d90, 0xd0f37027, 0xddb056fe, 0xd9714b49, 0xc7361b4c, 0xc3f706fb, 0xceb42022, 0xca753d95,
    0xf23a8028, 0xf6fb9d9f, 0xfbb8bb46, 0xff79a6f1, 0xe13ef6f4, 0xe5ffeb43, 0xe8bccd9a, 0xec7dd02d,
    0x34867077, 0x30476dc0, 0x3d044b19, 0x39c556ae, 0x278206ab, 0x23431b1c, 0x2e003dc5, 0x2ac12072,
    0x128e9dcf, 0x164f8078, 0x1b0ca6a1, 0x1fcdbb16, 0x018aeb13, 0x054bf6a4, 0x0808d07d, 0x0cc9cdca,
    0x7897ab07, 0x7c56b6b0, 0x71159069, 0x75d48dde, 0x6b93dddb, 0x6f52c06c, 0x6211e6b5, 0x66d0fb02,
    0x5e9f46bf, 0x5a5e5b08, 0x571d7dd1, 0x53dc6066, 0x4d9b3063, 0x495a2dd4, 0x44190b0d, 0x40d816ba,
    0xaca5c697, 0xa864db20, 0xa527fdf9, 0xa1e6e04e, 0xbfa1b04b, 0xbb60adfc, 0xb6238b25, 0xb2e29692,
    0x8aad2b2f, 0x8e6c3698, 0x832f1041, 0x87ee0df6, 0x99a95df3, 0x9d684044, 0x902b669d, 0x94ea7b2a,
    0xe0b41de7, 0xe4750050, 0xe9362689, 0xedf73b3e, 0xf3b06b3b, 0xf771768c, 0xfa325055, 0xfef34de2,
    0xc6bcf05f, 0xc27dede8, 0xcf3ecb31, 0xcbffd686, 0xd5b88683, 0xd1799b34, 0xdc3abded, 0xd8fba05a,
    0x690ce0ee, 0x6dcdfd59, 0x608edb80, 0x644fc637, 0x7a089632, 0x7ec98b85, 0x738aad5c, 0x774bb0eb,
    0x4f040d56, 0x4bc510e1, 0x46863638, 0x42472b8f, 0x5c007b8a, 0x58c1663d, 0x558240e4, 0x51435d53,
    0x251d3b9e, 0x21dc2629, 0x2c9f00f0, 0x285e1d47, 0x36194d42, 0x32d850f5, 0x3f9b762c, 0x3b5a6b9b,
    0x0315d626, 0x07d4cb91, 0x0a97ed48, 0x0e56f0ff, 0x1011a0fa, 0x14d0bd4d, 0x19939b94, 0x1d528623,
    0xf12f560e, 0xf5ee4bb9, 0xf8ad6d60, 0xfc6c70d7, 0xe22b20d2, 0xe6ea3d65, 0xeba91bbc, 0xef68060b,
    0xd727bbb6, 0xd3e6a601, 0xdea580d8, 0xda649d6f, 0xc423cd6a, 0xc0e2d0dd, 0xcda1f604, 0xc960ebb3,
    0xbd3e8d7e, 0xb9ff90c9, 0xb4bcb610, 0xb07daba7, 0xae3afba2, 0xaafbe615, 0xa7b8c0cc, 0xa379dd7b,
    0x9b3660c6, 0x9ff77d71, 0x92b45ba8, 0x9675461f, 0x8832161a, 0x8cf30bad, 0x81b02d74, 0x857130c3,
    0x5d8a9099, 0x594b8d2e, 0x5408abf7, 0x50c9b640, 0x4e8ee645, 0x4a4ffbf2, 0x470cdd2b, 0x43cdc09c,
    0x7b827d21, 0x7f436096, 0x7200464f, 0x76c15bf8, 0x68860bfd, 0x6c47164a, 0x61043093, 0x65c52d24,
    0x119b4be9, 0x155a565e, 0x18197087, 0x1cd86d30, 0x029f3d35, 0x065e2082, 0x0b1d065b, 0x0fdc1bec,
    0x3793a651, 0x3352bbe6, 0x3e119d3f, 0x3ad08088, 0x2497d08d, 0x2056cd3a, 0x2d15ebe3, 0x29d4f654,
    0xc5a92679, 0xc1683bce, 0xcc2b1d17, 0xc8ea00a0, 0xd6ad50a5, 0xd26c4d12, 0xdf2f6bcb, 0xdbee767c,
    0xe3a1cbc1, 0xe760d676, 0xea23f0af, 0xeee2ed18, 0xf0a5bd1d, 0xf464a0aa, 0xf9278673, 0xfde69bc4,
    0x89b8fd09, 0x8d79e0be, 0x803ac667, 0x84fbdbd0, 0x9abc8bd5, 0x9e7d9662, 0x933eb0bb, 0x97ffad0c,
    0xafb010b1, 0xab710d06, 0xa6322bdf, 0xa2f33668, 0xbcb4666d, 0xb8757bda, 0xb5365d03, 0xb1f740b4,
};

uint32_t data_center_string_hash(char16_t *string)
{
    uint32_t hash = 0;

    for (char16_t *current = string; *current; current++)
    {
        char16_t value = htole16(*current);

        for (size_t i = 0; i < 2; i++)
            hash = string_hash_table[(uint8_t)(hash ^ ((uint8_t *)&value)[i])] ^ hash >> 8;
    }

    return hash;
}

(Note that string_hash_table is the same as value_hash_table.)

Data Graph

The actual content in a data center file is stored as a directed acyclic graph, which is essentially XML serialized to a binary format.

Nodes

Each node is of the form:

struct DataCenterNode
{
    uint16_t name_index;
    uint8_t key_flags : 4;
    uint16_t key_index : 12;
    uint16_t attribute_count;
    uint16_t child_count;
    DataCenterAddress attribute_address;
    uint32_t padding_1;
    DataCenterAddress child_address;
    uint32_t padding_2;
};

name_index is a one-based index into the addresses region of the names table. If this value is 0, it indicates that this node has no name or associated data of any kind, and should be disregarded; in this case, all other fields of the node should be considered undefined. Such nodes are usually incidental leftovers in official data center files and need not be present.

key_flags is 0 in official data center files. It may have a combination of the following values:

enum DataCenterKeyFlags : uint8_t
{
    DATA_CENTER_KEY_FLAGS_NONE = 0b0000,
    DATA_CENTER_KEY_FLAGS_UNCACHED = 0b0001,
};

If DATA_CENTER_KEY_FLAGS_UNCACHED is set, the results of a query against this node will not be cached by the client.

key_index is a zero-based index into the keys region.

attribute_count and child_count indicate how many attributes and child nodes should be read for this node, respectively. If a count field is 0, then the corresponding address field should be considered undefined, though it will usually be 65535:65535 in official data center files.

attribute_address is an address into the attributes region. attribute_count attributes should be read at this address. These attributes must be sorted by their name index in ascending order.

child_address is an address into the node region. child_count nodes should be read at this address. These children must be sorted first by their name index, then by the values of key attributes (if any), in ascending order. Note that the sort must be stable since the order of multiple sibling nodes with the same name can be significant for the interpretation of the data.

padding_1 and padding_2 should be considered undefined. They were added in the 64-bit data center format, and are not present in the 32-bit format.

The root node of the data graph must be located at the address 0:0. It must have the name __root__ and have zero attributes.

Keys

Keys are used to signal to a data center reading layer (e.g. in the client) that certain attributes of a node will be used frequently for lookups. The reading layer can then decide to construct an optimized lookup table for those specific paths in the graph, transparently making those lookups faster. It is effectively a trade-off between speed and memory usage.

struct DataCenterKey
{
    uint16_t name_index_1;
    uint16_t name_index_2;
    uint16_t name_index_3;
    uint16_t name_index_4;
};

name_index_1 and friends are one-based indexes into the addresses region of the names table. A value of 0 indicates that the field does not define a key. A key definition can specify between zero and four keys. These fields may not refer to the special __value__ attribute.

There need not be any keys defined in a data center file at all, but the client will be very slow without certain key definitions. At minimum, a data center file must contain a key definition at index 0 with all fields 0 (i.e. with no keys) which all nodes can point to by default.

Attributes

Each node in the data graph has zero or more attributes, which are name/value pairs. They are of the form:

struct DataCenterAttribute
{
    uint16_t name_index;
    DataCenterTypeCode type_code : 2;
    uint16_t extended_code : 14;
    union
    {
        int32_t i;
        bool b;
        float f;
        DataCenterAddress a;
    } value;
    uint32_t padding_1;
};

name_index is a one-based index into the addresses region of the names table.

type_code specifies the kind of value the attribute holds. Valid values are as follows:

enum DataCenterTypeCode : uint8_t
{
    DATA_CENTER_TYPE_CODE_INT = 1,
    DATA_CENTER_TYPE_CODE_FLOAT = 2,
    DATA_CENTER_TYPE_CODE_STRING = 3,
};

extended_code specifies extra information based on the value of type_code:

If type_code is DATA_CENTER_TYPE_CODE_INT, then the lowest bit of extended_code is set if the attribute's value should be considered a Boolean, meaning that value can only be 1 (true) or 0 (false). Either way, the higher bits are 0.
If type_code is DATA_CENTER_TYPE_CODE_FLOAT, then extended_code is 0.
If type_code is DATA_CENTER_TYPE_CODE_STRING, then extended_code is given by the expression data_center_value_hash(value) where value is the string value.

value holds the attribute value and is interpreted according to type_code and extended_code. In the case of DATA_CENTER_TYPE_CODE_STRING, the a field holds an address into the data region of the values table. For other type codes, the value is written directly and is accessed through the i, b, or f fields.

padding_1 should be considered undefined. It was added in the 64-bit data center format, and is not present in the 32-bit format.

Some nodes will have a special attribute named __value__. In XML terms, this represents the text of a node. For example, <Foo>bar</Foo> would be serialized to a node called Foo containing an attribute named __value__ with the string value bar. It is worth noting that a node can have both text and child nodes, such as Foo in this example:

<Foo>
    bar

    <Baz>
        ...
    </Baz>
</Foo>

Note that the __value__ attribute, if present, may only be a string.

Value Hash

The data_center_value_hash function uses a bizarre variant of CRC32 combined with a minimal effort to ignore the casing of characters. It is defined as follows:

const uint32_t value_hash_table[256] =
{
    0x00000000, 0x04c11db7, 0x09823b6e, 0x0d4326d9, 0x130476dc, 0x17c56b6b, 0x1a864db2, 0x1e475005,
    0x2608edb8, 0x22c9f00f, 0x2f8ad6d6, 0x2b4bcb61, 0x350c9b64, 0x31cd86d3, 0x3c8ea00a, 0x384fbdbd,
    0x4c11db70, 0x48d0c6c7, 0x4593e01e, 0x4152fda9, 0x5f15adac, 0x5bd4b01b, 0x569796c2, 0x52568b75,
    0x6a1936c8, 0x6ed82b7f, 0x639b0da6, 0x675a1011, 0x791d4014, 0x7ddc5da3, 0x709f7b7a, 0x745e66cd,
    0x9823b6e0, 0x9ce2ab57, 0x91a18d8e, 0x95609039, 0x8b27c03c, 0x8fe6dd8b, 0x82a5fb52, 0x8664e6e5,
    0xbe2b5b58, 0xbaea46ef, 0xb7a96036, 0xb3687d81, 0xad2f2d84, 0xa9ee3033, 0xa4ad16ea, 0xa06c0b5d,
    0xd4326d90, 0xd0f37027, 0xddb056fe, 0xd9714b49, 0xc7361b4c, 0xc3f706fb, 0xceb42022, 0xca753d95,
    0xf23a8028, 0xf6fb9d9f, 0xfbb8bb46, 0xff79a6f1, 0xe13ef6f4, 0xe5ffeb43, 0xe8bccd9a, 0xec7dd02d,
    0x34867077, 0x30476dc0, 0x3d044b19, 0x39c556ae, 0x278206ab, 0x23431b1c, 0x2e003dc5, 0x2ac12072,
    0x128e9dcf, 0x164f8078, 0x1b0ca6a1, 0x1fcdbb16, 0x018aeb13, 0x054bf6a4, 0x0808d07d, 0x0cc9cdca,
    0x7897ab07, 0x7c56b6b0, 0x71159069, 0x75d48dde, 0x6b93dddb, 0x6f52c06c, 0x6211e6b5, 0x66d0fb02,
    0x5e9f46bf, 0x5a5e5b08, 0x571d7dd1, 0x53dc6066, 0x4d9b3063, 0x495a2dd4, 0x44190b0d, 0x40d816ba,
    0xaca5c697, 0xa864db20, 0xa527fdf9, 0xa1e6e04e, 0xbfa1b04b, 0xbb60adfc, 0xb6238b25, 0xb2e29692,
    0x8aad2b2f, 0x8e6c3698, 0x832f1041, 0x87ee0df6, 0x99a95df3, 0x9d684044, 0x902b669d, 0x94ea7b2a,
    0xe0b41de7, 0xe4750050, 0xe9362689, 0xedf73b3e, 0xf3b06b3b, 0xf771768c, 0xfa325055, 0xfef34de2,
    0xc6bcf05f, 0xc27dede8, 0xcf3ecb31, 0xcbffd686, 0xd5b88683, 0xd1799b34, 0xdc3abded, 0xd8fba05a,
    0x690ce0ee, 0x6dcdfd59, 0x608edb80, 0x644fc637, 0x7a089632, 0x7ec98b85, 0x738aad5c, 0x774bb0eb,
    0x4f040d56, 0x4bc510e1, 0x46863638, 0x42472b8f, 0x5c007b8a, 0x58c1663d, 0x558240e4, 0x51435d53,
    0x251d3b9e, 0x21dc2629, 0x2c9f00f0, 0x285e1d47, 0x36194d42, 0x32d850f5, 0x3f9b762c, 0x3b5a6b9b,
    0x0315d626, 0x07d4cb91, 0x0a97ed48, 0x0e56f0ff, 0x1011a0fa, 0x14d0bd4d, 0x19939b94, 0x1d528623,
    0xf12f560e, 0xf5ee4bb9, 0xf8ad6d60, 0xfc6c70d7, 0xe22b20d2, 0xe6ea3d65, 0xeba91bbc, 0xef68060b,
    0xd727bbb6, 0xd3e6a601, 0xdea580d8, 0xda649d6f, 0xc423cd6a, 0xc0e2d0dd, 0xcda1f604, 0xc960ebb3,
    0xbd3e8d7e, 0xb9ff90c9, 0xb4bcb610, 0xb07daba7, 0xae3afba2, 0xaafbe615, 0xa7b8c0cc, 0xa379dd7b,
    0x9b3660c6, 0x9ff77d71, 0x92b45ba8, 0x9675461f, 0x8832161a, 0x8cf30bad, 0x81b02d74, 0x857130c3,
    0x5d8a9099, 0x594b8d2e, 0x5408abf7, 0x50c9b640, 0x4e8ee645, 0x4a4ffbf2, 0x470cdd2b, 0x43cdc09c,
    0x7b827d21, 0x7f436096, 0x7200464f, 0x76c15bf8, 0x68860bfd, 0x6c47164a, 0x61043093, 0x65c52d24,
    0x119b4be9, 0x155a565e, 0x18197087, 0x1cd86d30, 0x029f3d35, 0x065e2082, 0x0b1d065b, 0x0fdc1bec,
    0x3793a651, 0x3352bbe6, 0x3e119d3f, 0x3ad08088, 0x2497d08d, 0x2056cd3a, 0x2d15ebe3, 0x29d4f654,
    0xc5a92679, 0xc1683bce, 0xcc2b1d17, 0xc8ea00a0, 0xd6ad50a5, 0xd26c4d12, 0xdf2f6bcb, 0xdbee767c,
    0xe3a1cbc1, 0xe760d676, 0xea23f0af, 0xeee2ed18, 0xf0a5bd1d, 0xf464a0aa, 0xf9278673, 0xfde69bc4,
    0x89b8fd09, 0x8d79e0be, 0x803ac667, 0x84fbdbd0, 0x9abc8bd5, 0x9e7d9662, 0x933eb0bb, 0x97ffad0c,
    0xafb010b1, 0xab710d06, 0xa6322bdf, 0xa2f33668, 0xbcb4666d, 0xb8757bda, 0xb5365d03, 0xb1f740b4,
};

uint16_t data_center_value_hash(char16_t *string)
{
    uint32_t hash = 0;

    for (char16_t *current = string; *current; current++)
    {
        char16_t value = *current;

        if (value == u'\x9c')
            value = u'\x8c';
        else if (value == u'\xff')
            value = u'\x9f';
        else if (value == u'\x151')
            value = u'\x150';
        else if (((value >= u'a' && value <= u'z') || (value >= u'\xe0' && value <= u'\xfe')) &&
            !(value == u'\xf0' || value == u'\xf7'))
            value = c - u' ';

        value = htole16(value);

        for (size_t i = 0; i < 2; i++)
            hash = value_hash_table[(uint8_t)(hash ^ ((uint8_t *)&value)[i])] ^ hash >> 8;
    }

    return hash & 0x3fff;
}

(Note that value_hash_table is the same as string_hash_table.)

Package Format

TODO: Describe the package file format (GPK).

Resource Container Format

This page describes the encrypted resource container format used by TERA.

C/C++-like primitive types and structs will be used.
Integers (uint8_t, int8_t, uint16_t, int16_t, etc) are little endian.
Characters (i.e. char16_t) are UTF-16 and little endian.
Strings (i.e. u16string) are a series of valid char16_t characters followed by a NUL character.
Fields are laid out in the declared order with no implied padding anywhere.

Encryption

Portions of resource container files are encrypted with a 256-bit XOR cipher.

The encryption key can be extracted from a TERA client. This can be done with a running TERA process or an unpacked executable. The key appears to be static; it does not change with each client build.

Physical Structure

The overall structure can be described like this:

struct ResourceContainerFile
{
    uint8_t data[];
    ResourceContainerDirectory directory;
    ResourceContainerFooter footer;
};

Reading a resource container file requires knowing the full size of the file in advance. A reader must first seek to the footer, read it, then seek to the directory, read and decrypt it, and finally seek to the data where each file entry can be read and decrypted according to the offsets in the directory.

File Footer

A resource container file ends with this footer:

struct ResourceContainerFooter
{
    uint32_t directory_size;
    uint32_t magic;
};

directory_size is the size of the directory. This enables a reader to know how far back it must seek in order to read the directory correctly.

magic must be 0x01001fff.

Launcher/Client Protocol

This page describes the communication protocol employed by launcher.exe, Tl.exe, and TERA.exe.

C/C++-like primitive types and structs will be used.
Integers (uint8_t, int8_t, uint16_t, int16_t, etc) are little endian.
Characters (i.e. char8_t and char16_t) are UTF-8 and UTF-16 respectively, and little endian.
Strings (i.e. u8string and u16string) are a series of valid char8_t and char16_t characters respectively, followed by a NUL character.
Fields are laid out in the declared order with no implied padding anywhere.

Note that strings are not always NUL-terminated. For this reason, the document will explicitly call out whether a NUL terminator is present.

Communication

The role of each program is as follows:

launcher.exe: The publisher-specific game launcher which performs authentication and server list URL resolution. Serves requests from Tl.exe and receives game events.
Tl.exe: The (mostly) publisher-agnostic game launcher which requests authentication data and the server list URL from launcher.exe. Serves requests and forwards game events from TERA.exe.
TERA.exe: The actual game client. Sends requests and game events to Tl.exe.

These programs all communicate via window messages, specifically WM_COPYDATA. The dwData field specifies the message ID, while lpData and cbData contain the payload pointer and length.

`Tl.exe` -> `launcher.exe` Messages

Messages sent between Tl.exe and launcher.exe consist of text encoded as UTF-8. A NUL terminator is present in most messages, but not all. Responses from launcher.exe should always use the same message ID that Tl.exe used in the corresponding request message. Notably, only the Hello Handshake and Game Event Notification messages have a static ID; the request messages use a message counter as the message ID, so the contents must be parsed to understand them.

Hello Handshake (`0x0dbadb0a`)

The protocol starts off with a handshake sent from Tl.exe. This message contains the NUL-terminated string Hello!!.

launcher.exe should respond with a NUL-terminated Hello!! or Steam!! to indicate the method of authentication in use. The former uses classic authentication while the latter uses Steam. (Steam authentication will not be documented here.)

Game Event Notification (`0x0`)

Tl.exe will occasionally notify launcher.exe of various game events sent by TERA.exe. The format of these messages can be described with the following regular expressions:

^csPopup$: Signals that launcher.exe should open the customer support website (e.g. in the default Web browser).
^gameEvent$(\d+)$$: Indicates some kind of notable action taken by the user in the game. The value in parentheses is a code specifying the type of event; the possible values are documented in the section on TERA.exe messages.
^endPopup$(\d+)$$: Indicates that the client has exited. The value in parentheses is an exit reason code (not the same as the process exit code); the possible values are documented in the section on TERA.exe messages.
^promoPopup$\d+$$: The exact purpose of this notification is currently unknown.

launcher.exe should not respond to these messages.

Server List URL Request

Tl.exe will request the server list URL from launcher.exe. This message does not have a static message ID. The message contains the NUL-terminated string slsurl.

launcher.exe should respond with the NUL-terminated server list URL.

The Web server at the URL should be prepared to receive an HTTP GET request, potentially with a query string (which can be ignored). The response should use the application/xml media type. Note that Tl.exe does not support chunked transfer encoding.

Server List Schema

The response received from the server list URL should be in XML format. It can be described with the following XSD schema:

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns="https://vezel.dev/novadrop/tl/ServerList"
            targetNamespace="https://vezel.dev/novadrop/tl/ServerList"
            elementFormDefault="qualified">
    <xsd:complexType name="serverlist">
        <xsd:sequence>
            <xsd:element name="server" type="serverlist_server" maxOccurs="unbounded" />
        </xsd:sequence>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server">
        <xsd:sequence>
            <xsd:element name="id" type="serverlist_server_id" />
            <xsd:element name="ip" type="serverlist_server_ip" />
            <xsd:element name="port" type="unsignedShort" />
            <xsd:element name="category" type="serverlist_server_category" />
            <xsd:element name="name" type="serverlist_server_name" />
            <xsd:element name="crowdness" type="serverlist_server_crowdness" />
            <xsd:element name="open" type="serverlist_server_open" />
            <xsd:element name="permission_mask" type="serverlist_server_permission_mask" />
            <xsd:element name="server_stat" type="serverlist_server_server_stat" />
            <xsd:element name="popup" type="xsd:string" />
            <xsd:element name="language" type="xsd:string" />
        </xsd:sequence>
    </xsd:complexType>

    <xsd:simpleType name="serverlist_server_id">
        <xsd:restriction base="xsd:unsignedInt">
            <xsd:minInclusive value="1" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:simpleType name="serverlist_server_ip">
        <xsd:restriction base="xsd:string">
            <xsd:pattern value="\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:complexType name="serverlist_server_category">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="sort" type="xsd:unsignedInt" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server_name">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="raw_name" type="xsd:string" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server_crowdness">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="sort" type="xsd:unsignedInt" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server_open">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="sort" type="xsd:unsignedInt" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:simpleType name="serverlist_server_permission_mask">
        <xsd:restriction base="xsd:string">
            <xsd:pattern value="0x[0-9A-Fa-f]{8}" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:simpleType name="serverlist_server_server_stat">
        <xsd:restriction base="xsd:string">
            <xsd:pattern value="0x[0-9A-Fa-f]{8}" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:element name="serverlist" type="serverlist" />
</xsd:schema>

Authentication Info Request

Tl.exe will request JSON-encoded authentication data from launcher.exe. This message does not have a static message ID. The message contains one of several NUL-terminated strings.

gamestr is only sent when the game is initially launched. This request asks for the full authentication data.

ticket, last_svr, and char_cnt are only sent when returning to the server list from an arbiter server.

ticket: Requests the authentication ticket. This can be the same ticket as before, but launcher.exe may also choose to authenticate anew and retrieve a fresh ticket.
last_svr: Requests the ID of the last server the the account connected to.
char_cnt: Requests character counts for each server in the server list.

launcher.exe should respond with the NUL-terminated JSON payload.

Authentication Info Schema

The authentication JSON response can be described with the following JSON Schema definition:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://vezel.dev/novadrop/tl/authentication.schema.json",
  "title": "AuthenticationInfo",
  "type": "object",
  "properties": {
    "result-message": {
      "type": "string"
    },
    "result-code": {
      "type": "integer",
      "minimum": 100,
      "maximum": 599
    },
    "account_bits": {
      "type": "string",
      "pattern": "^0x[0-9A-Fa-f]{10}$"
    },
    "ticket": {
      "type": "string"
    },
    "last_connected_server_id": {
      "type": "integer",
      "minimum": 0
    },
    "access_level": {
      "type": "integer",
      "minimum": 0
    },
    "master_account_name": {
      "type": "string"
    },
    "chars_per_server": {
      "anyOf": [
        {
          "type": "object",
          "properties": {},
          "additionalProperties": false
        },
        {
          "$ref": "#/$defs/chars_on_server"
        },
        {
          "type": "array",
          "items": {
            "$ref": "#/$defs/chars_on_server"
          }
        }
      ]
    },
    "user_permission": {
      "type": "integer",
      "minimum": 0
    },
    "game_account_name": {
      "type": "string"
    }
  },
  "required": [
    "result-code",
    "master_account_name",
    "game_account_name"
  ],
  "$defs": {
    "chars_on_server": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string",
          "pattern": "^[1-9]\\d*$"
        },
        "char_count": {
          "type": "string",
          "pattern": "^\\d+$"
        }
      },
      "additionalProperties": false,
      "required": [
        "id",
        "char_count"
      ]
    }
  }
}

Some properties are only required depending on the message:

gamestr: All properties are required.
ticket: The ticket property is required.
last_svr: The last_connected_server_id property is required.
char_cnt: The chars_per_server property is required.

result-message, account_bits, access_level, and user_permission are completely optional.

Web URL Request

Tl.exe will request a URL to be opened in the client's embedded Web browser. This message does not have a static message ID and is not NUL-terminated.

The message can be described by the regular expression ^getWebLinkUrl$(\d+),(.*)$$. The first group is the ID of a UIWindow node under the CoherentGTWeb data center sheet, and the second group is a set of arguments specific to that URL.

launcher.exe should respond with a URL to be opened (without a NUL terminator), or an empty payload to reject the request.

`TERA.exe` -> `Tl.exe` Messages

Messages sent between TERA.exe and Tl.exe use a simple binary protocol. All messages have static message IDs; responses have different IDs from requests.

Account Name Request (`0x1`)

TERA.exe will request the (game) account name from Tl.exe.

struct LauncherAccountNameRequest
{
};

Account Name Response (`0x2`)

Tl.exe should respond with the account name.

struct LauncherAccountNameResponse
{
    u16string account_name;
};

account_name is the name of the game account. It is not NUL-terminated.

Session Ticket Request (`0x3`)

TERA.exe will request the session ticket from Tl.exe.

struct LauncherSessionTicketRequest
{
};

Session Ticket Response (`0x4`)

Tl.exe should respond with the session ticket.

struct LauncherAccountNameResponse
{
    u8string session_ticket;
};

session_ticket is the authentication session ticket. It is not NUL-terminated.

Server List Request (`0x5`)

TERA.exe will request the server list from Tl.exe.

struct LauncherServerListRequest
{
    LauncherServerListSortCriteria sort_criterion;
};

sorting specifies how Tl.exe should sort the server list. Valid values are as follows:

enum LauncherServerListSortCriteria : int32_t
{
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_NONE = -1,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_CHARACTERS = 0,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_CATEGORY = 1,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_NAME = 2,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_POPULATION = 4,
};

A few notes on sorting:

The sort should be stable.
LAUNCHER_SERVER_LIST_SORT_CRITERIA_NONE indicates that Tl.exe is free to sort the list arbitrarily.
The resultant list should be maintained between requests and further sorted on each request, unless LAUNCHER_SERVER_LIST_SORTING_NONE is sent to reset the sorting.
If the same sorting is requested multiple times and is any sorting other than LAUNCHER_SERVER_LIST_SORT_CRITERIA_NONE, the list should simply be reversed without applying sorting.

Server List Response (`0x6`)

Tl.exe should respond with the server list encoded with Protocol Buffers.

struct LauncherServerListResponse
{
    uint8_t data[];
};

Server List Structures

The server list response can be described with the following message definitions:

syntax = "proto2";

message ServerList
{
    message ServerInfo
    {
        required fixed32 id = 1;
        required bytes name = 2;
        required bytes category = 3;
        required bytes title = 4;
        required bytes queue = 5;
        required bytes population = 6;
        required fixed32 address = 7;
        required fixed32 port = 8;
        required fixed32 available = 9;
        required bytes unavailable_message = 10;
        optional bytes host = 11;
    }

    repeated ServerInfo servers = 1;
    required fixed32 last_server_id = 2;
    required fixed32 sort_criterion = 3;
}

A few notes on these definitions:

They will not work correctly as proto3 due to required semantics.
bytes fields are really u16string without NUL terminators.
id must be a positive (non-zero) value.
port should be in the uint16_t range.
available is really a bool, so only the values 0 and 1 are allowed.
Either address or host must be set; not neither and not both.
- For address, a value of 0 has 'not set' semantics since the field is not marked optional.
sort_criterion should be the LauncherServerListSortCriteria value that was sent in the request.

Enter Lobby/World Notification (`0x7`)

TERA.exe will notify Tl.exe when entering the lobby (i.e. successfully connecting to an arbiter server) or when entering the world on a particular character.

struct LauncherEnterLobbyNotification
{
};

struct LauncherEnterWorldNotification
{
    u16string character_name;
};

The two cases can be distinguished by looking at the payload length.

character_name is the NUL-terminated name of the character that the user is entering the world on.

Voice Chat Requests

There is a set of requests for interacting with TeamSpeak:

Create Room (0x8)
Join Room (0xa)
Leave Room (0xc)
Set Volume (0x13)
Set Microphone (0x14)
Silence User (0x15)

These messages were only present in some regions and were likely never actually used. It is currently unknown what their payloads contain.

Voice Chat Responses

The following responses exist for the above TeamSpeak requests:

Create Room Result (0x9)
Join Room Result (0xb)
Leave Room Result (0xd)

Open Website Command (`0x19`)

TERA.exe asks Tl.exe to open a website in the default Web browser.

struct LauncherOpenWebsiteCommand
{
    uint32_t id;
};

id specifies the kind of website that should be opened. The possible values are currently unknown.

Web URL Request (`0x1a`)

This request is the TERA.exe equivalent of the request sent by Tl.exe to launcher.exe.

struct LauncherWebURLRequest
{
    uint32_t id;
    u16string arguments;
};

id refers to a UIWindow node under the CoherentGTWeb data center sheet.

arguments specifies the NUL-terminated arguments specific to the URL.

Web URL Response (`0x1b`)

struct LauncherWebURLResponse
{
    uint32_t id;
    u16string url;
};

id is the same value that was sent in the request.

url is the NUL-terminated URL to open. It can be the empty string or the special string | to indicate that no URL should be opened.

Game Start Notification (`0x3e8`)

TERA.exe will notify Tl.exe that it has launched.

struct LauncherGameStartNotification
{
    uint32_t source_revision;
    uint32_t unknown_1;
    u16string windows_account_name;
};

source_revision is the SrcRegVer value from the client's ReleaseRevision.txt file.

The meaning of unknown_1 is currently unknown.

windows_account_name is the NUL-terminated name of the current Windows user account.

Game Event Notification (`0x3e9` - `0x3f8`)

TERA.exe will occasionally notify Tl.exe of various notable actions taken by the user.

struct LauncherGameEventNotification
{
    LauncherGameEvent event;
};

event specifies the kind of event that occurred. Valid values are as follows:

enum LauncherGameEvent : uint32_t
{
    LAUNCHER_GAME_EVENT_ENTERED_INTO_CINEMATIC = 1001,
    LAUNCHER_GAME_EVENT_ENTERED_SERVER_LIST = 1002,
    LAUNCHER_GAME_EVENT_ENTERING_LOBBY = 1003,
    LAUNCHER_GAME_EVENT_ENTERED_LOBBY = 1004,
    LAUNCHER_GAME_EVENT_ENTERING_CHARACTER_CREATION = 1005,
    LAUNCHER_GAME_EVENT_LEFT_LOBBY = 1006,
    LAUNCHER_GAME_EVENT_DELETED_CHARACTER = 1007,
    LAUNCHER_GAME_EVENT_CANCELED_CHARACTER_CREATION = 1008,
    LAUNCHER_GAME_EVENT_ENTERED_CHARACTER_CREATION = 1009,
    LAUNCHER_GAME_EVENT_CREATED_CHARACTER = 1010,
    LAUNCHER_GAME_EVENT_ENTERED_WORLD = 1011,
    LAUNCHER_GAME_EVENT_FINISHED_LOADING_SCREEN = 1012,
    LAUNCHER_GAME_EVENT_LEFT_WORLD = 1013,
    LAUNCHER_GAME_EVENT_MOUNTED_PEGASUS = 1014,
    LAUNCHER_GAME_EVENT_DISMOUNTED_PEGASUS = 1015,
    LAUNCHER_GAME_EVENT_CHANGED_CHANNEL = 1016,
};

Game Exit Notification (`0x3fc`)

TERA.exe will notify Tl.exe that it is exiting.

struct LauncherGameExitNotification
{
    uint32_t length;
    uint32_t code;
    LauncherGameExitReason reason;
};

length is the length of the payload. It must be 12.

code is the exit code of the TERA.exe process. This can be 0 or 1.

reason provides a more specific exit reason. Some known values are as follows:

enum LauncherGameExitReason : uint32_t
{
    LAUNCHER_GAME_EXIT_REASON_SUCCESS = 0x0,
    LAUNCHER_GAME_EXIT_REASON_INVALID_DATA_CENTER = 0x6,
    LAUNCHER_GAME_EXIT_REASON_CONNECTION_DROPPED = 0x8,
    LAUNCHER_GAME_EXIT_REASON_INVALID_AUTHENTICATION_INFO = 0x9,
    LAUNCHER_GAME_EXIT_REASON_OUT_OF_MEMORY = 0xa,
    LAUNCHER_GAME_EXIT_REASON_SHADER_MODEL_3_UNAVAILABLE = 0xc,
    LAUNCHER_GAME_EXIT_REASON_SPEED_HACK_DETECTED = 0x10,
    LAUNCHER_GAME_EXIT_REASON_UNSUPPORTED_VERSION = 0x13,
    LAUNCHER_GAME_EXIT_REASON_ALREADY_ONLINE = 0x106,
};

(The above enumeration is an incomplete list.)

Game Crash Notification (`0x3fd`)

TERA.exe will notify Tl.exe that it has crashed (e.g. because of a memory access violation). Note that, since a crash could mean things are arbitrarily broken, this message may not be produced if the crash is sufficiently severe.

struct LauncherGameCrashNotification
{
    u16string details;
};

details contains various details about the crash such as the instruction pointer and exception type. It is not NUL-terminated.

Anti-Cheat Event Notifications (`0x3fe` - `0x400`)

TERA.exe will notify Tl.exe of various events relating to the anti-cheat module (e.g. XIGNCODE3 or GameGuard). Note that only some regions have a client build with an anti-cheat module.

Anti-Cheat Starting Notification (`0x3fe`)

TERA.exe will notify Tl.exe that the anti-cheat module is starting.

struct LauncherAntiCheatStartingNotification
{
};

Anti-Cheat Started Notification (`0x3ff`)

TERA.exe will notify Tl.exe that the anti-cheat module has successfully started.

struct LauncherAntiCheatStartedNotification
{
};

Anti-Cheat Error Notification (`0x400`)

TERA.exe will notify Tl.exe that the anti-cheat module failed to start.

struct LauncherAntiCheatStartedNotification
{
    uint64_t error;
};

error contains the error code from the anti-cheat module.

Open Support Website Command (`0x401`)

TERA.exe asks Tl.exe to open the customer support website in the default Web browser.

struct LauncherOpenSupportWebsiteCommand
{
};

Unknown Command 1027 (`0x403`)

The exact purpose of this message is currently unknown.

Network Protocol

This page describes TERA's network protocol, which is built on top of TCP. It is a structured protocol that can be serialized and deserialized automatically based on uniform packet definitions; no special serialization logic is required for any fields.

C/C++-like primitive types and structs will be used.
bool is equivalent to uint8_t but only allows the values true (1) and false (0).
Integers (uint8_t, int8_t, uint16_t, int16_t, etc) are little endian.
offset_t values are uint16_t indexes into a packet, including the header.
float and double are IEEE 754 binary32 and binary64, respectively.
Characters (i.e. char16_t) are UTF-16 and little endian.
Strings (i.e. u16string) are a series of valid char16_t characters followed by a NUL character.
Fields are laid out in the declared order with no implied padding anywhere.

Encryption

TODO: Describe the (insecure) key exchange and the PIKE algorithm.

Packet Header

A packet starts with a simple header:

struct PacketHeader
{
    uint16_t length;
    uint16_t code;
};

length specifies the full length of the packet, including the header. Thus, the maximum length of a packet is 65535 bytes (or 65531 bytes for the payload). For certain large packets (e.g. achievements and inventory), the game works around this by sending 'continuation' packets after the first packet.

code is the operation code. This tells the client or server what the structure of the payload is.

Packet Body

The packet body consists of a series of fields. Some examples:

struct CJoinPrivateChannelPacket
{
    PacketHeader header;
    u16string channel_name;
    uint16_t password;
};

struct CEditPrivateChannelPacket
{
    struct
    {
        uint32_t player_id;
    } members[];
    u16string channel_name;
    uint16_t password;
};

Fields are written in the order that they are declared. However, complex fields (strings, object arrays, and byte arrays) are written as offset_t values that point to the actual data elsewhere in the payload. Primitive types such as integers, floating point numbers, and Boolean values are written in the obvious way as they appear.

Complex Types

Complex types are written after all primitive types in the current 'object' (be that the root packet body, or an object nested arbitrarily within arrays). At the place where the complex field appears, an offset_t value is written pointing to where the actual data for the field is written in the payload. For object arrays and byte arrays, this value is accompanied by a uint16_t value representing the number of elements in the array.

When there are multiple complex fields in an object, they are written at the end of the object in the order that they appear in the structure. For example, in CEditPrivateChannelPacket, the contents of the members array are written after password, and the channel_name string contents after that.

Strings

String pointers are represented as follows:

struct PacketStringPointer
{
    offset_t start;
};

When writing the string contents, the characters are written contiguously, followed by a NUL character.

Object Arrays

Object array pointers are represented as follows:

struct PacketObjectArrayPointer
{
    uint16_t count;
    offset_t start;
};

Each element within the array has an object pointer that links to the next element:

struct PacketObjectPointer
{
    offset_t here;
    offset_t next;
};

start points to a PacketObjectPointer, which is immediately followed by the contents of the first element. The next pointer points to another PacketObjectPointer, which is immediately followed by the contents of the second element, and so on. This continues count times. The next value for the last element is 0. In each element, here is just a pointer to itself; it is not clear what purpose it serves.

Due to the way that arrays are serialized, in theory, it would be possible to spread the elements all over the place in a payload, in whatever way would make the most sense for compactness and locality. In practice, the client and official servers almost always do the straightforward thing, with only a few curious (and seemingly nonsensical) exceptions. Regardless, neither party actually cares how arrays are laid out for the purposes of serialization.

Byte Arrays

Byte array pointers are represented as follows:

struct PacketByteArrayPointer
{
    offset_t start;
    uint16_t count;
};

Note that count comes after start, unlike object arrays.

Byte arrays are serialized in a more compact fashion than object arrays: start points to an area in the payload containing count bytes, making up the contents of the byte array. There are no pointer values to follow for each individual element.

Tools

novadrop-dc

novadrop-dc <command> <arguments...> [options...]

The novadrop-dc tool allows manipulation of TERA's data center files. It supports the following tasks:

Extraction of data center files to XML data sheets with corresponding XSD schemas.
Reasonably accurate inference of XSD schemas from a packed data center file.
Validation of data sheets against schemas, preventing many mistakes when authoring data sheets.
Packing of data sheets in accordance with type and key information in schemas to a fresh data center file usable by the client.
Format integrity verification of data center files, optionally with strict compliance checks.
Support for various iterations of the data center format throughout the game's history.

In general, novadrop-dc is quite fast: It exploits as many cores as are available for parallel extraction, validation, and packing. On a modern system, unpacking an official data center file takes around 15 seconds, while packing the resulting data sheets takes around 25 seconds.

novadrop-dc pack

novadrop-dc pack <input> <output> [options...]

Packs the data sheets in a directory to a data center file. If validation of the data sheets fails during packing, a list of problems will be printed and the exit code will be non-zero.

The input argument specifies a directory containing data sheets and schemas. The output argument specifies the path of the resulting data center file.

Option

Description

--format <format>

Specifies the data center format variant (defaults to V6X64).

--revision <value>

Specifies the data tree revision number (defaults to latest known revision).

--compression <level>

Specifies a compression level (defaults to Optimal).

--encryption-key <key>

Specifies an encryption key (defaults to the latest known key).

--encryption-iv <iv>

Specifies an encryption IV (defaults to the latest known IV).

novadrop-dc repack

novadrop-dc repack <input> <output> [options...]

Repacks a data center file without unpacking to disk. This command is primarily useful for development of novadrop-dc.

The input argument specifies the input data center file. The output argument specifies the path of the resulting data center file.

Option

Description

--decryption-key <key>

Specifies a decryption key (defaults to the latest known key).

--decryption-iv <iv>

Specifies a decryption IV (defaults to the latest known IV).

--architecture <architecture>

Specifies the data center format architecture (defaults to X64).

--strict

Enables strict format compliance checks while reading the input file.

--format <format>

Specifies the data center format variant (defaults to V6X64).

--revision <value>

Specifies the data tree revision number (defaults to latest known revision).

--compression <level>

Specifies a compression level (defaults to Optimal).

--encryption-key <key>

Specifies an encryption key (defaults to the latest known key).

--encryption-iv <iv>

Specifies an encryption IV (defaults to the latest known IV).

novadrop-dc schema

novadrop-dc schema <input> <output> [options...]

Performs best-effort inference of the schemas for the data tree in a data center file. This may be useful when working with older data centers for which Novadrop's included schemas are not correct.

Please note that, regardless of whether the Conservative or Aggressive inference strategy is used, the generated schemas will require a human touch after they have been generated. Among other things, this command cannot tell whether a given path in the tree with repeated node names represents a truly recursive structure.

The input argument specifies the input data center file. The output argument specifies the path of the directory to write inferred schemas to.

Option

Description

--decryption-key <key>

Specifies a decryption key (defaults to the latest known key).

--decryption-iv <iv>

Specifies a decryption IV (defaults to the latest known IV).

--architecture <architecture>

Specifies the data center format architecture (defaults to X64).

--strict

Enables strict format compliance checks while reading the input file.

--strategy <level>

Specifies a schema inference strategy (defaults to Conservative).

--subdirectories

Enables using output subdirectories based on data sheet names.

novadrop-dc unpack

novadrop-dc unpack <input> <output> [options...]

Unpacks the data tree in a data center file to a series of data sheets and schemas in a directory. This command is primarily intended for unpacking official data center files into a form that is easily maintainable by humans.

The input argument specifies the input data center file. The output argument specifies the path of the directory to extract data sheets and schemas to.

Option

Description

--decryption-key <key>

Specifies a decryption key (defaults to the latest known key).

--decryption-iv <iv>

Specifies a decryption IV (defaults to the latest known IV).

--architecture <architecture>

Specifies the data center format architecture (defaults to X64).

--strict

Enables strict format compliance checks while reading the input file.

novadrop-dc validate

novadrop-dc validate <input>

Validates the data sheets in a directory against their schemas. If validation of the data sheets fails, a list of problems will be printed and the exit code will be non-zero.

The input argument specifies a directory containing data sheets and schemas.

novadrop-dc verify

novadrop-dc verify <input> [options...]

Verifies the format integrity of a data center file. This means traversing the entire data tree, reading all nodes and attributes, and ensuring that no errors occur.

The input argument specifies the input data center file.

Option

Description

--decryption-key <key>

Specifies a decryption key (defaults to the latest known key).

--decryption-iv <iv>

Specifies a decryption IV (defaults to the latest known IV).

--architecture <architecture>

Specifies the data center format architecture (defaults to X64).

--strict

Enables strict format compliance checks while reading the input file.

novadrop-rc

The novadrop-rc tool allows manipulation of TERA's resource container files. It supports the following tasks:

Extraction of files contained within resource container files.
Packing of files to a fresh resource container file usable by the client.
Format integrity verification of resource container files, optionally with strict compliance checks.

novadrop-rc pack

Packs the files in a directory to a resource container file.

The input argument specifies a directory containing files. The output argument specifies the path of the resulting resource container file.

Option

Description

novadrop-rc repack

Repacks a resource container file without unpacking to disk. This command is primarily useful for development of novadrop-rc.

The input argument specifies the input resource container file. The output argument specifies the path of the resulting resource container file.

Option

Description

novadrop-rc unpack

Unpacks the files in a resource container file to a directory.

The input argument specifies the input resource container file. The output argument specifies the path of the directory to extract files to.

Option

Description

novadrop-rc verify

Verifies the format integrity of a resource container file. This means loading all contained files into memory and verifying that no errors occur.

The input argument specifies the input resource container file.

Option

Description

Data Center Format

This page describes the encrypted and compressed data center format used by TERA.

C/C++-like primitive types, enums, structs, and unions will be used.
bool is equivalent to uint8_t but only allows the values true (1) and false (0).
Integers (uint8_t, int8_t, uint16_t, int16_t, etc) are little endian.
float and double are IEEE 754 binary32 and binary64, respectively.
Characters (i.e. char16_t) are UTF-16 and little endian.
Fields are laid out in the declared order with no implied padding anywhere.

Note that the format uses both zero-based and one-based array indexes in various, seemingly random places.

Encryption

Data center files are encrypted with the AES algorithm in CFB mode and with block, key, and feedback sizes all set to 128 bits. No padding is done for the final block.

Physical Structure

The overall structure can be described like this:

struct DataCenterFile
{
    DataCenterCompressionHeader compression_header;
    DataCenterHeader header;
    DataCenterSimpleRegion<DataCenterKey, false> keys;
    DataCenterSegmentedRegion<DataCenterAttribute> attributes;
    DataCenterSegmentedRegion<DataCenterNode> nodes;
    DataCenterStringTable<1024> values;
    DataCenterStringTable<512> names;
    DataCenterFooter footer;
};

Compression Header

After decryption, there is a small header of the form:

struct DataCenterCompressionHeader
{
    uint32_t uncompressed_size;
    uint16_t zlib_header;
};

All data immediately following this header is compressed with the Deflate algorithm.

uncompressed_size is the size of the data center file once inflated.

zlib_header is a zlib (§2.2) header. In official data center files, it is usually of the form 0x9c78, but it can be any valid zlib header.

File Header

After decompression, a data center file starts with this header:

struct DataCenterHeader
{
    uint32_t version;
    double timestamp;
    uint32_t revision;
    int16_t unknown_1;
    int16_t unknown_2;
    int32_t unknown_3;
    int32_t unknown_4;
    int32_t unknown_5;
};

version is currently 6. A past version 3 also existed. Note that this field has no distinguishing value for the 32-bit and 64-bit formats, but version 3 was only 32-bit.

timestamp is a Unix timestamp indicating when the file was produced.

File Footer

A data center file ends with this footer:

struct DataCenterFooter
{
    int32_t marker;
}

marker is always 0 and has no known purpose in the client.

Regions

Most of the content in data center files is arranged into regions, which may be segmented. The region structures used throughout the format are described here:

struct DataCenterSimpleRegion<typename T, bool off_by_one>
{
    uint32_t count;
    T elements[off_by_one ? count - 1 : count];
};

struct DataCenterSegmentedSimpleRegion<typename T, uint32_t count>
{
    DataCenterSimpleRegion<T, false> segments[count];
};

struct DataCenterRegion<typename T>
{
    uint32_t full_count;
    uint32_t used_count;
    T elements[full_count];
};

struct DataCenterSegmentedRegion<typename T>
{
    uint32_t count;
    DataCenterRegion<T> segments[count];
};

A DataCenterSegmentedSimpleRegion is mostly the same as a DataCenterSegmentedRegion, with the main difference being that it has a static amount of segments.

Addresses are frequently used to refer to elements within both types of segmented regions:

struct DataCenterAddress
{
    uint16_t segment_index;
    uint16_t element_index;
};

Here, segment_index is a zero-based index into the segments array of the segmented region, while element_index is a zero-based index into the elements array of the segment.

String Tables

All strings, whether they are names or values, are arranged into string tables, which are effectively used as hash tables by the client. A string table has the form:

struct DataCenterStringTable<uint32_t count>
{
    DataCenterSegmentedRegion<char16_t> data;
    DataCenterSegmentedSimpleRegion<DataCenterString, count> table;
    DataCenterSimpleRegion<DataCenterAddress, true> addresses;
};

A string entry in the table region looks like this:

struct DataCenterString
{
    uint32_t hash;
    uint32_t length;
    uint32_t index;
    DataCenterAddress address;
};

length is the length of the string in terms of characters, including the NUL character.

index is a one-based index into the string table's addresses region. The address at this index must match the address field exactly.

String Hash

The data_center_string_hash function is a bizarre variant of CRC32. It is defined as follows:

const uint32_t string_hash_table[256] =
{
    0x00000000, 0x04c11db7, 0x09823b6e, 0x0d4326d9, 0x130476dc, 0x17c56b6b, 0x1a864db2, 0x1e475005,
    0x2608edb8, 0x22c9f00f, 0x2f8ad6d6, 0x2b4bcb61, 0x350c9b64, 0x31cd86d3, 0x3c8ea00a, 0x384fbdbd,
    0x4c11db70, 0x48d0c6c7, 0x4593e01e, 0x4152fda9, 0x5f15adac, 0x5bd4b01b, 0x569796c2, 0x52568b75,
    0x6a1936c8, 0x6ed82b7f, 0x639b0da6, 0x675a1011, 0x791d4014, 0x7ddc5da3, 0x709f7b7a, 0x745e66cd,
    0x9823b6e0, 0x9ce2ab57, 0x91a18d8e, 0x95609039, 0x8b27c03c, 0x8fe6dd8b, 0x82a5fb52, 0x8664e6e5,
    0xbe2b5b58, 0xbaea46ef, 0xb7a96036, 0xb3687d81, 0xad2f2d84, 0xa9ee3033, 0xa4ad16ea, 0xa06c0b5d,
    0xd4326d90, 0xd0f37027, 0xddb056fe, 0xd9714b49, 0xc7361b4c, 0xc3f706fb, 0xceb42022, 0xca753d95,
    0xf23a8028, 0xf6fb9d9f, 0xfbb8bb46, 0xff79a6f1, 0xe13ef6f4, 0xe5ffeb43, 0xe8bccd9a, 0xec7dd02d,
    0x34867077, 0x30476dc0, 0x3d044b19, 0x39c556ae, 0x278206ab, 0x23431b1c, 0x2e003dc5, 0x2ac12072,
    0x128e9dcf, 0x164f8078, 0x1b0ca6a1, 0x1fcdbb16, 0x018aeb13, 0x054bf6a4, 0x0808d07d, 0x0cc9cdca,
    0x7897ab07, 0x7c56b6b0, 0x71159069, 0x75d48dde, 0x6b93dddb, 0x6f52c06c, 0x6211e6b5, 0x66d0fb02,
    0x5e9f46bf, 0x5a5e5b08, 0x571d7dd1, 0x53dc6066, 0x4d9b3063, 0x495a2dd4, 0x44190b0d, 0x40d816ba,
    0xaca5c697, 0xa864db20, 0xa527fdf9, 0xa1e6e04e, 0xbfa1b04b, 0xbb60adfc, 0xb6238b25, 0xb2e29692,
    0x8aad2b2f, 0x8e6c3698, 0x832f1041, 0x87ee0df6, 0x99a95df3, 0x9d684044, 0x902b669d, 0x94ea7b2a,
    0xe0b41de7, 0xe4750050, 0xe9362689, 0xedf73b3e, 0xf3b06b3b, 0xf771768c, 0xfa325055, 0xfef34de2,
    0xc6bcf05f, 0xc27dede8, 0xcf3ecb31, 0xcbffd686, 0xd5b88683, 0xd1799b34, 0xdc3abded, 0xd8fba05a,
    0x690ce0ee, 0x6dcdfd59, 0x608edb80, 0x644fc637, 0x7a089632, 0x7ec98b85, 0x738aad5c, 0x774bb0eb,
    0x4f040d56, 0x4bc510e1, 0x46863638, 0x42472b8f, 0x5c007b8a, 0x58c1663d, 0x558240e4, 0x51435d53,
    0x251d3b9e, 0x21dc2629, 0x2c9f00f0, 0x285e1d47, 0x36194d42, 0x32d850f5, 0x3f9b762c, 0x3b5a6b9b,
    0x0315d626, 0x07d4cb91, 0x0a97ed48, 0x0e56f0ff, 0x1011a0fa, 0x14d0bd4d, 0x19939b94, 0x1d528623,
    0xf12f560e, 0xf5ee4bb9, 0xf8ad6d60, 0xfc6c70d7, 0xe22b20d2, 0xe6ea3d65, 0xeba91bbc, 0xef68060b,
    0xd727bbb6, 0xd3e6a601, 0xdea580d8, 0xda649d6f, 0xc423cd6a, 0xc0e2d0dd, 0xcda1f604, 0xc960ebb3,
    0xbd3e8d7e, 0xb9ff90c9, 0xb4bcb610, 0xb07daba7, 0xae3afba2, 0xaafbe615, 0xa7b8c0cc, 0xa379dd7b,
    0x9b3660c6, 0x9ff77d71, 0x92b45ba8, 0x9675461f, 0x8832161a, 0x8cf30bad, 0x81b02d74, 0x857130c3,
    0x5d8a9099, 0x594b8d2e, 0x5408abf7, 0x50c9b640, 0x4e8ee645, 0x4a4ffbf2, 0x470cdd2b, 0x43cdc09c,
    0x7b827d21, 0x7f436096, 0x7200464f, 0x76c15bf8, 0x68860bfd, 0x6c47164a, 0x61043093, 0x65c52d24,
    0x119b4be9, 0x155a565e, 0x18197087, 0x1cd86d30, 0x029f3d35, 0x065e2082, 0x0b1d065b, 0x0fdc1bec,
    0x3793a651, 0x3352bbe6, 0x3e119d3f, 0x3ad08088, 0x2497d08d, 0x2056cd3a, 0x2d15ebe3, 0x29d4f654,
    0xc5a92679, 0xc1683bce, 0xcc2b1d17, 0xc8ea00a0, 0xd6ad50a5, 0xd26c4d12, 0xdf2f6bcb, 0xdbee767c,
    0xe3a1cbc1, 0xe760d676, 0xea23f0af, 0xeee2ed18, 0xf0a5bd1d, 0xf464a0aa, 0xf9278673, 0xfde69bc4,
    0x89b8fd09, 0x8d79e0be, 0x803ac667, 0x84fbdbd0, 0x9abc8bd5, 0x9e7d9662, 0x933eb0bb, 0x97ffad0c,
    0xafb010b1, 0xab710d06, 0xa6322bdf, 0xa2f33668, 0xbcb4666d, 0xb8757bda, 0xb5365d03, 0xb1f740b4,
};

uint32_t data_center_string_hash(char16_t *string)
{
    uint32_t hash = 0;

    for (char16_t *current = string; *current; current++)
    {
        char16_t value = htole16(*current);

        for (size_t i = 0; i < 2; i++)
            hash = string_hash_table[(uint8_t)(hash ^ ((uint8_t *)&value)[i])] ^ hash >> 8;
    }

    return hash;
}

(Note that string_hash_table is the same as value_hash_table.)

Data Graph

The actual content in a data center file is stored as a directed acyclic graph, which is essentially XML serialized to a binary format.

Nodes

Each node is of the form:

struct DataCenterNode
{
    uint16_t name_index;
    uint8_t key_flags : 4;
    uint16_t key_index : 12;
    uint16_t attribute_count;
    uint16_t child_count;
    DataCenterAddress attribute_address;
    uint32_t padding_1;
    DataCenterAddress child_address;
    uint32_t padding_2;
};

key_flags is 0 in official data center files. It may have a combination of the following values:

enum DataCenterKeyFlags : uint8_t
{
    DATA_CENTER_KEY_FLAGS_NONE = 0b0000,
    DATA_CENTER_KEY_FLAGS_UNCACHED = 0b0001,
};

If DATA_CENTER_KEY_FLAGS_UNCACHED is set, the results of a query against this node will not be cached by the client.

key_index is a zero-based index into the keys region.

attribute_address is an address into the attributes region. attribute_count attributes should be read at this address. These attributes must be sorted by their name index in ascending order.

padding_1 and padding_2 should be considered undefined. They were added in the 64-bit data center format, and are not present in the 32-bit format.

The root node of the data graph must be located at the address 0:0. It must have the name __root__ and have zero attributes.

Keys

struct DataCenterKey
{
    uint16_t name_index_1;
    uint16_t name_index_2;
    uint16_t name_index_3;
    uint16_t name_index_4;
};

Attributes

Each node in the data graph has zero or more attributes, which are name/value pairs. They are of the form:

struct DataCenterAttribute
{
    uint16_t name_index;
    DataCenterTypeCode type_code : 2;
    uint16_t extended_code : 14;
    union
    {
        int32_t i;
        bool b;
        float f;
        DataCenterAddress a;
    } value;
    uint32_t padding_1;
};

name_index is a one-based index into the addresses region of the names table.

type_code specifies the kind of value the attribute holds. Valid values are as follows:

enum DataCenterTypeCode : uint8_t
{
    DATA_CENTER_TYPE_CODE_INT = 1,
    DATA_CENTER_TYPE_CODE_FLOAT = 2,
    DATA_CENTER_TYPE_CODE_STRING = 3,
};

extended_code specifies extra information based on the value of type_code:

If type_code is DATA_CENTER_TYPE_CODE_INT, then the lowest bit of extended_code is set if the attribute's value should be considered a Boolean, meaning that value can only be 1 (true) or 0 (false). Either way, the higher bits are 0.
If type_code is DATA_CENTER_TYPE_CODE_FLOAT, then extended_code is 0.
If type_code is DATA_CENTER_TYPE_CODE_STRING, then extended_code is given by the expression data_center_value_hash(value) where value is the string value.

padding_1 should be considered undefined. It was added in the 64-bit data center format, and is not present in the 32-bit format.

<Foo>
    bar

    <Baz>
        ...
    </Baz>
</Foo>

Note that the __value__ attribute, if present, may only be a string.

Value Hash

The data_center_value_hash function uses a bizarre variant of CRC32 combined with a minimal effort to ignore the casing of characters. It is defined as follows:

const uint32_t value_hash_table[256] =
{
    0x00000000, 0x04c11db7, 0x09823b6e, 0x0d4326d9, 0x130476dc, 0x17c56b6b, 0x1a864db2, 0x1e475005,
    0x2608edb8, 0x22c9f00f, 0x2f8ad6d6, 0x2b4bcb61, 0x350c9b64, 0x31cd86d3, 0x3c8ea00a, 0x384fbdbd,
    0x4c11db70, 0x48d0c6c7, 0x4593e01e, 0x4152fda9, 0x5f15adac, 0x5bd4b01b, 0x569796c2, 0x52568b75,
    0x6a1936c8, 0x6ed82b7f, 0x639b0da6, 0x675a1011, 0x791d4014, 0x7ddc5da3, 0x709f7b7a, 0x745e66cd,
    0x9823b6e0, 0x9ce2ab57, 0x91a18d8e, 0x95609039, 0x8b27c03c, 0x8fe6dd8b, 0x82a5fb52, 0x8664e6e5,
    0xbe2b5b58, 0xbaea46ef, 0xb7a96036, 0xb3687d81, 0xad2f2d84, 0xa9ee3033, 0xa4ad16ea, 0xa06c0b5d,
    0xd4326d90, 0xd0f37027, 0xddb056fe, 0xd9714b49, 0xc7361b4c, 0xc3f706fb, 0xceb42022, 0xca753d95,
    0xf23a8028, 0xf6fb9d9f, 0xfbb8bb46, 0xff79a6f1, 0xe13ef6f4, 0xe5ffeb43, 0xe8bccd9a, 0xec7dd02d,
    0x34867077, 0x30476dc0, 0x3d044b19, 0x39c556ae, 0x278206ab, 0x23431b1c, 0x2e003dc5, 0x2ac12072,
    0x128e9dcf, 0x164f8078, 0x1b0ca6a1, 0x1fcdbb16, 0x018aeb13, 0x054bf6a4, 0x0808d07d, 0x0cc9cdca,
    0x7897ab07, 0x7c56b6b0, 0x71159069, 0x75d48dde, 0x6b93dddb, 0x6f52c06c, 0x6211e6b5, 0x66d0fb02,
    0x5e9f46bf, 0x5a5e5b08, 0x571d7dd1, 0x53dc6066, 0x4d9b3063, 0x495a2dd4, 0x44190b0d, 0x40d816ba,
    0xaca5c697, 0xa864db20, 0xa527fdf9, 0xa1e6e04e, 0xbfa1b04b, 0xbb60adfc, 0xb6238b25, 0xb2e29692,
    0x8aad2b2f, 0x8e6c3698, 0x832f1041, 0x87ee0df6, 0x99a95df3, 0x9d684044, 0x902b669d, 0x94ea7b2a,
    0xe0b41de7, 0xe4750050, 0xe9362689, 0xedf73b3e, 0xf3b06b3b, 0xf771768c, 0xfa325055, 0xfef34de2,
    0xc6bcf05f, 0xc27dede8, 0xcf3ecb31, 0xcbffd686, 0xd5b88683, 0xd1799b34, 0xdc3abded, 0xd8fba05a,
    0x690ce0ee, 0x6dcdfd59, 0x608edb80, 0x644fc637, 0x7a089632, 0x7ec98b85, 0x738aad5c, 0x774bb0eb,
    0x4f040d56, 0x4bc510e1, 0x46863638, 0x42472b8f, 0x5c007b8a, 0x58c1663d, 0x558240e4, 0x51435d53,
    0x251d3b9e, 0x21dc2629, 0x2c9f00f0, 0x285e1d47, 0x36194d42, 0x32d850f5, 0x3f9b762c, 0x3b5a6b9b,
    0x0315d626, 0x07d4cb91, 0x0a97ed48, 0x0e56f0ff, 0x1011a0fa, 0x14d0bd4d, 0x19939b94, 0x1d528623,
    0xf12f560e, 0xf5ee4bb9, 0xf8ad6d60, 0xfc6c70d7, 0xe22b20d2, 0xe6ea3d65, 0xeba91bbc, 0xef68060b,
    0xd727bbb6, 0xd3e6a601, 0xdea580d8, 0xda649d6f, 0xc423cd6a, 0xc0e2d0dd, 0xcda1f604, 0xc960ebb3,
    0xbd3e8d7e, 0xb9ff90c9, 0xb4bcb610, 0xb07daba7, 0xae3afba2, 0xaafbe615, 0xa7b8c0cc, 0xa379dd7b,
    0x9b3660c6, 0x9ff77d71, 0x92b45ba8, 0x9675461f, 0x8832161a, 0x8cf30bad, 0x81b02d74, 0x857130c3,
    0x5d8a9099, 0x594b8d2e, 0x5408abf7, 0x50c9b640, 0x4e8ee645, 0x4a4ffbf2, 0x470cdd2b, 0x43cdc09c,
    0x7b827d21, 0x7f436096, 0x7200464f, 0x76c15bf8, 0x68860bfd, 0x6c47164a, 0x61043093, 0x65c52d24,
    0x119b4be9, 0x155a565e, 0x18197087, 0x1cd86d30, 0x029f3d35, 0x065e2082, 0x0b1d065b, 0x0fdc1bec,
    0x3793a651, 0x3352bbe6, 0x3e119d3f, 0x3ad08088, 0x2497d08d, 0x2056cd3a, 0x2d15ebe3, 0x29d4f654,
    0xc5a92679, 0xc1683bce, 0xcc2b1d17, 0xc8ea00a0, 0xd6ad50a5, 0xd26c4d12, 0xdf2f6bcb, 0xdbee767c,
    0xe3a1cbc1, 0xe760d676, 0xea23f0af, 0xeee2ed18, 0xf0a5bd1d, 0xf464a0aa, 0xf9278673, 0xfde69bc4,
    0x89b8fd09, 0x8d79e0be, 0x803ac667, 0x84fbdbd0, 0x9abc8bd5, 0x9e7d9662, 0x933eb0bb, 0x97ffad0c,
    0xafb010b1, 0xab710d06, 0xa6322bdf, 0xa2f33668, 0xbcb4666d, 0xb8757bda, 0xb5365d03, 0xb1f740b4,
};

uint16_t data_center_value_hash(char16_t *string)
{
    uint32_t hash = 0;

    for (char16_t *current = string; *current; current++)
    {
        char16_t value = *current;

        if (value == u'\x9c')
            value = u'\x8c';
        else if (value == u'\xff')
            value = u'\x9f';
        else if (value == u'\x151')
            value = u'\x150';
        else if (((value >= u'a' && value <= u'z') || (value >= u'\xe0' && value <= u'\xfe')) &&
            !(value == u'\xf0' || value == u'\xf7'))
            value = c - u' ';

        value = htole16(value);

        for (size_t i = 0; i < 2; i++)
            hash = value_hash_table[(uint8_t)(hash ^ ((uint8_t *)&value)[i])] ^ hash >> 8;
    }

    return hash & 0x3fff;
}

(Note that value_hash_table is the same as string_hash_table.)

Launcher/Client Protocol

This page describes the communication protocol employed by launcher.exe, Tl.exe, and TERA.exe.

C/C++-like primitive types and structs will be used.
Integers (uint8_t, int8_t, uint16_t, int16_t, etc) are little endian.
Characters (i.e. char8_t and char16_t) are UTF-8 and UTF-16 respectively, and little endian.
Strings (i.e. u8string and u16string) are a series of valid char8_t and char16_t characters respectively, followed by a NUL character.
Fields are laid out in the declared order with no implied padding anywhere.

Note that strings are not always NUL-terminated. For this reason, the document will explicitly call out whether a NUL terminator is present.

Communication

The role of each program is as follows:

launcher.exe: The publisher-specific game launcher which performs authentication and server list URL resolution. Serves requests from Tl.exe and receives game events.
Tl.exe: The (mostly) publisher-agnostic game launcher which requests authentication data and the server list URL from launcher.exe. Serves requests and forwards game events from TERA.exe.
TERA.exe: The actual game client. Sends requests and game events to Tl.exe.

These programs all communicate via window messages, specifically WM_COPYDATA. The dwData field specifies the message ID, while lpData and cbData contain the payload pointer and length.

`Tl.exe` -> `launcher.exe` Messages

Hello Handshake (`0x0dbadb0a`)

The protocol starts off with a handshake sent from Tl.exe. This message contains the NUL-terminated string Hello!!.

Game Event Notification (`0x0`)

Tl.exe will occasionally notify launcher.exe of various game events sent by TERA.exe. The format of these messages can be described with the following regular expressions:

^csPopup$: Signals that launcher.exe should open the customer support website (e.g. in the default Web browser).
^gameEvent$(\d+)$$: Indicates some kind of notable action taken by the user in the game. The value in parentheses is a code specifying the type of event; the possible values are documented in the section on TERA.exe messages.
^endPopup$(\d+)$$: Indicates that the client has exited. The value in parentheses is an exit reason code (not the same as the process exit code); the possible values are documented in the section on TERA.exe messages.
^promoPopup$\d+$$: The exact purpose of this notification is currently unknown.

launcher.exe should not respond to these messages.

Server List URL Request

Tl.exe will request the server list URL from launcher.exe. This message does not have a static message ID. The message contains the NUL-terminated string slsurl.

launcher.exe should respond with the NUL-terminated server list URL.

Server List Schema

The response received from the server list URL should be in XML format. It can be described with the following XSD schema:

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns="https://vezel.dev/novadrop/tl/ServerList"
            targetNamespace="https://vezel.dev/novadrop/tl/ServerList"
            elementFormDefault="qualified">
    <xsd:complexType name="serverlist">
        <xsd:sequence>
            <xsd:element name="server" type="serverlist_server" maxOccurs="unbounded" />
        </xsd:sequence>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server">
        <xsd:sequence>
            <xsd:element name="id" type="serverlist_server_id" />
            <xsd:element name="ip" type="serverlist_server_ip" />
            <xsd:element name="port" type="unsignedShort" />
            <xsd:element name="category" type="serverlist_server_category" />
            <xsd:element name="name" type="serverlist_server_name" />
            <xsd:element name="crowdness" type="serverlist_server_crowdness" />
            <xsd:element name="open" type="serverlist_server_open" />
            <xsd:element name="permission_mask" type="serverlist_server_permission_mask" />
            <xsd:element name="server_stat" type="serverlist_server_server_stat" />
            <xsd:element name="popup" type="xsd:string" />
            <xsd:element name="language" type="xsd:string" />
        </xsd:sequence>
    </xsd:complexType>

    <xsd:simpleType name="serverlist_server_id">
        <xsd:restriction base="xsd:unsignedInt">
            <xsd:minInclusive value="1" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:simpleType name="serverlist_server_ip">
        <xsd:restriction base="xsd:string">
            <xsd:pattern value="\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:complexType name="serverlist_server_category">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="sort" type="xsd:unsignedInt" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server_name">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="raw_name" type="xsd:string" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server_crowdness">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="sort" type="xsd:unsignedInt" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:complexType name="serverlist_server_open">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string">
                <xsd:attribute name="sort" type="xsd:unsignedInt" />
            </xsd:extension>
        </xsd:simpleContent>
    </xsd:complexType>

    <xsd:simpleType name="serverlist_server_permission_mask">
        <xsd:restriction base="xsd:string">
            <xsd:pattern value="0x[0-9A-Fa-f]{8}" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:simpleType name="serverlist_server_server_stat">
        <xsd:restriction base="xsd:string">
            <xsd:pattern value="0x[0-9A-Fa-f]{8}" />
        </xsd:restriction>
    </xsd:simpleType>

    <xsd:element name="serverlist" type="serverlist" />
</xsd:schema>

Authentication Info Request

Tl.exe will request JSON-encoded authentication data from launcher.exe. This message does not have a static message ID. The message contains one of several NUL-terminated strings.

gamestr is only sent when the game is initially launched. This request asks for the full authentication data.

ticket, last_svr, and char_cnt are only sent when returning to the server list from an arbiter server.

ticket: Requests the authentication ticket. This can be the same ticket as before, but launcher.exe may also choose to authenticate anew and retrieve a fresh ticket.
last_svr: Requests the ID of the last server the the account connected to.
char_cnt: Requests character counts for each server in the server list.

launcher.exe should respond with the NUL-terminated JSON payload.

Authentication Info Schema

The authentication JSON response can be described with the following JSON Schema definition:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://vezel.dev/novadrop/tl/authentication.schema.json",
  "title": "AuthenticationInfo",
  "type": "object",
  "properties": {
    "result-message": {
      "type": "string"
    },
    "result-code": {
      "type": "integer",
      "minimum": 100,
      "maximum": 599
    },
    "account_bits": {
      "type": "string",
      "pattern": "^0x[0-9A-Fa-f]{10}$"
    },
    "ticket": {
      "type": "string"
    },
    "last_connected_server_id": {
      "type": "integer",
      "minimum": 0
    },
    "access_level": {
      "type": "integer",
      "minimum": 0
    },
    "master_account_name": {
      "type": "string"
    },
    "chars_per_server": {
      "anyOf": [
        {
          "type": "object",
          "properties": {},
          "additionalProperties": false
        },
        {
          "$ref": "#/$defs/chars_on_server"
        },
        {
          "type": "array",
          "items": {
            "$ref": "#/$defs/chars_on_server"
          }
        }
      ]
    },
    "user_permission": {
      "type": "integer",
      "minimum": 0
    },
    "game_account_name": {
      "type": "string"
    }
  },
  "required": [
    "result-code",
    "master_account_name",
    "game_account_name"
  ],
  "$defs": {
    "chars_on_server": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string",
          "pattern": "^[1-9]\\d*$"
        },
        "char_count": {
          "type": "string",
          "pattern": "^\\d+$"
        }
      },
      "additionalProperties": false,
      "required": [
        "id",
        "char_count"
      ]
    }
  }
}

Some properties are only required depending on the message:

gamestr: All properties are required.
ticket: The ticket property is required.
last_svr: The last_connected_server_id property is required.
char_cnt: The chars_per_server property is required.

result-message, account_bits, access_level, and user_permission are completely optional.

Web URL Request

Tl.exe will request a URL to be opened in the client's embedded Web browser. This message does not have a static message ID and is not NUL-terminated.

launcher.exe should respond with a URL to be opened (without a NUL terminator), or an empty payload to reject the request.

`TERA.exe` -> `Tl.exe` Messages

Messages sent between TERA.exe and Tl.exe use a simple binary protocol. All messages have static message IDs; responses have different IDs from requests.

Account Name Request (`0x1`)

TERA.exe will request the (game) account name from Tl.exe.

struct LauncherAccountNameRequest
{
};

Account Name Response (`0x2`)

Tl.exe should respond with the account name.

struct LauncherAccountNameResponse
{
    u16string account_name;
};

account_name is the name of the game account. It is not NUL-terminated.

Session Ticket Request (`0x3`)

TERA.exe will request the session ticket from Tl.exe.

struct LauncherSessionTicketRequest
{
};

Session Ticket Response (`0x4`)

Tl.exe should respond with the session ticket.

struct LauncherAccountNameResponse
{
    u8string session_ticket;
};

session_ticket is the authentication session ticket. It is not NUL-terminated.

Server List Request (`0x5`)

TERA.exe will request the server list from Tl.exe.

struct LauncherServerListRequest
{
    LauncherServerListSortCriteria sort_criterion;
};

sorting specifies how Tl.exe should sort the server list. Valid values are as follows:

enum LauncherServerListSortCriteria : int32_t
{
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_NONE = -1,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_CHARACTERS = 0,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_CATEGORY = 1,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_NAME = 2,
    LAUNCHER_SERVER_LIST_SORT_CRITERIA_POPULATION = 4,
};

A few notes on sorting:

The sort should be stable.
LAUNCHER_SERVER_LIST_SORT_CRITERIA_NONE indicates that Tl.exe is free to sort the list arbitrarily.
The resultant list should be maintained between requests and further sorted on each request, unless LAUNCHER_SERVER_LIST_SORTING_NONE is sent to reset the sorting.
If the same sorting is requested multiple times and is any sorting other than LAUNCHER_SERVER_LIST_SORT_CRITERIA_NONE, the list should simply be reversed without applying sorting.

Server List Response (`0x6`)

Tl.exe should respond with the server list encoded with Protocol Buffers.

struct LauncherServerListResponse
{
    uint8_t data[];
};

Server List Structures

The server list response can be described with the following message definitions:

syntax = "proto2";

message ServerList
{
    message ServerInfo
    {
        required fixed32 id = 1;
        required bytes name = 2;
        required bytes category = 3;
        required bytes title = 4;
        required bytes queue = 5;
        required bytes population = 6;
        required fixed32 address = 7;
        required fixed32 port = 8;
        required fixed32 available = 9;
        required bytes unavailable_message = 10;
        optional bytes host = 11;
    }

    repeated ServerInfo servers = 1;
    required fixed32 last_server_id = 2;
    required fixed32 sort_criterion = 3;
}

A few notes on these definitions:

They will not work correctly as proto3 due to required semantics.
bytes fields are really u16string without NUL terminators.
id must be a positive (non-zero) value.
port should be in the uint16_t range.
available is really a bool, so only the values 0 and 1 are allowed.
Either address or host must be set; not neither and not both.
- For address, a value of 0 has 'not set' semantics since the field is not marked optional.
sort_criterion should be the LauncherServerListSortCriteria value that was sent in the request.

Enter Lobby/World Notification (`0x7`)

TERA.exe will notify Tl.exe when entering the lobby (i.e. successfully connecting to an arbiter server) or when entering the world on a particular character.

struct LauncherEnterLobbyNotification
{
};

struct LauncherEnterWorldNotification
{
    u16string character_name;
};

The two cases can be distinguished by looking at the payload length.

character_name is the NUL-terminated name of the character that the user is entering the world on.

Voice Chat Requests

There is a set of requests for interacting with TeamSpeak:

Create Room (0x8)
Join Room (0xa)
Leave Room (0xc)
Set Volume (0x13)
Set Microphone (0x14)
Silence User (0x15)

These messages were only present in some regions and were likely never actually used. It is currently unknown what their payloads contain.

Voice Chat Responses

The following responses exist for the above TeamSpeak requests:

Create Room Result (0x9)
Join Room Result (0xb)
Leave Room Result (0xd)

Open Website Command (`0x19`)

TERA.exe asks Tl.exe to open a website in the default Web browser.

struct LauncherOpenWebsiteCommand
{
    uint32_t id;
};

id specifies the kind of website that should be opened. The possible values are currently unknown.

Web URL Request (`0x1a`)

This request is the TERA.exe equivalent of the request sent by Tl.exe to launcher.exe.

struct LauncherWebURLRequest
{
    uint32_t id;
    u16string arguments;
};

id refers to a UIWindow node under the CoherentGTWeb data center sheet.

arguments specifies the NUL-terminated arguments specific to the URL.

Web URL Response (`0x1b`)

struct LauncherWebURLResponse
{
    uint32_t id;
    u16string url;
};

id is the same value that was sent in the request.

url is the NUL-terminated URL to open. It can be the empty string or the special string | to indicate that no URL should be opened.

Game Start Notification (`0x3e8`)

TERA.exe will notify Tl.exe that it has launched.

struct LauncherGameStartNotification
{
    uint32_t source_revision;
    uint32_t unknown_1;
    u16string windows_account_name;
};

source_revision is the SrcRegVer value from the client's ReleaseRevision.txt file.

The meaning of unknown_1 is currently unknown.

windows_account_name is the NUL-terminated name of the current Windows user account.

Game Event Notification (`0x3e9` - `0x3f8`)

TERA.exe will occasionally notify Tl.exe of various notable actions taken by the user.

struct LauncherGameEventNotification
{
    LauncherGameEvent event;
};

event specifies the kind of event that occurred. Valid values are as follows:

enum LauncherGameEvent : uint32_t
{
    LAUNCHER_GAME_EVENT_ENTERED_INTO_CINEMATIC = 1001,
    LAUNCHER_GAME_EVENT_ENTERED_SERVER_LIST = 1002,
    LAUNCHER_GAME_EVENT_ENTERING_LOBBY = 1003,
    LAUNCHER_GAME_EVENT_ENTERED_LOBBY = 1004,
    LAUNCHER_GAME_EVENT_ENTERING_CHARACTER_CREATION = 1005,
    LAUNCHER_GAME_EVENT_LEFT_LOBBY = 1006,
    LAUNCHER_GAME_EVENT_DELETED_CHARACTER = 1007,
    LAUNCHER_GAME_EVENT_CANCELED_CHARACTER_CREATION = 1008,
    LAUNCHER_GAME_EVENT_ENTERED_CHARACTER_CREATION = 1009,
    LAUNCHER_GAME_EVENT_CREATED_CHARACTER = 1010,
    LAUNCHER_GAME_EVENT_ENTERED_WORLD = 1011,
    LAUNCHER_GAME_EVENT_FINISHED_LOADING_SCREEN = 1012,
    LAUNCHER_GAME_EVENT_LEFT_WORLD = 1013,
    LAUNCHER_GAME_EVENT_MOUNTED_PEGASUS = 1014,
    LAUNCHER_GAME_EVENT_DISMOUNTED_PEGASUS = 1015,
    LAUNCHER_GAME_EVENT_CHANGED_CHANNEL = 1016,
};

Game Exit Notification (`0x3fc`)

TERA.exe will notify Tl.exe that it is exiting.

struct LauncherGameExitNotification
{
    uint32_t length;
    uint32_t code;
    LauncherGameExitReason reason;
};

length is the length of the payload. It must be 12.

code is the exit code of the TERA.exe process. This can be 0 or 1.

reason provides a more specific exit reason. Some known values are as follows:

enum LauncherGameExitReason : uint32_t
{
    LAUNCHER_GAME_EXIT_REASON_SUCCESS = 0x0,
    LAUNCHER_GAME_EXIT_REASON_INVALID_DATA_CENTER = 0x6,
    LAUNCHER_GAME_EXIT_REASON_CONNECTION_DROPPED = 0x8,
    LAUNCHER_GAME_EXIT_REASON_INVALID_AUTHENTICATION_INFO = 0x9,
    LAUNCHER_GAME_EXIT_REASON_OUT_OF_MEMORY = 0xa,
    LAUNCHER_GAME_EXIT_REASON_SHADER_MODEL_3_UNAVAILABLE = 0xc,
    LAUNCHER_GAME_EXIT_REASON_SPEED_HACK_DETECTED = 0x10,
    LAUNCHER_GAME_EXIT_REASON_UNSUPPORTED_VERSION = 0x13,
    LAUNCHER_GAME_EXIT_REASON_ALREADY_ONLINE = 0x106,
};

(The above enumeration is an incomplete list.)

Game Crash Notification (`0x3fd`)

struct LauncherGameCrashNotification
{
    u16string details;
};

details contains various details about the crash such as the instruction pointer and exception type. It is not NUL-terminated.

Anti-Cheat Event Notifications (`0x3fe` - `0x400`)

TERA.exe will notify Tl.exe of various events relating to the anti-cheat module (e.g. XIGNCODE3 or GameGuard). Note that only some regions have a client build with an anti-cheat module.

Anti-Cheat Starting Notification (`0x3fe`)

TERA.exe will notify Tl.exe that the anti-cheat module is starting.

struct LauncherAntiCheatStartingNotification
{
};

Anti-Cheat Started Notification (`0x3ff`)

TERA.exe will notify Tl.exe that the anti-cheat module has successfully started.

struct LauncherAntiCheatStartedNotification
{
};

Anti-Cheat Error Notification (`0x400`)

TERA.exe will notify Tl.exe that the anti-cheat module failed to start.

struct LauncherAntiCheatStartedNotification
{
    uint64_t error;
};

error contains the error code from the anti-cheat module.

Open Support Website Command (`0x401`)

TERA.exe asks Tl.exe to open the customer support website in the default Web browser.

struct LauncherOpenSupportWebsiteCommand
{
};

Unknown Command 1027 (`0x403`)

The exact purpose of this message is currently unknown.

Novadrop

Home

Game

Data Center Format

Encryption

Physical Structure

Compression Header

File Header

File Footer

Regions

String Tables

String Hash

Data Graph

Nodes

Keys

Attributes

Value Hash

Package Format

Resource Container Format

Encryption

Physical Structure

File Footer

Directory

Entries

Launcher/Client Protocol

Communication

Tl.exe -> launcher.exe Messages

Hello Handshake (0x0dbadb0a)

Game Event Notification (0x0)

Server List URL Request

Server List Schema

Authentication Info Request

Authentication Info Schema

Web URL Request

TERA.exe -> Tl.exe Messages

Account Name Request (0x1)

Account Name Response (0x2)

Session Ticket Request (0x3)

Session Ticket Response (0x4)

Server List Request (0x5)

Server List Response (0x6)

Enter Lobby/World Notification (0x7)

Voice Chat Requests

Voice Chat Responses

Open Website Command (0x19)

Web URL Request (0x1a)

Web URL Response (0x1b)

Game Start Notification (0x3e8)

Game Event Notification (0x3e9 - 0x3f8)

Game Exit Notification (0x3fc)

Game Crash Notification (0x3fd)

Anti-Cheat Event Notifications (0x3fe - 0x400)

Anti-Cheat Starting Notification (0x3fe)

Anti-Cheat Started Notification (0x3ff)

Anti-Cheat Error Notification (0x400)

Open Support Website Command (0x401)

Unknown Command 1027 (0x403)

Network Protocol

Encryption

Packet Header

Packet Body

Complex Types

Strings

Object Arrays

Byte Arrays

Tools

novadrop-dc

novadrop-dc pack

novadrop-dc repack

novadrop-dc schema

novadrop-dc unpack

novadrop-dc validate

novadrop-dc verify

novadrop-rc

novadrop-rc pack

novadrop-rc repack

novadrop-rc unpack

novadrop-rc verify

Data Center Format

Encryption

`Tl.exe` -> `launcher.exe` Messages

Hello Handshake (`0x0dbadb0a`)

Game Event Notification (`0x0`)

`TERA.exe` -> `Tl.exe` Messages

Account Name Request (`0x1`)

Account Name Response (`0x2`)

Session Ticket Request (`0x3`)

Session Ticket Response (`0x4`)

Server List Request (`0x5`)

Server List Response (`0x6`)

Enter Lobby/World Notification (`0x7`)

Open Website Command (`0x19`)

Web URL Request (`0x1a`)

Web URL Response (`0x1b`)

Game Start Notification (`0x3e8`)

Game Event Notification (`0x3e9` - `0x3f8`)

Game Exit Notification (`0x3fc`)

Game Crash Notification (`0x3fd`)

Anti-Cheat Event Notifications (`0x3fe` - `0x400`)

Anti-Cheat Starting Notification (`0x3fe`)

Anti-Cheat Started Notification (`0x3ff`)

Anti-Cheat Error Notification (`0x400`)

Open Support Website Command (`0x401`)

Unknown Command 1027 (`0x403`)

`Tl.exe` -> `launcher.exe` Messages

Hello Handshake (`0x0dbadb0a`)

Game Event Notification (`0x0`)

`TERA.exe` -> `Tl.exe` Messages

Account Name Request (`0x1`)

Account Name Response (`0x2`)

Session Ticket Request (`0x3`)

Session Ticket Response (`0x4`)

Server List Request (`0x5`)

Server List Response (`0x6`)

Enter Lobby/World Notification (`0x7`)

Open Website Command (`0x19`)

Web URL Request (`0x1a`)

Web URL Response (`0x1b`)

Game Start Notification (`0x3e8`)

Game Event Notification (`0x3e9` - `0x3f8`)

Game Exit Notification (`0x3fc`)

Game Crash Notification (`0x3fd`)

Anti-Cheat Event Notifications (`0x3fe` - `0x400`)

Anti-Cheat Starting Notification (`0x3fe`)

Anti-Cheat Started Notification (`0x3ff`)