FileBarge protocol

This document specifies version 1 of the FileBarge TCP protocol. The key words "MUST", "MUST NOT", "REQUIRED", "SHOULD", "SHOULD NOT", and "MAY" are to be interpreted as described by RFC 2119.

1. Scope and conventions

FileBarge transfers regular-file contents over one TCP connection. It does not represent directories or symbolic links. The sender starts the connection; the receiver answers. A session carries one or more offered files, followed by a normal completion exchange.

All multibyte integers are unsigned and use big-endian network byte order. Bit 0 is the most significant bit of the first transmitted byte. Text names and hello messages are UTF-8. Error and skip explanations are ASCII. Variable-length text is never NUL terminated.

Packet layouts use network-bit order. ~ marks the vertical boundary of a variable-length field.

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                         32-bit field                          |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

An implementation MUST reject malformed lengths, unknown opcodes, invalid state transitions, nonzero reserved fields, and unsupported flag combinations. Either peer MAY close the TCP connection at any time; a close before normal completion is a hard abort.

2. Opcodes

Opcode 0 is reserved. Opcode 1 is an error packet sent by either peer. Other even opcodes are sent by the sender and other odd opcodes by the receiver. The gaps in the opcode table are intentional. Every opcode not currently assigned is reserved for future use and may be assigned in a future protocol revision. Implementations MUST reject reserved opcode values as unknown opcodes.

Opcode	Direction	Packet
1	Either	Error
2	Sender to receiver	Sender Hello
3	Receiver to sender	Receiver Hello
4	Sender to receiver	Request File
5	Receiver to sender	Accept File
7	Receiver to sender	Skip File
8	Sender to receiver	Verify Checksum
9	Receiver to sender	Checksum Okay
10	Sender to receiver	End Files
11	Receiver to sender	Have a Nice Day
12	Sender to receiver	Bear with Me
13	Receiver to sender	Bear with Me
14	Sender to receiver	Encryption Handshake
15	Receiver to sender	Encryption Handshake

3. Session establishment

The sender first sends Sender Hello and the receiver answers with Receiver Hello. Both packets have a protocol version of 1. Version 0 is reserved. The peer that cannot support the version MUST return error 1, "Unsupported protocol version".

Sender Hello is encoded as follows:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 2  |  Version = 1  |             Flags             |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |    Timeout    | Encrypt Flags |   Reserved    | Message Length|
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                                               |
 ~                    Message (variable)                         ~
 |                                                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The sender flags use their three least significant bits:

0000000000000PRM

P means POSIX permissions are supplied in file requests, R permits relative paths in file names, and M permits one or multiple file offers. With M clear, the sender promises exactly one file. All other bits are reserved.

Receiver Hello is encoded as follows:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 3  |  Version = 1  |          Reserved = 0         |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |    Timeout    |E|EType|       Reserved        | Message Length|
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                                               |
 ~                    Message (variable)                         ~
 |                                                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

E selects encryption. When set, EType selects a bit offered in the sender's encryption flags as 1 << EType; when clear, EType has no meaning. All receiver flags are reserved.

The timeout byte represents an activity timeout in 100 ms increments plus 100 ms: 0 represents 100 ms, 79 represents the default eight seconds, and 255 represents 25.6 seconds. Each peer applies the timeout advertised by the other. Every valid received packet resets it.

The message length is an 8-bit byte count. A zero-length message is valid. Hello messages are informational and do not affect interoperability.

4. File negotiation

The sender offers each file using Request File:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 4  | Checksum Flags|  Compr. Flags |    Reserved   |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                                               |
 +                  Modification Date (64 bits)                  +
 |                                                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |  Permissions    |T|        File Size (high 22 bits)           |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                    File Size (low 32 bits)                    |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |       Name Length             |                               |
 +-------------------------------+                               |
 ~                       Name (variable)                         ~
 |                                                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The modification time is microseconds since 1970-01-01 00:00:00 UTC. Dates before that epoch are not represented. permissions carries only the nine POSIX read, write, and execute bits when the sender's P flag is set; otherwise it is zero. The 54-bit file size represents a known byte count from 0 through 2^54 - 1.

T is the high bit of the two-byte size field. It selects chunked transfer, for streams whose final size is not known. The advertised size is then an estimate. Compression also requires chunked transfer, even if T is clear.

The name is a UTF-8 byte sequence. With R clear it MUST be a bare filename; with R set it MAY be a relative /-separated path. Receivers MUST reject absolute paths, . components, empty components, and .. components. A Windows receiver MUST reject \ within a component before converting / to a native separator.

The checksum and compression bytes are bit sets. Bit 0 currently means Fletcher32 and Squinch respectively. Zero means no algorithm is offered. Other bits are reserved.

The receiver either accepts or skips the file. Accept File is:

  0                   1                   2
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 5  |S|SType|C|CType|   Reserved    |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

S selects checksumming and SType selects a sender-offered type. C selects compression and CType selects a sender-offered type. Type 0 is Fletcher32 or Squinch as appropriate. When either enable bit is clear, its type has no meaning. The sender MUST provide a requested checksum and MUST use requested compression. It MAY send a supported checksum that was not requested.

Skip File is:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 7  | Reserved = 0  | Reason Length |  Reason Code  |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                                               |
 ~                    Reason String (variable)                   ~
 |                                                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Reason code 1 means "File exists" and 2 means "File too old". A received file with an equal timestamp is too old because newer-only replacement is strictly newer. After a skip, the sender MUST NOT send file data and MAY offer the next file.

5. File data and checksums

For an uncompressed, known-size transfer without encryption, the sender transmits exactly the advertised number of raw data bytes. No packet framing surrounds those bytes.

Chunked transfers use repeated chunks:

  0                   1
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |          Chunk Length         |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               |
 ~          Chunk Data           ~
 |                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

A normal chunk contains at most 65,534 bytes. A zero-length chunk terminates the file. Length 65,535 is a keep-alive marker without data and does not contribute to the file. A sender MAY use it while preparing another chunk. With encryption, one encrypted block MUST contain whole chunks only, and the maximum actual chunk data is 65,533 bytes.

The sender may then send Verify Checksum:

  0                   1
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 8  | Length  |Type |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                               |
 ~       Checksum (variable)     ~
 |                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The high five bits encode checksum length and the low three bits encode type. Fletcher32 has type 0 and length 4. A receiver that validates it sends Checksum Okay:

  0                   1
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 9  | Reserved = 0  |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

If a requested checksum is absent, the session has a protocol error. A checksum mismatch produces recoverable error 32769. If decompression fails, the receiver discards the temporary output, sends recoverable error 32768, and consumes chunk framing through the terminating zero chunk. The sender stops file data when it receives that error, sends the zero terminator, and does not send Verify Checksum for that file.

6. Session completion and keep-alives

After the final offer, the sender sends End Files and the receiver replies Have a Nice Day:

  0                   1
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |  Opcode = 10  | Reserved = 0  |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  0                   1
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |  Opcode = 11  | Reserved = 0  |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The TCP connection may then close normally.

Bear with Me is sent when the peer expected to speak cannot do so before the activity timeout:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Opcode = 12/13|   Reserved    |            Counter            |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The sender uses opcode 12 and the receiver uses opcode 13. Each peer's counter strictly increases and wraps to zero after 65535. Recipients discard the packet without changing state, but it resets the activity timeout. A peer MUST send enough keep-alives to avoid approximately half the other peer's advertised timeout elapsing without an outgoing packet.

7. Errors

Either peer may send Error:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   Opcode = 1  | String Length |          Error Code           |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                                               |
 ~                    Error String (variable)                    ~
 |                                                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The explanation is optional. Error codes are machine-readable and have the following classification.

Code	Meaning	Classification
1	Unsupported protocol version	Fatal
2	Invalid flag combination	Fatal
3	Unknown opcode	Fatal
4	Unexpected opcode	Fatal
5	Unsafe filename	Fatal
6	Filename too long	Fatal
7	File too large	Fatal
8	Destination path failure	Fatal
9	Checksum required	Fatal
10	Encryption required	Fatal
11	Decryption failure	Fatal
12	No compatible checksum	Fatal
13	No compatible compression	Fatal
32768	Decompression failure	Recoverable
32769	Checksum verification failure	Recoverable

A recoverable error is valid only when both peers retain synchronized state and can safely continue with a later file. Malformed framing, truncation, and desynchronization are always fatal.

8. Encryption

When Receiver Hello selects encryption, the peers exchange opcodes 14 and 15 before any encrypted protocol or file data:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Opcode = 14/15| Type| Reserved| Type-specific data ...
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Both peers then send one encrypted Bear with Me packet to verify the derived keys. A decryption failure MUST produce error 11 and terminate the connection.

Version 1 defines type 0, Ascon-AEAD128. Its handshake payload is:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Opcode = 14/15| 0   | Reserved|      Nonce (16 bytes) ...
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The sender nonce is SNonce; the receiver nonce is RNonce. Implementations MUST use a cryptographically secure random source where available.

Using Ascon CXOF128(customization, input, outputLength), derive two independent 16-byte keys:

SKey = CXOF128("FileBarge Sender Key", SNonce || SecretLength || Secret || RNonce, 16)
RKey = CXOF128("FileBarge Receiver Key", RNonce || SecretLength || Secret || SNonce, 16)

SecretLength is a 16-bit big-endian length. || means byte concatenation. The sender uses SKey for sender-to-receiver traffic and the receiver uses RKey for receiver-to-sender traffic.

Initial 16-byte block nonces are:

SBN = CXOF128("FileBarge Sender Nonce", SNonce || RNonce, 16)
RBN = CXOF128("FileBarge Receiver Nonce", RNonce || SNonce, 16)

The sending peer increments its nonce for every following encrypted block, modulo 2^128. Type 0 uses a 16-byte IV and 16-byte authentication tag.

After the handshake, each protocol packet and file-data segment is carried in an encrypted block:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |     Block Size (8-byte units) | IV Size | Auth. Data Size     |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |         Content Size          |                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
 |                       IV (variable)                           |
 +                                                               +
 |                    Content (variable)                         |
 +                                                               +
 |                Content Padding (variable)                     |
 +                                                               +
 |             Authentication Data (variable)                    |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The block size is the complete block, including its six-byte header, in eight-byte units. The total block size MUST be a multiple of eight. Content size is the unpadded byte count. The header, IV, content padding, and encrypted content MUST be authenticated; only the content and padding are encrypted. Encryption therefore wraps packets after compression and unwraps them before protocol or decompression processing.

9. State machine

Except for Bear with Me, which leaves the current state unchanged, the following incoming packets are valid.

Receiving peer and state	Allowed packets
Receiver before hello	Sender Hello
Receiver during encryption setup	Sender Encryption Handshake, encrypted Sender Bear with Me
Receiver between files	Request File, End Files
Receiver after requesting a checksum	Verify Checksum
Sender before hello	Receiver Hello
Sender during encryption setup	Receiver Encryption Handshake, encrypted Receiver Bear with Me
Sender after Request File	Accept File, Skip File, Error
Sender after checksum verification	Checksum Okay, Error
Sender after End Files	Have a Nice Day

An unknown opcode produces error 3. A known opcode in the wrong state produces error 4. A reserved field or invalid selected flag combination produces error 2.

10. Receiver responsibilities

The receiver MUST write incoming contents to a temporary file and only make it visible after all data arrives, transfer framing ends correctly, and any requested checksum succeeds. For a new destination it MAY atomically rename the verified temporary file into place. When replacing an existing destination, it MAY copy verified contents into the existing inode to preserve ownership and applicable metadata; that final copy is not atomic.

When sender permissions are available and not disabled locally, the receiver SHOULD apply the transmitted POSIX permissions where supported. It MUST NOT assume that ownership can be transferred. Existing files that are skipped are not protocol errors.