Diving into Source Engine network protocol as a client

I’ve seen it asked several times over the internet, spanning several years; requests for explanation about how Source Engine’s multiplayer network protocol works, and noone has been able to provide any meaningful documentation.

Valve’s developer wiki [LINK] is excellent at explaining the benefits and nuances of Sources networking, but not how a client actually goes from knowing a server <ip_address:port> to being connected and able to interact with the game world. To be fair, why should it? Even your standard developer shouldn’t need to worry about exactly HOW the netcode works. The engines source code provides a set of (reasonably) simple macros for extending and implementing new types of data that should be synced across clients and server.

Regardless of the lack of documentation on the subject, the leak a few years back of some 2007 code from Source Engine games was finally able to shed some light on this subject. By some I mean we basically have some rather difficult to decipher C++, consisting of a few thousand lines in a single .cpp, and a couple of headers with definitions of different data types to send and receive.

Disclaimer: This write-up is based on investigations into Counter-Strike: Source’s protocols. Other games may vary wildly, particularly those based on Source 2009 or later.

Brief technology overview

This information is more broadly available, but lets quickly cover it for completeness’ sake. Source Engine communicates primarily over UDP. Packet contents is of a proprietary format, but doesn’t appear to be encrypted.

Anyway, lets crack on and establish first contact with a server.

First connection

As one would hope for any network protocol that would expect multiple back and forths, as well as a degree of trust between each end, we start with a client asking the server about connecting. So before we look at our first packet, we need to consider from the server’s point of view; who is this packet coming from?

Packet types

Source has 2 main formats of data it sends and receives. We’ll refer to these as Connectionless, and Connected packets.

A Connectionless packet can be easily identifed; as the contents will always begin with a 4 bytes like this: FF FF FF FF. Both the client and server will send packets like this before the client has proved their identity with the server. Following that is a single byte that represents the data type. Communication between both parties requires packets to be sent and received in the right order (obviously), but since this is UDP, assuming the neither party receives a packet that would cause them to stop communication (e.g. a Disconnect packet), it doesn’t seem to matter if they receive a load of junk or duplicate packets at this stage.

A Connected packet is a little more complicated, as it contains a kind of packet header before the main data. We’ll look at this header in a bit, as it’s contents won’t make much sense at this point.

First packet

Anyway, the client obviously establishes contact with the server, using a specific Connectionless packet, that is formatted like so:

[4]byte{255, 255, 255, 255}	// This is our Connectionless header
byte('q')					// This is our packet type. 'q' is used for our first connection request
int32(167679079)			// This is our challenge data. The specific value seems to be unimportant
string("0000000000")		// Must be exactly 10 "0"s. Doesn't appear to serve any specific purpose
byte(0)						// We must null terminate the string above. Source always null-terminates strings

The structure above is the entire data we would send over UDP. It is important to note this there is no 1-/4-byte alignment in this packet, so the total data size is 20 bytes.

Now, after we send this packet to our target server, we can expect to receive a response pretty quickly, assuming our request was correctly defined. If the server fails to respond, our request was likely malformed. From what I’ve seen, servers appear to discard junk packets, rather than respond with anything. This is reasonable.

So the server responds with a Connectionless packet. It has a simple structure too, just like this:

[4]byte{255, 255, 255, 255} // Connectionless header
byte('A')					// 'A' is what we want. '9' means we were refused
int32()						// Server issues a challenge value. This helps prove who we are later
int32()						// Server returns our challenge we sent just now

So the server accepted our request, and we can progress to verification of who we are.

Second packet

So at this point we have 2 values we received from the server: our and their challenge values. We will be using these now in our second packet; which has a structure, just like this:

[4]byte{255, 255, 255, 255}	// This is our Connectionless header
byte('k')					// packet type id
int32(0x18)					// is always 0x18, reason unknown
int32(0x03)					// is always 0x03, reason unknown
int32(serverChallenge)		// server challenge received from the previous response
int32(clientChallenge)		// our client challenge. can be obtained from the previous response
string(playerName)			// our desired playername
byte(0)						// null terminate playername
string(test789)				// server password ("test789" appears to be a default value sent if no password specified)
byte(0)						// null terminate password
string("4630212")			// Game version. I don't know how to obtain this currently without packet sniffing
byte(0)						// null terminate game version
int16(242)					// is always 242, reason unknown
uint64(steamID)				// SteamID. You can get this from Steamworks SDK SteamUser()->getSteamID()
[]byte(steamKey)			// An authentication ticket. Obtained from Steamworks SDK SteamUser()->CreateAuthTicket(). This value can be empty under certain circumstances

Whats immediately obvious here it that we are already progressing quickly with our connection. We’re already providing our player information, and client info now. There isn’t much of real interest here, until our last 2 entries. What is happening here is we are attempting to prove that we own the game we are trying to connect to. In order to do so, we must provide our Steam id to the server, along with a 1-use authentication ticket, provided to us by Steam [LINK].

It is the responsibility of the server to verify the authentication ticket is valid. This is as far as our interaction with steam goes. The server verifies our ticket when it receives our packet; and then is responsible for revoking that tickets validity when we leave the server.

Anyways, we send our packet back to the server, and wait for a response. Similar to before, we want a packet with a specific header byte. If we receive something else, we may need to handle it (such as Connection Refused packets).

What we want is something like this:

[4]byte{255, 255, 255, 255} // Connectionless header
byte('B')					// 'B' is for successful connection
[]byte{}					// Various padding. May have meaning, but not required to connect

Future

I plan on updating this post as I progress more through writing an implementation. You can track the progress of my netcode implementation here: https://github.com/Galaco/sourcenet