👀 Notes on this right here page 🤠

Generals64 keeps the multiplayer model used by Command & Conquer: Generals and Zero Hour: every player runs the full simulation, and the network distributes commands rather than world state.

There is no authoritative server sending unit positions, projectile states, or economy snapshots to clients. If I order five tanks to move, the network sends the order. Every machine then executes that order on the same logic frame and independently reaches the same result.

That sounds simple, but it gives the networking code a strict job. It must make sure every player has the same commands, executes them in the same order, uses the same random state, and does not advance until everyone is ready.

That is lockstep.

Commands replace state replication

A modern action game commonly sends snapshots and corrects clients when their local state drifts. Generals64 does almost the opposite. The synchronized unit is a GameMessage.

Commands generated by the UI enter the message stream. The network recognizes messages in the network range, wraps each one in a NetGameCommandMsg, assigns the local player and a unique command ID, and schedules it for a future logic frame:

Int Network::getExecutionFrame() {
    Int logicFrame = TheGameLogic->getFrame() + m_runAhead;
    if (logicFrame > m_lastExecutionFrame)
        m_lastExecutionFrame = logicFrame;
    return m_lastExecutionFrame;
}

The run-ahead is intentional input delay. If the simulation is on frame 1,000 and the run-ahead is four, a new order is assigned to frame 1,004. That gives the command time to reach every other machine before any of them needs to execute it.

sendLocalGameMessage() fills in the rest of the envelope:

NetCommandMsg *netmsg = newInstance(NetGameCommandMsg)(msg);
netmsg->setExecutionFrame(frame);
netmsg->setPlayerID(m_localSlot);
netmsg->setID(currentID);
sendLocalCommand(netmsg);

The receiving side stores commands by player and execution frame. It does not execute them as they arrive. Network arrival order is irrelevant.

A frame is complete when the counts match

The useful part of this protocol is how a peer knows it has received everything for a frame.

At the point where a local frame can no longer accept new commands, each player sends a NetFrameCommandMsg. It contains the frame number and the number of commands that player submitted for it:

UnsignedShort count = m_frameData[m_localSlot]->getCommandCount(frame);

NetFrameCommandMsg *msg = newInstance(NetFrameCommandMsg);
msg->setExecutionFrame(frame);
msg->setCommandCount(count);
msg->setPlayerID(m_localSlot);

This message means, in effect, “I am finished with frame N, and you should have K commands from me.”

Each peer keeps a FrameDataManager for every human slot. A frame is ready for one player only when the declared count equals the number actually received:

if (m_frameCommandCount == m_commandCount)
    return FRAMEDATA_READY;

if (m_commandCount > m_frameCommandCount)
    return FRAMEDATA_RESEND;

return FRAMEDATA_NOTREADY;

The game checks this for every active player. If any one of them is not ready, the simulation does not advance. The networking code continues receiving and retransmitting packets while the renderer can keep drawing, but the next logic tick waits.

Once all frame data is present, the network assembles the command list and opens the gate:

if (AllCommandsReady(TheGameLogic->getFrame())) {
    if (timeForNewFrame()) {
        RelayCommandsToCommandList(TheGameLogic->getFrame());
        m_frameDataReady = TRUE;
    }
}

The main game loop advances only when isFrameDataReady() returns true. Generals64 also keeps rendering separate from this lockstep gate and interpolates visual transforms between logic ticks. Network logic can remain fixed and deterministic without making camera movement and unit animation look like they are limited to the network rate.

Every peer must use the same order

Receiving the same set of commands is not enough. Every machine must process that set in the same order.

NetCommandList sorts commands first by network command type, then by player ID, then by the command's sort number, normally its unique command ID. Duplicates are rejected using the originating player and command ID.

That ordering matters whenever two players issue commands that interact during the same logic frame. Arrival timing cannot be allowed to decide which command wins. One peer may receive player 2's order first and another may receive player 0's order first, but both construct the same sorted list before handing it to game logic.

The result is a deterministic transaction for each frame:

Collect commands from every player.
Confirm the expected command counts.
Sort the combined list deterministically.
Execute the list on every machine.
Advance the simulation by one logic tick.

UDP, acknowledgements, and the packet router

The normal transport is UDP. Generals64 keeps packets below a conservative payload limit of 1,100 bytes to reduce fragmentation, and each transport packet carries its own CRC and the 0xF00D Generals magic value. There is also a separate LANAPI relay mode that carries the same destination-addressed packets through a TCP relay. That transport choice is independent of the lockstep and client-server simulation topology.

UDP does not provide delivery guarantees, so the connection layer adds them where the game needs them. Game commands, frame information, run-ahead changes, player-leave messages, and several control commands require acknowledgements. A Connection retains those messages after sending and retries them until the matching acknowledgement removes them from the queue.

Multiplayer also has a packet-router role. In the normal peer-to-peer topology, a non-router sends a command to the router with a recipient bitmask. The router adds the command to its own frame data and fans it out to the other players. The router is a traffic coordinator, not an authoritative simulation server.

Generals64 also supports a client-server network topology. In that mode, clients maintain a direct connection only to the host, while still keeping frame data for every human player. The host relays commands between them. The topology changes, but the simulation contract does not: every machine still runs the world and waits for the same lockstep command set.

The two-stage acknowledgement path follows the relay. The first stage confirms that the router received a command. The second confirms that the required recipients received the relayed copies. This lets the original sender retain pending commands if the router disappears halfway through forwarding them.

Recovering missing frame data

Normal packet loss is handled by command retries. There is also a frame-level recovery path.

If a peer receives more commands than the announced count, it clears the inconsistent frame and requests it again. Missing commands normally arrive through the acknowledgement and retry path. Generals64 keeps recent frame data in a ring buffer instead of discarding it immediately, so a peer can resend the stored commands and the corresponding NetFrameCommandMsg for each player.

for (UnsignedInt frame = startingFrame;
     frame < TheGameLogic->getFrame(); ++frame) {
    sendSingleFrameToPlayer(playerID, frame);
}

Keeping recent frames also helps when the packet router leaves. In peer-to-peer mode, the next slot in the shared fallback order becomes router, pending commands are sent again, and peers can exchange recent frame data to catch up. No world snapshot needs to migrate because every surviving machine already owns the world.

Client-server mode is deliberately different. If its server leaves, the match ends because clients do not have the full connection mesh needed for router promotion.

Run-ahead is the latency budget

A lockstep game needs enough lead time for commands to arrive, but every extra frame of lead is added input latency.

The packet router measures round-trip latency and reported simulation throughput. It computes run-ahead from its latency estimate, a configurable slack percentage, and the current logic frame rate:

const Real runAheadSlackScale = 1.0f +
    ((Real)TheGlobalData->m_networkRunAheadSlack / 100.0f);

Int newRunAhead = ceilf(
    getMaximumLatency() * runAheadSlackScale * (Real)minFps);

newRunAhead = clamp<Int>(
    MIN_RUNAHEAD, newRunAhead, MAX_FRAMES_AHEAD / 2);

A run-ahead change is itself a synchronized network command. Everyone applies the new value on the same execution frame. This avoids one client scheduling new orders against a different future-frame window than the others.

There is also a packet-arrival cushion. If commands are arriving too close to their execution frame, the local pacing code temporarily slows the logic rate to let the slow peer recover. This is why lockstep performance is tied to the slowest participant. The simulation can only advance as fast as every participant can produce and receive its frame data.

Determinism starts with the seed

Command lockstep works only if the simulation is deterministic.

The lobby distributes one game seed. At game start, every peer initializes the game random generator from that same value:

InitRandom(m_currentGame->getSeed());

From there, game-visible randomness must come from the synchronized generator in the same call order. The CRC calculation includes the game-logic random seed, so an extra random call on one machine becomes visible as a state mismatch.

The same rule applies to object iteration, AI, scripts, pathfinding, floating-point behavior, and container ordering. A network bug can cause a desync, but ordinary gameplay code can cause one just as easily if it produces a different result from identical input.

This is one of the difficult parts of moving an old deterministic simulation to 64-bit code. Lockstep exposes assumptions that a single-player run may never reveal.

The logic CRC detects divergence

Generals64 regularly walks the simulation and calculates a logic CRC. The input includes objects, the random seed, game logic, terrain, the partition manager, players, AI, scripts, sides, and the cave system.

The result is sent as another synchronized GameMessage. The current default checks every logic frame. Debug builds expose -NetCRCInterval for changing that interval:

m_CRC = getCRC(CRC_RECALC);
GameMessage *msg = newInstance(GameMessage)(
    GameMessage::MSG_LOGIC_CRC);
msg->appendIntegerArgument(m_CRC);
msg->appendBooleanArgument(isPlayback);
GameMessageList *messageList = TheMessageStream;
messageList->appendMessage(msg);

Because CRC messages travel through the same lockstep command path, every peer compares values representing the same simulation point. After processing the frame's command list, the game verifies that it received one CRC from every connected player and that all values match.

std::map<Int, UnsignedInt>::const_iterator crcIt =
    m_cachedCRCs.begin();
Int validatorCRC = crcIt->second;

while (++crcIt != m_cachedCRCs.end()) {
    if (validatorCRC != crcIt->second)
        sawCRCMismatch = TRUE;
}

A mismatch does not repair the world. At that point the simulations have already diverged, and there is no authoritative state to choose as correct. Instead, Generals64 records the failure, opens the mismatch flow, and writes additional diagnostics. The repository also includes a desync simulation harness that injects controlled divergence and checks the logging pipeline.

The packet CRC and logic CRC solve different problems. The packet CRC detects damaged network data. The logic CRC detects two valid command streams producing different worlds.

The tradeoff

Lockstep is a good fit for an RTS because player intent is much smaller than simulation state. A move order may be a few fields, while the result can involve hundreds of units, projectiles, pathfinding decisions, and economy updates. Sending commands saves bandwidth and makes replays natural because a replay is largely the initial state plus the same ordered command stream.

The cost is strict coupling. One slow peer can stall everyone. Input needs enough run-ahead to survive network latency. Every gameplay system must remain deterministic. A desync cannot be corrected by accepting a server snapshot.

Generals64's multiplayer code is therefore less about synchronizing objects and more about protecting a sequence. It schedules commands into future frames, confirms the exact count from every player, sorts them identically, retries lost data, and checks that the resulting worlds still match.

As long as that sequence remains identical, every machine can simulate the battle locally and arrive at the same result without sending the battle itself over the network.

I have collected a lot of sound effects and music over the years. Finding the right one should be a quick job: open a folder, type a few letters, listen, and drag the file into whatever I am working on.

In practice, every audio library manager and sampler I tried got in the way somehow. Some were slow with large libraries. Some installed an entire ecosystem when I only wanted to browse sounds. Others wanted an account, a cloud library, or a subscription before they would let me work with files already sitting on my disk.

I got frustrated and made my own. It is called Fast Sound Library, or FSL.

FSL is a native C++ Windows/macOS application built around one workflow: type to search, use the arrow keys to audition results, and drag the sound into a DAW or another application. It also has tags, favorites, waveform scrubbing, loop regions, reverse playback, speed and pitch controls, but those features all have to stay out of the way of browsing.

The main requirement was speed. I did not want an application that merely looked minimal while doing a large amount of work behind the scenes. Startup, search, rendering, decoding, and playback all had to be designed around that requirement.

A hardware-accelerated interface

FSL does not use Electron or a general-purpose cross-platform UI framework. I wrote a small retained-mode UI system on top of Win32, Direct2D and DirectWrite. The renderer creates a D3D11 hardware device and presents through a DXGI flip-model swap chain. The maximum frame latency is set to one, which keeps input and drawing responsive. There is a WARP fallback for machines where a hardware D3D device cannot be created, but the normal path is GPU accelerated.

This is not just used for a decorative visualizer. The complete interface, including the library, waveform, text, SVG icons and controls, is rendered through Direct2D on the D3D11 surface. Some visualizer backgrounds also use HLSL shaders directly.

The UI is retained rather than redrawn as a pile of unrelated immediate-mode calls. Controls form an element tree with measure, arrange and paint passes. Dirty flags determine what needs another layout or paint pass. The library list is virtualized, so a library containing thousands of files still only creates and paints the rows visible on screen.

The DXGI swap chain uses two buffers and flip-sequential presentation. Combined with a 60 Hz transport tick and a one-frame latency limit, it feels like a native tool because it is one.

Loading a library without decoding it

Scanning an audio library and playing an audio file are separate jobs in FSL.

When a folder is added, FSL first enumerates files and filters them by supported extension. It does not decode every file just to put it in the library. The second phase performs a lightweight metadata probe to read the duration, sample rate, channel count and format. Media Foundation handles the normal Windows codecs, and ffprobe can fill in the gaps when FFmpeg is available.

The resulting index is kept in memory and saved as JSON under the user's local application data folder. On the next launch, FSL loads that index directly instead of scanning and decoding the complete library again. Paths and numeric IDs have their own lookup tables, while search uses a prebuilt lowercase search blob and a lazily rebuilt name order. Typing into the search box does one pass over the existing index without sorting the complete result again for every key press.

A full rescan runs away from the UI thread. Directory enumeration stays single threaded so several iterators do not fight over the same disk, then metadata probing is distributed across a small worker pool bounded by the machine's hardware concurrency. Routine saves have their own thread and are debounced, so changing a tag does not perform disk I/O inside an input handler.

FSL also watches library folders with ReadDirectoryChangesW using overlapped I/O. Added, removed, modified and renamed files update the index in the background. This avoids the usual choice between constantly rescanning a library and letting it become stale.

Fast auditioning and broad codec support

The expensive part, a full audio decode, only happens when a sound is selected for playback.

That decode runs on a dedicated worker thread. Audition requests use a generation number, so moving quickly through the list cancels stale work and makes the newest selection win. The UI thread remains free to handle input while Media Foundation or FFmpeg opens the file. Recently decoded sounds are kept in a small LRU-style cache, limited to three entries and 256 MB, which makes moving back and forth between nearby results nearly instant.

Media Foundation is the first decoder for WAV, AIFF, MP3, FLAC, M4A, WMA, OGG and the other formats available through the installed Windows codecs. If it cannot open a file, FSL can fall back to FFmpeg. That adds formats such as APE, Opus, tracker modules, WavPack and quite a few older or less common codecs without making FFmpeg a requirement for ordinary use.

The decoder converts audio to interleaved 32-bit floating point PCM. Files that decode to less than 64 MB stay in memory. Larger previews spill to a temporary file and are memory mapped, so opening a long recording does not require one enormous allocation. A peak envelope for the waveform is generated once when the file loads rather than being recalculated every frame.

Playback goes through XAudio2. The decoded buffer is submitted to a source voice, and XAudio2 handles low-latency playback, seeking, looping and frequency-ratio changes. Loop selections are sent as actual XAudio2 loop points. Speed and pitch use the source voice frequency ratio, which gives FSL its 0.25x to 4x varispeed range without rebuilding the buffer every time a control moves.

The result is the application I wanted in the first place: local, hardware accelerated, quick to open, and quick to audition. There is no account to create, no subscription to maintain, and no attempt to move a sound library into somebody else's cloud. FSL opens the folders I already have and lets me get back to work.

You can try the offline web edition at fastsoundlibrary.com. The native Windows edition is also coming to Steam.

Multiplayer in Froggy Works does not use a traditional game server that owns the world and sends the result to everyone else. Every player runs a complete copy of the simulation locally.

The cloud server still has a job, but it is deliberately small. It creates sessions, assigns join codes, relays WebRTC signaling, tracks membership, elects a coordinator, and stores recovery snapshots. It does not run farms or tick game worlds.

This was an important distinction for me. I wanted local input to feel like local input, even when the game was online. Waiting for a server round trip before a frog starts walking would make the controls feel worse for everyone, including the person who created the farm.

A complete world on every device

Each participant owns a local FfServer and FfClient. The server half advances the world, and the client half handles input, prediction, presentation, and rendering. Windows, macOS, and iOS use the native C/C++ implementation. The browser runs the same C core compiled to WebAssembly.

With one player, online play is almost the same as offline play. There is no peer connection to maintain. Commands are applied to the local simulation, and the result is rendered immediately.

When another player joins, WebRTC data channels are opened between the peers. The architecture does not suddenly switch to streaming a host's world. Both players continue running their own worlds. They exchange commands and compact owned-unit state so those worlds converge.

The basic command path is:

Predict and apply the command locally.
Render the local result immediately.
Wrap the command with the player's stable owner ID.
Send it to the coordinator over WebRTC.
Validate and relay it to the other peers.
Apply it to each peer's local simulation.

The local command router is intentionally ordered this way:

status = ff_core_client_predict_locally(
    client, command, &track_predicted_sequence);

status = ff_server_apply_command_json(local_server, command_wire);
status = ff_core_client_drain_local_server(client);

if (status == FF_OK && client->peer_local_replication != 0) {
    status = ff_core_client_send_command_wire(client, command_wire);
}

The network send comes after the local application. The player does not wait for WebRTC, SignalR, the coordinator, or another simulation tick before seeing the action.

Commands instead of world dumps

Gameplay messages use a small envelope around the normal Froggy Works command format:

{
  "kind": "peer-command",
  "ownerId": "stable-player-id",
  "command": { "$type": "Move", "TargetX": 312, "TargetY": 208 }
}

There are matching peer-join and peer-leave messages. This makes player lifecycle explicit and, importantly, idempotent. Receiving the same join twice does not spawn two characters, and receiving a repeated leave after the character is already gone does nothing.

The command body is the same command system used by local play. Planting, building, harvesting, weather changes, movement, possession, and the rest of the game do not need separate multiplayer versions. Multiplayer is mostly a transport around the existing simulation boundary.

Native clients parse and validate the envelope before touching the world. The coordinator also checks that the ownerId in the message matches the identity assigned to that WebRTC peer:

if (!parsePeerMessage(commandJson, message) ||
    message.kind != PeerMessageKind::Command ||
    message.ownerId != stableOwner ||
    !server.applyPlayerCommand(stableOwner, message.commandJson)) {
    return;
}

transport.broadcastCommand(commandJson);

That check does not make the game cheat-proof. Peer-local multiplayer deliberately trusts the clients, and a modified client can cheat. It does stop a peer from casually putting another player's identity into an otherwise valid packet.

For movement, each player owns the feel of their controlled character and publishes compact position updates at 20 Hz. Other peers apply those updates to their local copy. Most actions are event-like commands, so there is no reason to continuously serialize and transmit the whole farm for normal gameplay.

The coordinator is not a simulation host

One peer is called the coordinator, but the name can be misleading. It is not the only peer running the game, and the guests are not remote renderers.

The coordinator is a stable ordering and relay point. That matters when two players try to claim the same thing at the same time. Possessing another unit is the obvious example. Sitting on furniture has the same problem because a seat can only have one occupant.

Those contested operations take a different route. A guest predicts presentation where appropriate, but does not commit the operation to its local simulation first. It sends the request to the coordinator, which validates and relays the accepted result. Ordinary commands still use the immediate local-first path.

The split is small in the core:

if (command->type == FF_CMD_POSSESS) {
    return true;
}
if (command->type != FF_CMD_USE_STRUCTURE) {
    return false;
}
return def != NULL && def->seats.len > 0;

This avoids turning every action into a network round trip just because a few actions need a single ordering decision.

WebRTC for gameplay, SignalR for introductions

The cloud server uses SignalR on /webrtchub, but gameplay does not travel through that hub. SignalR handles the control plane: session creation, joining, protocol negotiation, SDP offers and answers, ICE candidates, membership changes, and coordinator election.

Once peers have negotiated WebRTC, the game data moves over four named data channels:

createDataChannel("ctl", reliable);
createDataChannel("cmd", reliable);
createDataChannel("snap", reliable);
createDataChannel("pos", unreliableUnordered);

ctl carries welcome and lifecycle data. cmd carries the peer command envelopes and is reliable and ordered. snap and pos remain for compatibility with the older host-mirror path. A current peer-local guest already has a full simulation, so it ignores those legacy streams during normal play.

The server relays signaling only between peers in the same session. It also limits signaling payload sizes and rates before forwarding them. TURN credentials are generated server-side when direct peer connectivity is not possible, so no long-lived provider secret is shipped in the game.

Checkpoints and host migration

Local-first does not mean disposable. The coordinator uploads a full recovery save when the session is created, every 12 seconds, when players join or leave, and during shutdown when possible.

The cloud stores that snapshot with the permanent farm code and membership data. It never simulates the snapshot. Before storage, it removes player characters whose stable identities are no longer connected, so a stale disconnect cannot be resurrected by a later checkpoint.

When the last player leaves, the farm becomes paused rather than deleted. Joining the same code later restores the snapshot into a local simulation, and the returning player becomes the new coordinator.

Migration while people are still connected uses the same idea. The signaling server elects a surviving peer and sends it the latest recovery snapshot. The promoted guest ends its guest stream, loads the snapshot as a recovery fence, installs the coordinator broadcaster, and continues:

activation.migrationSnapshotJson = std::move(snapshotJson);
activation.migration = true;

if (guestActive) {
    guestBridge->endStream();
}

hostBridge->activate(activation, *runtime, &unitEntityId, &error);
roleValue = P2PSessionRole::Host;

Because every peer was already running a complete world, migration is mostly a routing change. The promoted peer is not being upgraded from a passive renderer into a game server.

The tradeoff

This design optimizes for responsive cooperative play, not competitive security. The server cannot fully validate a simulation it never runs. Two clients can also diverge if a command is dropped, interpreted differently, or applied in a different order, which is why the protocol has strict message validation, stable owner identities, coordinator ordering for contested actions, position convergence, and durable recovery snapshots.

The useful result is that online play still feels local. The network carries intent and recovery data, not every frame of the game. The cloud can disappear from the hot gameplay path without disappearing from session management or persistence.

That is the main idea behind Froggy Works multiplayer: run the game everywhere, synchronize the decisions that matter, and keep the server out of the way unless it has a specific job to do.

The aim of these notes is to take the OSM planet PBF file, process it, and create an indexed SQLite database of all the POIs and tags.

During the process I will also demonstrate how to split the PBF file into chunks for parallel processing. WSL2 Ubuntu RAM disks will be used to speed up the processing. osm and osm.bz2 files are created for each chunk and all chunks merged as well.

Initially all the pre-processing is done on WSL2 Ubuntu with 128GB of RAM and 36 CPU cores.

First, the WSL2 virtual machine has to be configured to use more memory than its default configuration. To do this, create a file on the Windows user’s home directory named .wslconfig containing:

[wsl2]
memory=110GB
swap=110GB

Save and restart the VM by running wsl --shutdown. You can use htop to verify how much memory and swap WSL has after this modification.

Download the latest OSM planet PBF file (e.g. planet-250526.osm.pbf) and copy it into your working directory. In this example, the working directory is /mnt/f/osm_vm_data and the RAM disk is mounted at /mnt/ramdisk/.

Osmium and Parallel are needed for our processing scripts:

sudo apt install -y parallel
sudo apt install -y osmium-tool
sudo apt install -y osmosis

We run a script to create a RAM disk, split and create PBF files. Adjust paths and use RAM disk as desired:

#!/bin/bash
# Run on Ubuntu with osmium-tool, parallel, and Overpass API installed
set -x
# Configuration
INPUT_PBF="/mnt/ramdisk/planet-250526.osm.pbf"
CHUNKS=33  # Number of parallel chunks (adjust based on CPU cores)
RAMDISK_SIZE="110G"  # Adjust based on available RAM

# Check dependencies
command -v osmium >/dev/null 2>&1 || { echo "osmium-tool not installed. Run: sudo apt install osmium-tool"; exit 1; }
command -v parallel >/dev/null 2>&1 || { echo "parallel not installed. Run: sudo apt install parallel"; exit 1; }
command -v bzip2 >/dev/null 2>&1 || { echo "bzip2 not installed. Run: sudo apt install bzip2"; exit 1; }

# Create RAM disk for faster I/O
#sudo mkdir -p /mnt/ramdisk
#sudo mount -t tmpfs -o size="$RAMDISK_SIZE" tmpfs /mnt/ramdisk

# Copy input file to RAM disk
cp "/mnt/f/osm_vm_data/planet-250526.osm.pbf" /mnt/ramdisk/

seq 0 $((CHUNKS-1)) | parallel -j8 'min_lon=$(({}*360/'$CHUNKS'-180)); max_lon=$((({}+1)*360/'$CHUNKS'-180)); osmium extract -b "$min_lon,-90,$max_lon,90" "'$INPUT_PBF'" -o "/mnt/f/osm_vm_data/chunks/chunk{}.pbf"'

# Process chunks in parallel to extract all amenity nodes
ls /mnt/f/chunks/chunk*.pbf | parallel -j"$CHUNKS" osmium tags-filter {} n/amenity -o {.}.osm

# Merge results
osmium merge /mnt/ramdisk/chunk*.osm -o /mnt/ramdisk/amenities.osm

# Compress for Overpass
bzip2 /mnt/ramdisk/amenities.osm

This script can be adjusted based on needs to create chunks or wholes of PBF, OSM, bz2 files with only the POIs.

More POIs can be extracted by adding more types to the script. For example:

osmium tags-filter /mnt/ramdisk/planet-latest.osm.pbf n/amenity,shop,tourism,leisure,office,craft,emergency,highway=bus_stop -o /mnt/ramdisk/pois.pbf

Build JSONs from OSM data

In this section, I will build a text file, new-line separated, where each line is a separate JSON file containing the data of a POI.

#include <osmium/io/any_input.hpp>
#include <osmium/handler.hpp>
#include <osmium/visitor.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/thread/pool.hpp>
#include "nlohmann/json.hpp"
#include <fstream>
#include <vector>
#include <thread>
#include <mutex>
#include <queue>
#include <condition_variable>
#include <iostream>
#include <string>
#include <bzlib.h>
#include <stdexcept>

// Thread-safe queue for JSON lines
class ThreadSafeQueue {
    std::queue<std::string> queue_;
    std::mutex mutex_;
    std::condition_variable cond_;
    bool done_ = false;

public:
    void push(std::string item) {
        std::lock_guard<std::mutex> lock(mutex_);
        queue_.push(std::move(item));
        cond_.notify_one();
    }

    bool pop(std::string& item) {
        std::unique_lock<std::mutex> lock(mutex_);
        cond_.wait(lock, [this] { return !queue_.empty() || done_; });
        if (queue_.empty() && done_) return false;
        item = std::move(queue_.front());
        queue_.pop();
        return true;
    }

    void set_done() {
        std::lock_guard<std::mutex> lock(mutex_);
        done_ = true;
        cond_.notify_all();
    }
};

// Osmium handler to process nodes
struct AmenityHandler : public osmium::handler::Handler {
    ThreadSafeQueue& output_queue_;
    explicit AmenityHandler(ThreadSafeQueue& queue) : output_queue_(queue) {}

    void node(const osmium::Node& node) {
        if (node.tags().has_key("amenity")) {
            nlohmann::json j;
            j["id"] = node.id();
            j["lat"] = node.location().lat();
            j["lon"] = node.location().lon();
            j["tags"] = nlohmann::json::object();
            for (const auto& tag : node.tags()) {
                j["tags"][tag.key()] = tag.value();
            }
            output_queue_.push(j.dump());
        }
    }
};

// Writer thread function
void writer_thread(ThreadSafeQueue& queue, const std::string& output_file) {
    std::ofstream out(output_file, std::ios::out | std::ios::binary);
    if (!out.is_open()) {
        throw std::runtime_error("Cannot open output file: " + output_file);
    }
    std::string line;
    while (queue.pop(line)) {
        out << line << '\n';
    }
    out.close();
}

int main(int argc, char* argv[]) {
    if (argc != 3) {
        std::cerr << "Usage: " << argv[0] << " <input_file> <output_file>\n";
        return 1;
    }

    std::string input_file = argv[1];
    std::string output_file = argv[2];

    try {
        // Initialize thread-safe queue
        ThreadSafeQueue queue;

        // Start writer thread
        std::thread writer(writer_thread, std::ref(queue), output_file);

        // Set up Osmium reader
        osmium::io::File file(input_file);
        osmium::io::Reader reader(file, osmium::osm_entity_bits::node);

        // Set up handler and thread pool
        AmenityHandler handler(queue);
        osmium::thread::Pool pool(std::thread::hardware_concurrency());

        // Apply handler to OSM data
        osmium::apply(reader, handler);

        // Close reader
        reader.close();

        // Signal writer to finish
        queue.set_done();
        writer.join();
    } catch (const std::exception& e) {
        std::cerr << "Error: " << e.what() << '\n';
        return 1;
    }

    std::cout << "Output written to " << output_file << '\n';
    return 0;
}

To build this, we need libosmium, nlohmann/json and libbz2:

sudo apt update
sudo apt install libosmium2-dev libbz2-dev zlib1g-dev

Usage:

./osm_to_jsonl amenities.osm.bz2 amenities.jsonl

JSON to SQLite

We’ll use a C# program to build an indexed SQLite database file from the JSONs.

using System.Collections.Concurrent;
using System.Data.SQLite;
using System.Text;

using Newtonsoft.Json.Linq;

class Program
{
    private class PoiData
    {
        public long Id { get; set; }
        public double Lat { get; set; }
        public double Lon { get; set; }
        public string Name { get; set; }
        public string Wikidata { get; set; }
        public JObject Tags { get; set; }
        public string RawJson { get; set; }
        public int LatInt { get; set; }
        public int LonInt { get; set; }
    }

    private class SQLiteConnectionWrapper
    {
        private readonly SQLiteConnection _connection;
        private readonly object _lock = new object();

        public SQLiteConnectionWrapper(string dbPath)
        {
            _connection = new SQLiteConnection($"Data Source={dbPath};Version=3;");
            _connection.Open();
        }

        public void ExecuteNonQuery(string sql, params SQLiteParameter[] parameters)
        {
            lock (_lock)
            {
                using (var cmd = new SQLiteCommand(sql, _connection))
                {
                    if (parameters != null) cmd.Parameters.AddRange(parameters);
                    cmd.ExecuteNonQuery();
                }
            }
        }

        public long ExecuteScalar(string sql, params SQLiteParameter[] parameters)
        {
            lock (_lock)
            {
                using (var cmd = new SQLiteCommand(sql, _connection))
                {
                    if (parameters != null) cmd.Parameters.AddRange(parameters);
                    return Convert.ToInt64(cmd.ExecuteScalar());
                }
            }
        }

        public void Vacuum()
        {
            ExecuteNonQuery("VACUUM");
        }

        public void Close()
        {
            lock (_lock)
            {
                _connection.Close();
            }
        }
    }

    static async Task Main(string[] args)
    {
        string inputFile = args.Length > 0 ? args[0] : "r:/amenities.jsonl";
        string dbFile = args.Length > 1 ? args[1] : "r:/amenities.db";
        const int batchSize = 100000;

        try
        {
            // Initialize SQLite database
            if (File.Exists(dbFile)) File.Delete(dbFile);
            var db = new SQLiteConnectionWrapper(dbFile);

            // Optimize SQLite settings
            db.ExecuteNonQuery("PRAGMA cache_size = -2000000;"); // 2GB cache
            db.ExecuteNonQuery("PRAGMA temp_store_directory = 'r:/';");

            // Create tables
            db.ExecuteNonQuery(@"
                CREATE TABLE pois (
                    id INTEGER PRIMARY KEY,
                    lat REAL NOT NULL,
                    lon REAL NOT NULL,
                    name TEXT,
                    lat_int INTEGER NOT NULL,
                    lon_int INTEGER NOT NULL,
                    wikidata TEXT,
                    raw_json TEXT
                );

                CREATE TABLE tags (
                    tag_id INTEGER PRIMARY KEY AUTOINCREMENT,
                    key TEXT NOT NULL,
                    value TEXT NOT NULL
                );

                CREATE TABLE poi_tags (
                    poi_id INTEGER NOT NULL,
                    tag_id INTEGER NOT NULL,
                    PRIMARY KEY (poi_id, tag_id),
                    FOREIGN KEY (poi_id) REFERENCES pois(id),
                    FOREIGN KEY (tag_id) REFERENCES tags(tag_id)
                );
            ");

            // Deduplicate tags
            var tagCache = new ConcurrentDictionary<(string key, string value), long>();

            // Process JSON Lines in parallel with partitioning
            await Task.Run(() =>
            {
                var lines = File.ReadLines(inputFile);
                Parallel.ForEach(Partitioner.Create(0, lines.LongCount(), batchSize), range =>
                {
                    var batch = lines.Skip((int)range.Item1).Take((int)(range.Item2 - range.Item1)).Select(line =>
                    {
                        try
                        {
                            var json = JObject.Parse(line);
                            return new PoiData
                            {
                                Id = json["id"].Value<long>(),
                                Lat = json["lat"].Value<double>(),
                                Lon = json["lon"].Value<double>(),
                                Tags = json["tags"].Value<JObject>(),
                                RawJson = line,
                                LatInt = (int)(json["lat"].Value<double>() * 100000),
                                LonInt = (int)(json["lon"].Value<double>() * 100000),
                                Name = json["tags"]?["name"]?.Value<string>(),
                                Wikidata = json["tags"]?["wikidata"]?.Value<string>()
                            };
                        }
                        catch (Exception ex)
                        {
                            Console.WriteLine($"Error parsing line: {ex.Message}");
                            return null;
                        }
                    }).Where(poi => poi != null).ToList();

                    if (batch.Count > 0)
                    {
                        ProcessBatch(db, batch, tagCache);
                    }

                    Console.WriteLine($"Processed batch from {range.Item1} to {range.Item2} with {batch.Count} POIs.");
                });
            });

            Console.WriteLine("Building indexes...");

            // Create indexes in a transaction
            db.ExecuteNonQuery("BEGIN TRANSACTION;");
            db.ExecuteNonQuery(@"
                CREATE INDEX idx_pois_name ON pois(name);
                CREATE INDEX idx_pois_lat_int ON pois(lat_int);
                CREATE INDEX idx_pois_lon_int ON pois(lon_int);
                CREATE INDEX idx_pois_wikidata ON pois(wikidata);
                CREATE INDEX idx_tags_key ON tags(key);
                CREATE INDEX idx_tags_value ON tags(value);
                CREATE INDEX idx_poi_tags_poi_id ON poi_tags(poi_id);
                CREATE INDEX idx_poi_tags_tag_id ON poi_tags(tag_id);
            ");
            db.ExecuteNonQuery("COMMIT;");

            Console.WriteLine("Indexes created.");

            // Compact the database
            db.Vacuum();
            db.Close();

            Console.WriteLine($"Database created at {dbFile}");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error: {ex.Message}");
        }
    }

    static void ProcessBatch(SQLiteConnectionWrapper db, List<PoiData> batch, ConcurrentDictionary<(string key, string value), long> tagCache)
    {
        // Batch INSERT for pois
        var poiSql = new StringBuilder("INSERT INTO pois (id, lat, lon, name, lat_int, lon_int, wikidata, raw_json) VALUES ");
        var poiParams = new List<SQLiteParameter>();
        for (int i = 0; i < batch.Count; i++)
        {
            var poi = batch[i];
            poiSql.Append($"(@id{i}, @lat{i}, @lon{i}, @name{i}, @lat_int{i}, @lon_int{i}, @wikidata{i}, @raw_json{i})");
            if (i < batch.Count - 1) poiSql.Append(",");
            poiParams.AddRange(new[]
            {
                new SQLiteParameter($"@id{i}", poi.Id),
                new SQLiteParameter($"@lat{i}", poi.Lat),
                new SQLiteParameter($"@lon{i}", poi.Lon),
                new SQLiteParameter($"@name{i}", poi.Name ?? (object)DBNull.Value),
                new SQLiteParameter($"@lat_int{i}", poi.LatInt),
                new SQLiteParameter($"@lon_int{i}", poi.LonInt),
                new SQLiteParameter($"@wikidata{i}", poi.Wikidata ?? (object)DBNull.Value),
                new SQLiteParameter($"@raw_json{i}", poi.RawJson)
            });
        }

        if (batch.Count > 0)
        {
            db.ExecuteNonQuery(poiSql.ToString(), poiParams.ToArray());
        }

        // Batch INSERT for tags
        var tagSql = new StringBuilder("INSERT OR IGNORE INTO tags (key, value) VALUES ");
        var tagParams = new List<SQLiteParameter>();
        var uniqueTags = new HashSet<(string key, string value)>();
        int tagIndex = 0;

        foreach (var poi in batch)
        {
            foreach (var tag in poi.Tags)
            {
                var keyValue = (key: tag.Key, value: tag.Value.ToString());
                if (uniqueTags.Add(keyValue))
                {
                    tagSql.Append($"(@key{tagIndex}, @value{tagIndex})");
                    tagSql.Append(",");
                    tagParams.AddRange(new[]
                    {
                        new SQLiteParameter($"@key{tagIndex}", keyValue.key),
                        new SQLiteParameter($"@value{tagIndex}", keyValue.value)
                    });
                    tagIndex++;
                }
            }
        }

        if (uniqueTags.Count > 0)
        {
            tagSql.Length--; // Remove trailing comma
            db.ExecuteNonQuery(tagSql.ToString(), tagParams.ToArray());

            // Update tagCache
            foreach (var kv in uniqueTags)
            {
                tagCache.GetOrAdd(kv, _ =>
                {
                    return db.ExecuteScalar(
                        "SELECT tag_id FROM tags WHERE key = @key AND value = @value",
                        new SQLiteParameter("@key", kv.key),
                        new SQLiteParameter("@value", kv.value)
                    );
                });
            }
        }

        // Batch INSERT for poi_tags
        var poiTagSql = new StringBuilder("INSERT OR IGNORE INTO poi_tags (poi_id, tag_id) VALUES ");
        var poiTagParams = new List<SQLiteParameter>();
        int poiTagIndex = 0;

        foreach (var poi in batch)
        {
            foreach (var tag in poi.Tags)
            {
                var keyValue = (key: tag.Key, value: tag.Value.ToString());
                if (tagCache.TryGetValue(keyValue, out var tagId))
                {
                    poiTagSql.Append($"(@poi_id{poiTagIndex}, @tag_id{poiTagIndex})");
                    poiTagSql.Append(",");
                    poiTagParams.AddRange(new[]
                    {
                        new SQLiteParameter($"@poi_id{poiTagIndex}", poi.Id),
                        new SQLiteParameter($"@tag_id{poiTagIndex}", tagId)
                    });
                    poiTagIndex++;
                }
            }
        }

        if (poiTagIndex > 0)
        {
            poiTagSql.Length--; // Remove trailing comma
            db.ExecuteNonQuery(poiTagSql.ToString(), poiTagParams.ToArray());
        }
    }
}

The free ImDisk utility for Windows allows RAM drives to be created, which is helpful to speed up the process.

Ed 2.23 is out on Steam. These are the new features and changelog for this build. I started writing changes on Post-its and I plan to collect them for every release to compile a changelog. Old school methods seem faster than loading Trello/Jira’s JavaScript!

Changelog

CameraTarget flag: Make camera move and focus on this tile*.
TileDef list sort: Sort by Ref, Handle or default.
Draw borders around tiles of the current board (CTRL+G to toggle)
Duplicate Ref finder assistant: Find assets with the same Ref and quickly rename them to unique values
Fix bugs related to Rule Brush, Folder and TileDef management
Fix undo drawing making the camera jump to 0,0.^
Make undo drawing faster.^
Make zooming in and out with scroll wheel smoother.
Zoom and move the camera to cursor.
Circle brush with size 3 is + instead of a square.
Fix issues with animations and optimize drawing performance.
Announcer UI and chimes
IsPlayer TileDef flag
Performance improvement: Don’t run polish on tiles that are being drawn
Refactor renderer
Pan camera with space bar
CTRL+D to deselect
Export animations to Unity and play them*
Export collisions to Unity*
Flag to clone TileDef per tile on runtime or use static (shared) data
New animation code in Editor and Unity*
CTRL+S to save
Annotation TileDef flag: Ignore exporting/spawning sprites in game*
Unity tile Become()*
Unity tile Mimic()*
Tile-level collision inheritance*
Board-level collision override*
Draw colliders*
Duplicating assets assigns unique Refs
Gravity multiplier*
Rotation and position locks*
Physics simulation mode per TileDef (with export to Unity RigidBody2D)*
Propagate colliders to all TileDefs of a Rule Brush

(*Unity Experimental)

(^Known issues exist when undoing draws)

KiloTexture - OptimizedWall

Unity integration is coming along, with a focus on making a 2D platformer game mode work. I added a specific TileDef flag named OptimizedWall, which separates colliders from sprites, batches sprites into 2048x2048 pixels (or smaller) textures and draws those bigger textures instead of spawning small tiles individually. I’m tempted to call it KiloTexture as an homage to idTech’s MegaTexture.

The colliders are run through a merging algorithm that returns polygons of the union of rect colliders, and those polygons are spawned as Polygon Collider 2Ds in Unity.

Animated tiles are excluded from this process and are rendered separately.

Unity Export

Another challenge is to find the most optimal way to store data for Unity. Currently I’m writing data as code, because it’s fast at runtime, and can be accessed in any way they want by the programmer, git friendly, and readable. However, Unity Editor tends to re-compile scripts every time they change, and these large source files make the development and debug cycle slow, so I might move data out to resource files.

This is how large the main data source file is.

On another note, spawning a medium-sized level at edit times and saving the Unity scene file makes the scene file as large as 500 MB or more, and that already categorizes the file as LFS by Github. What’s the point of making things text-based if they aren’t properly handled by Git? So I think I should rethink how I’m serializing data in general.

Games

The main reason I’m focused on Unity integration and specifically on making the 2D platformer thing work is, if I can reduce the development to release cycle of an okay 2D platformer down to a month or so, Ed can be used at game jams to make platformers, and meanwhile I can bring artists in to create a game per month, early 90s style. We’ll see how it goes.

I’ve been working on a level editor for tile based games for quite some time now. Initially, it started as a tool to quickly design level for my Toy Box Jam submission, as I already had the gameplay code in place and needed to make some levels and swap sprites.

It’s now a polished tool, available for Windows, macOS and iOS, which you can use anywhere you want to design your games.

Here’s a list of features:

Rule-based brushes

Set up rule-based brushes for terrains, platforms and other features by assigning rules to sprites, so the appropriate sprites are automatically chosen depending on their neighbors. Built-in support for bitmasking, 47-tiles and 4x4 tiles terrains.

Tile definition classes

Set up tile definitions with various animation settings, background and tint colors and override their sizes and offsets on the grid. Additionally, you can assign variables and values to them and write custom scripts for tile definitions.

Sprite management

Import and slice your sprite sheets, remove opaque backgrounds by specifying a key color and set up tile bitmasks and brushes right within the sprite sheet editor and export them to rule brushes.

Auto terrain

Automatically generate an entire 47-tile terrain set from a single sprite. Adjust corner radiuses individually, and optionally stamp decal sprites on different sides. Export generated terrains as PNG and further edit them in your image editing programs.

Auto brush

White box your levels and puzzles with generic tiles and define auto brush regions that automatically transform them into high quality environments. Swap environments instantly without having to redraw levels.

Multi-layer level design

Create multiple boards per level and benefit from onion tools to draw on different layers with ease. Export levels as PNGs or ASCII-based scripts. Use Quads to add non-grid based 2D entities to your levels.

Fully-customizable boards

Adjust the grid size of boards individually, as well as modifying the horizontal and vertical gaps, background colors, alpha blending modes and tint colors. Each object in a level has its own parallax settings to enable parallax scrolling effects.

Create and draw patterns

Draw repetitive areas by creating patterns from a selected set of tiles. Stamp them with precision, or use the rectangle or ellipse fill tool to paint the pattern in a repetitive style on a designated area. Patterns can be created from any board and drawn on different boards or even other levels.

Color palettes and Pixel art

Import color palettes, create your own, and draw solid colors on grids. Create your pixel art directly in Ed, export them as a linked-sprite sheet back to the sprite sheet editor and slice them into sprites, redrawing them on your game levels. End-to-end, from pixel art into games, can be achieved without ever leaving Ed.

Design anytime, anywhere

Ed is available on your desktop, tablet and even iPhone. Open your Ed files on your mobile device and design your game on the go. Cloud sync and team accounts coming soon!

Download

You can get Ed on the following platforms:

> Get on Steam for Windows and MacOS

> Microsoft Store for Windows

> App Store for iPad and iPhone

If you’re looking for a way to develop cross-platform desktop-oriented applications, Avalonia UI might just be the framework for you. However, the documentation and third-party libraries can still be a bit immature. In this post, I’ll be sharing my experience developing an application that required playing audio files on different platforms, and how I managed to get it to work.

Goal

Our goal is to play audio files from a local file or a URL. To achieve this, the cross-platform code calls a function pointer that is defined separately on each platform as Func<string, Task>, where the input is the path/URL to the audio file. You can set this up with interfaces and dependency injection, but I chose to keep it old school.

Windows

Avalonia’s default template comes with one single Desktop project. However, to make platform-specific calls, we’ll need to fork that into two separate projects for Windows and macOS. To get started with Windows, change the TargetFramework of your Windows project to net7.0-windows10.0.17763.0. Note that you can’t target Windows 7, and at a minimum, you’ll need to target Windows 10 to get access to the APIs we’ll be using. Your csproj should look like this:

  <PropertyGroup>
    <OutputType>WinExe</OutputType>
    <TargetFramework>net7.0-windows10.0.17763.0</TargetFramework>
    <Nullable>enable</Nullable>
    <BuiltInComInteropSupport>true</BuiltInComInteropSupport>
  </PropertyGroup>

Once you’ve made this change and reloaded your project, you’ll gain access to the Windows.Media namespaces, which allow you to easily play media files using the Windows.Media.Playback.MediaPlayer class:

var player = new MediaPlayer();
player.Source = MediaSource.CreateFromUri(new Uri("file://c:/media.m4a"));
player.Play();

MediaSource supports loading local files using file:// Uris or remote files.

macOS

On macOS, we can use the afplay command-line utility to play audio files. To do this, we can simply write a function that runs a new process for afplay [path] to play the sound effect we want:

new Process()
{
    StartInfo = new ProcessStartInfo
    {
        FileName = "/bin/bash",
        Arguments = $"-c \"afplay 'media.mp4'\"",
        RedirectStandardOutput = true,
        RedirectStandardInput = true,
        UseShellExecute = false,
        CreateNoWindow = true,
    }
}.Start();

This is exactly what the NetCoreAudio project does for its macOS implementation and in general on Unix.

If you want to download a remote file, you can use System.Net.Http.HttpClient to do that:

using var client = new HttpClient();
var bytes = await client.GetByteArrayAsync(url);
File.WriteAllBytes(localPath, bytes);

WebAssembly

Playing audio on the web can be done using an Audio object in JavaScript and calling the play() method on it. To call JavaScript functions from your C# code, you can use the JSImport attribute.

First, define a function in main.js that plays an audio file, for example:

globalThis.playSound = function (url) {
    var audio = new Audio(url);
    audio.play();
}

Then you can JSImport this as a C# method anywhere you want like this:

using System.Runtime.InteropServices.JavaScript;

...

[JSImport("globalThis.playSound")]
public static partial void PlaySound(string url);

Finally, call the PlaySound method in C# to run the globalThis.playSound JavaScript function:

PlaySound("http://sound.com/audio.mp4");

However, when dealing with a website, you might run into CORS (Cross-Origin Resource Sharing) issues. Make sure the remote host allows you to download the file you want. It’s always a good idea to look at your browser’s DevTools console for hints about any issues related to cross-site scripting errors.

Check out my submission for Toy Box Jam 3, This Is Neat-O!

Thirty years ago, Captain Neat-O lost his way home after entering a doomed trans-dimensional portal somewhere around Mars.

It’s up to you to help him find his way back home, before his parents find out…

Easy Mode: Collect all Neat Coins in each level to open the exit portal. Go to the portal to get to the next level.

Hard Mode: Collect all pickups in each level and then finish the level. It turns out Captain Neat-O doesn’t mind vegetables, but hates CHICKEN!

If you don’t feel like finishing the levels or get stuck, feel free to skip levels by pressing +. You can even skip to the end and see what happens. I don’t mind :)

Play in browser

While working on a 2D platformer on Unity, I decided to implement ladders with tile maps with a specific layer designated for ladders. I added a Tilemap Collider 2D component to it with Is Trigger set to true. The plan was to detect the overlap of player and a ladder tile using Unity’s OnTriggerEnter2D(Collider2D) and OnTriggerExit2D(Collider2D) methods.

The Collider2D object refer to the collider attached to the tilemap. I thought collider2D.transform or .bounds properties would return the position of the actual overlapping tile, but I was wrong.

In the end I solved the problem using a different approach:

1, Get the Tilemap object for the ladder,

2, Use the .WorldToCell(Vector3) method of the tilemap to get the discrete cell position of the tile that overlaps with the passed position, in this case player’s transform.position.

3, The center of the overlapping tile would be the world position of the tile map + the cell position + half of tileMap.cellSize.

foreach (var collision in Colliders.Where(x => x.gameObject.layer == LadderLayerId))
{
    IsLadder = true;
    var tileMap = collision.GetComponent<Tilemap>();
    var cellPos = tileMap.WorldToCell(transform.position);
    var tile = tileMap.GetTile(cellPos);
    if (tile != null)
        LadderCenter = tileMap.transform.position + cellPos + tileMap.cellSize * 0.5f;
}

I have an ELM327 OBD2 BLE dongle that shows up as IOS-Vlink. To communicate with it, the service ID is: E7810A71-73AE-499D-8C15-FAA9AEF0C3F2 and both read and write is done using the characteristic BEF8D6C9-9C21-4C9E-B632-BD58C1009F9F.

Once the car (ECU) is on and the Bluetooth is connected, we need to initialize the dongle, by sending:

AT E0\r
AT L0\r
AT SP 00\r
01 00\r

AT E0 is to turn echo off. AT SP 00 searches for the right protocol to use to communicate with the ECU.

Once initialized, we can run a loop to query for the values we want:

RPM is 01 0C\r
Speed is 01 0D\r
Throttle position is 01 11\r
and so on.

This Wikipedia article has a table of PIDs that can be queried from the car and how to translate the results to human readable values.

Generally when a PID is queried (e.g. 01 0C\r), the result is something like 41 0C 0B C0 where 0C is the PID for this response and 0B C0 are the actual returned values in hex.

I often get corrupt replies from the dongle such as 41 0C 0B C, where the message is not fully received, which I ignore.

Responses end with >, so that’s something to look for. Read until you get >, split by line breaks, parse the lines.

Some PIDs take longer to get a response for than the others, so make sure you query what you want, so you can get the desired values with the highest frequency. Also note that ECUs have different speeds, some return results faster, some slower, some need waits between commands.

Sam Afshari's Notes - Contact me: sa at neat.lu

👀 Notes on this right here page 🤠

How Lockstep Multiplayer Works in Generals64 - Mon, Jul 20, 2026

Commands replace state replication

A frame is complete when the counts match

Every peer must use the same order

UDP, acknowledgements, and the packet router

Recovering missing frame data

Run-ahead is the latency budget

Determinism starts with the seed

The logic CRC detects divergence

The tradeoff

Building Fast Sound Library - Sun, Jul 19, 2026

A hardware-accelerated interface

Loading a library without decoding it

Fast auditioning and broad codec support

How Multiplayer Works in Froggy Works - Mon, Jul 13, 2026

A complete world on every device

Commands instead of world dumps

The coordinator is not a simulation host

WebRTC for gameplay, SignalR for introductions

Checkpoints and host migration

The tradeoff

Extracting OSM POIs - Sun, Jun 1, 2025

Build JSONs from OSM data

JSON to SQLite

Ed 2.23 is out - Fri, Sep 20, 2024

Changelog

KiloTexture - OptimizedWall

Unity Export

Games

Ed - The Level Editor - Wed, Jan 10, 2024

Rule-based brushes

Tile definition classes

Sprite management

Auto terrain

Auto brush

Multi-layer level design

Fully-customizable boards

Create and draw patterns

Color palettes and Pixel art

Design anytime, anywhere

Download

Playing Sound Effects in Avalonia UI - Mon, Apr 10, 2023

Goal

Windows

macOS

WebAssembly

This Is Neat-O! - Sun, Feb 13, 2022

Unity Tilemap Collision Detection - Tue, Jan 25, 2022

Talking to a Bluetooth (BLE) ELM327 Dongle - Tue, Sep 14, 2021