What 18 months building a self-hosted media server taught me about playback

Project: https://quven.tv/ Security model: https://quven.tv/security/ For the last 18 months, I have been building Quven, a self-hosted media server for personal movie, TV, and documentary libraries. I started with a seemingly simple goal: let people keep their media on their own hardware while giving them a polished client experience. Playback quickly became the hardest part. A media server does not simply send a video file to a screen. It has to understand the source, the client, the network, and the user's choices, then select a playback path without making any of that complexity feel visible. These are some of the lessons I learned. 1. "Can this file play?" is the wrong question The real question is whether a particular client can play a particular combination of: container; video codec and profile; audio codec and channel layout; subtitle format; resolution, bitrate, and frame rate; HDR format; network conditions. A client might support the video codec but not the audio track. A browser might decode the video but require a different container. Enabling an image-based subtitle can turn an otherwise direct-playable file into a video transcode. Playback compatibility is therefore not a boolean property of a file. It is a negotiation between the source and the active client. 2. Direct play should be the preferred outcome, not a promise Direct play preserves the original file and avoids unnecessary server work. When the client supports the selected combination, it is usually the best path. But forcing direct play at all costs produces a worse experience. A high-bitrate file may technically be supported while still exceeding the available connection. A selected subtitle might require burning into the video. A television may accept a container while rejecting one of its audio formats. The practical hierarchy I settled on is: Direct play when the complete source is compatible. Remux when the streams are compatible but the container is not. Transcode only what must chan