标签：#digitalocean

Every time I put a model behind an endpoint I make the same lazy decision. I pick whatever I used last time, or whatever I read about most recently, and I tell myself I'll benchmark it properly later, and later never arrives because there is always something with an actual deadline on it and comparing model latencies feels like procrastination even when it isn't. I never do it. Not once. So I built the thing that would make me do it. One prompt, fired at six models at once, streaming side by side in columns, with time to first token and cost per run underneath each one. About 390 lines of Python. Code's here , MIT, take it. Then I ran it, and three things happened that I didn't plan for. The integration is two lines, and that's the least interesting part DigitalOcean's inference endpoint speaks OpenAI, so this is the whole thing: client = OpenAI ( base_url = " https://inference.do-ai.run/v1/ " , api_key = os . environ [ " DIGITAL_OCEAN_MODEL_ACCESS_KEY " ], ) Every model below goes through that one client. Llama, DeepSeek, Mistral, Qwen, OpenAI's open-weight gpt-oss line. Only the model string changes. That is the pitch, and it's real, and I'll move past it quickly because you already knew an OpenAI-compatible endpoint would work like an OpenAI- compatible endpoint. What I didn't know is everything that follows. One footnote before you paste that snippet. The credential is a model access key , created under the Gradient AI Platform. It is not the API token from Settings, API. Different thing, different page. (Although, as I found out later, the endpoint doesn't care nearly as much about that distinction as the docs do.) Six streams, no event loop I wanted the columns to fill simultaneously. Real racing, not six sequential progress bars pretending. The tidy way to do that is one endpoint that fans out server side and multiplexes everything back down a single connection. I didn't do the tidy way. The browser opens one EventSource per model instead: GET /stream?model=<

A quick note on the links below. The DigitalOcean and Vultr links in this article are referral links. If you sign up via them, you get a free credit on your new account (currently $200 over 60 days for DigitalOcean and up to $300 for Vultr) and the author of this article gets a small referral credit too, at no extra cost to you. AWS does not run an equivalent referral program, so the AWS links are normal links. The review below is the author's own evaluation; the credits do not change the recommendations. If you have ever spent a workday watching your website refuse to load, you are not alone. In a recent outage , a single building in Northern Virginia hosting one of Amazon's availability zones (the cloud-industry term for one campus's worth of servers in one region ) got too hot. The hardware shut itself down. AWS calls this a thermal event. Customers around the world have other names for it. Big enterprises ride out outages like this. They have multi-region setups, dedicated SRE teams, and SLA credits that will refund a small fraction of their monthly bill. Small and mid-size businesses do not. They lose a day of revenue, scramble to reassure customers, and then read a post-mortem in a few weeks that explains what went wrong in language that does not help them recover the lost revenue. The cloud was supposed to make small businesses look big. After each new outage, it is fair to ask: is AWS actually the right cloud for small businesses at all? Two providers worth a serious look, DigitalOcean and Vultr , are simpler, cheaper at the entry level, and built around use cases that more closely match what a small business actually needs. Here is what each one does, where AWS is still the right answer, and how to decide. Why AWS hits small businesses harder than big ones When a giant company has an AWS outage, three teams kick into gear. There is the engineering team that fails workloads over to a backup region. There is the customer-success team that updates the status p