标签：#AR

If your LLM costs are climbing, the instinct is almost always the same: swap to a cheaper model. GPT-4 to GPT-4-mini. Claude Opus to Claude Haiku. Sometimes that helps a little. It rarely fixes the actual problem. The actual problem, in most workflows I've looked at, is that every step gets routed through the LLM, even the steps that don't need language reasoning at all. This post breaks down a simple mental model for deciding what should and shouldn't touch an LLM, with a working example you can adapt. The four components of any AI workflow Every automated workflow — whether it's a support ticket router, a fraud check, or a content pipeline — is built from some combination of four building blocks. They get treated the same once a workflow diagram is drawn flat, but they have wildly different cost and latency profiles. Component What it does Think of it as Typical cost Trigger Starts the workflow The doorbell ~$0 Deterministic ML Structured predictions — classify, score, rank The calculator Cents per 1,000 calls LLM / Generative Reads, writes, reasons in language The writer Dollars per 1,000 calls Tool / API Fetches or writes real data The hands Cents per 1,000 calls The gap between row 2 and row 3 is the whole article. A classifier and an LLM call can solve the exact same problem, but one costs roughly 100-1000x more than the other, depending on model and provider. If you're not deliberately deciding which one handles which step, you're probably defaulting to the expensive one — because in frameworks like LangChain or a quick custom agent loop, it's just easier to shove everything into a prompt. Where this actually shows up Here's a workflow I see constantly: an automated support ticket triage system. flowchart LR A[New support ticket] --> B{Classify intent} B --> C[Route to team] B --> D[Auto-draft response] D --> E[Update CRM] A naive build sends the entire ticket text to an LLM and asks it to do everything at once: classify the intent, decide routing, draft a re

A look at what's actually changing in Nepal's job market, what it means for students and working professionals, and what separates training that gets you hired from training that just gives you something to print on a resume. Nepal is at an interesting crossroads right now. On one side, the country still carries the weight of a job market that hasn't kept up with its graduates. Every year, more than 500,000 young people enter the workforce. The economy, for all its resilience, simply does not generate enough traditional jobs to absorb that number. The result is familiar to most Nepali families: children who studied hard, passed their exams, collected their certificates, and then spent months, sometimes years, waiting for something to happen. On the other side, something genuinely different is building. Nepal's IT exports crossed $1 billion in 2025, according to NASIT's estimates. Software and BPO exports grew over 20% in the first seven months of fiscal year 2024/25 alone. The government's 16th development plan has set a target of 250,000 new IT jobs and a 5% GDP contribution from the sector by 2029. International companies, from Indian IT majors to US-based outsourcing firms, are paying attention to Nepal in ways they weren't a decade ago. These two realities exist at the same time, in the same country, often in the same family. A brother driving a taxi while his younger sister lands a remote software development contract earning more than their father ever did in a government job. The difference between those two outcomes, more often than not, comes down to whether someone made the decision to learn something the market actually needs, and found a way to learn it properly. That's what this piece is about. The Skills Gap Problem Nobody Talks About Enough Nepal's IT sector is growing, but that growth comes with a problem attached: a persistent, widening mismatch between what employers need and what most fresh graduates can actually do on day one. Companies like Deer

There's a headline pattern that feels like sharp marketing writing but quietly costs conversions. "You code. We cloud." It's clever. The parallel structure is tight. It names a clear division of labor. But it describes the service delivery model , not the developer outcome — and those are different things to someone scanning a landing page in five seconds. The audit fastapicloud.com is a managed hosting product built specifically for FastAPI developers. The hero H1 is: "You code. We cloud." On the surface this reads as clean, confident B2B positioning. In practice, it names the mechanism: You = who does the coding We cloud = who handles the infrastructure What's missing is the output. What does the developer actually walk away with? The gap (mechanism-first H1): The headline describes the service model without anchoring it in the developer outcome. The visitor has to make a three-step inference: "they handle the cloud" → "that means I don't do ops" → "so my app gets to production without a week of DevOps work." In five seconds of scrolling, most won't finish that chain. The headline earns a nod of recognition. It doesn't earn the scroll. The fix One line changes the frame completely. Before: "You code. We cloud." After: "Your FastAPI app is live in production — zero config rabbit holes, zero deploy-day surprises." The rewrite keeps the same promise — they handle the infrastructure — but anchors it in the developer's world. The outcome (app in production) is first. The pain points ("config rabbit holes," "deploy-day surprises") are the exact things a FastAPI developer has already lived through. "Zero config rabbit holes" names the experience of spinning up a production server for the first time. "Zero deploy-day surprises" names the dread: the Sunday night broken deploy that wasn't caught in staging. Any backend developer who reads that line knows exactly what it's describing. The mechanism (managed cloud, they handle ops) is still implied. But the headline earns the

AI wrote the first 80% of my feature in 10 minutes. The code was clean. The logic made sense. The...

When you've worked with a concept long enough, there's a gap that can quietly open up between "I know...

We live in an era where our most intimate data—heart rates, sleep cycles, and step counts—is constantly uploaded to the cloud for "analysis." But what if you could have a world-class AI medical assistant living entirely on your device? Today, we are pushing the boundaries of Edge AI and Privacy-preserving machine learning by deploying a quantized Llama-3 model directly onto an iPhone using MLC-LLM . By leveraging Apple HealthKit and hardware acceleration via Metal , we can transform "Pixels and Pulses" into actionable insights without a single byte leaving the device. This tutorial dives deep into the architecture of on-device LLMs, specifically focusing on how to bridge the gap between high-performance C++ runtimes and a React Native UI. If you're interested in more advanced patterns for production-grade AI integration, be sure to explore the engineering deep-dives at the WellAlly Blog , which served as a massive inspiration for this architecture. 🚀 The Architecture: Why On-Device? The challenge with running Llama-3 on mobile isn't just memory—it's the data pipeline. We need to fetch sensitive data from HealthKit, format it into a prompt, and run inference using the phone's GPU. System Data Flow graph TD A[User Query: How was my sleep?] --> B[React Native UI] B --> C{Swift Bridge} C --> D[Apple HealthKit API] D --> E[Health Data Context] E --> F[MLC-LLM Engine] G[Quantized Llama-3 Weights] --> F F --> H[On-Device Inference via Metal] H --> I[AI Generated Health Report] I --> B 🛠 Prerequisites MLC-LLM : Our compiler stack for universal LLM deployment. TVM (Tensor Virtual Machine) : The backbone for hardware acceleration. React Native : For the cross-platform UI. Xcode & Swift : To interface with Apple's HealthKit. Llama-3-8B-Instruct (Quantized) : We'll use 4-bit quantization (q4f16_1) to fit within mobile RAM limits. Step 1: Quantizing Llama-3 for Mobile Standard Llama-3 is too heavy for a phone. We use the MLC-LLM CLI to compile the model into a format that the iP

There's a quiet assumption baked into a lot of agent code: that a bigger context window means a better memory. Vendors ship 200K, then 1M, then 2M token windows, and the implied promise is "just put everything in and the model will remember." After building agents that run for weeks, I've come to think this conflates two things that are not the same — and treating them as the same is exactly why long-running agents get dumber over time. The context window is working memory. Real memory is what survives when the window is gone. Mixing them up is like confusing your desk with your filing cabinet. Two different clocks Working memory (the context window) lives for one session, maybe one turn. It's fast, expensive, and volatile. It's where reasoning happens right now . Durable memory lives across sessions. It's slow, cheap, and persistent. It's what the agent knows when it wakes up tomorrow with an empty window. These have different lifespans, different costs, and different access patterns. The moment you try to make one do the other's job, things break: Use the window as memory → everything you "remember" has to be re-loaded every turn, you pay for it every turn, and the instant the session ends it's gone. Use durable storage as working memory → you're reading and writing files mid-reasoning for things that only matter for the next 30 seconds. A good agent keeps them separate on purpose. Why "just use a bigger window" fails Say you have a 1M token window and you stuff the entire history in. Three problems show up, none of which a bigger number fixes: Cost scales with every turn, not every session. That 1M tokens isn't paid once — it's re-sent on each step of a multi-turn task. A 20-step task can mean 20× the bill, mostly re-reading the same stale history. Attention dilutes. "Lost in the middle" is real: models attend most reliably to the start and end of a long context. Bury the one fact that matters under 900K tokens of transcript and recall quality drops, even though

46.4%. That number — ChatGPT's June 2026 market share — ends a streak that held since November 2022. For the first time since the product launched, OpenAI holds less than half the AI assistant market. Gemini is at 27.7%. Claude is at 10.3%. The monopoly phase of AI assistants is over. The data comes from a June 2026 market report tracking monthly active users across major AI assistants. ChatGPT still leads with 1.11 billion monthly users — a number that would define the entire category in any other software market. But Gemini has 662 million, up 129 million in five months. Claude sits at 245 million, nearly four times its December 2025 count of 60.2 million. The trajectory is the story, not the absolute numbers. Why the 50% Threshold Actually Matters Below 50% doesn't mean decline. ChatGPT's absolute user count keeps growing. What the threshold signals is the end of single-platform dominance — the condition where building for "AI users" meant building for ChatGPT users. That assumption no longer holds in mid-2026. For context: search engine market share stayed above 90% for Google for nearly a decade after competitors entered. Social network market share for Facebook stayed above 70% for years after Instagram and Twitter had genuine scale. The pace of AI assistant fragmentation is meaningfully faster than those precedents. Three products above 10% share in under two years of real competition is an unusually fast split. What fragmentation means practically: the community knowledge base — YouTube tutorials, Reddit threads, prompt libraries — that once pointed almost exclusively at ChatGPT now covers three platforms with genuine depth. That changes how you can expect your users to arrive at your AI-integrated product, and what they already know about AI when they get there. Gemini's 662 Million Users Are Not What They Look Like Gemini's surge from under 500 million to 662 million monthly users in five months is impressive on paper. The driver is less impressive: Google

OpenAI is attempting to tackle the security issues of the open source software community.

The lawsuit, led by a Detroit pension fund, alleges Uber's board and management has cut too many compliance corners, resulting in thousands of lawsuits.

Good architecture is not only about how a system is built. It is also about how well the team can understand that system once it is running. That is where observability belongs in the architecture conversation. It is common for observability to be treated as something that comes after the main engineering work. The service gets built. The API works. The deployment succeeds. Then, somewhere near the end, the team starts thinking about logs, dashboards, alerts, and operational visibility. That approach creates a gap. The architecture may look clean on paper, but once the system is in production, the team has to understand how it behaves under real conditions. Real users do not follow the happy path perfectly. Dependencies slow down. Queues back up. Data arrives in unexpected shapes. Deployments change behavior in ways that are not always obvious. If the system does not give the team a way to see those things clearly, the architecture is incomplete. Observability is not decoration around the system. It is part of the system design. Architecture Describes the System. Observability Shows the Truth. Architecture is built on assumptions. During design, teams make reasonable guesses about usage patterns, service boundaries, dependency behavior, data flow, scale, latency, and failure modes. Some of those assumptions are based on experience. Some are based on current requirements. Some are simply the best call the team can make with the information available at the time. That is normal. The problem is not that architecture contains assumptions. Every architecture does. The problem is when those assumptions cannot be tested once the system is real. A design might assume that an external dependency will be reliable enough. Production may show that it is the slowest part of the request path. A queue might look like a clean decoupling point during design. Production may reveal retry behavior, duplication, or ordering concerns that were not obvious upfront. A serverless function m

The first text message ever sent was not a love note, a meeting reminder, or a meme. It was a Christmas greeting. On December 3, 1992, a 22-year-old engineer named Neil Papworth sat at a desktop computer, typed two words, and sent the world's first SMS to a mobile phone: "Merry Christmas." More than thirty years later, that humble two-word message has grown into one of the most quietly important protocols in connected technology, and it still shows up in the IoT devices we build today. The engineer who sent the first SMS Neil Papworth was working for the Anglo-French firm Sema Group Telecoms, part of a team building a Short Message Service Centre (SMSC) for the British carrier Vodafone. The SMSC was the piece of infrastructure that would store and forward text messages across the cellular network. To prove it worked, Papworth sent a test message from a computer terminal to the Orbitel 901 handset of Richard Jarvis, a Vodafone director who was at a company Christmas party. The message arrived. Jarvis read it. But he could not reply, because mobile phones at the time had no way to compose a text. There was no keypad-driven messaging app, no T9, no touchscreen. SMS started life as a one-way novelty riding on a spare slice of the network's signalling channel, and almost nobody involved thought it would matter very much. Why SMS was designed the way it was The technical detail that makes this story relevant to anyone building connected hardware is how SMS was engineered. Text messages were squeezed into the control channel that phones already used to talk to cell towers, the same channel that handles things like call setup. That is why a single SMS is capped at 160 characters: it had to fit inside a small, fixed-size signalling packet. This constraint turned out to be a feature. SMS is lightweight, store-and-forward, and works even when a data connection is weak or absent. The message waits in the SMSC until the device is reachable, then gets delivered. No persistent con

The move comes after the company left potentially sensitive data from the initiative exposed internally.

Employees had previously raised concerns about the initiative, which involves collecting workers’ keystroke data to train AI models.

Spoiler: It's more Siri stuff.

What does an AI company do after one of those not-acqui-hire deals? Groq raised money, is leaning into its neocloud business, and is hiring new execs.

"Winning" bets were made on cloned website and would have lost money, WSJ finds.

An AI agent that can email but can't hold a calendar slot is only half useful. The moment a conversation turns into "let's meet Thursday at 2," the agent needs a real calendar — one that sends invitations people accept in Google Calendar or Outlook, receives invites at its own address, and RSVPs back so the organizer sees a real response next to everyone else's. Bolting a scheduling library onto a shared mailbox doesn't get you there; the agent needs a calendar identity of its own. An Agent Account ships with exactly that. Every account gets a primary calendar that hosts events, accepts invitations over standard iCalendar, and RSVPs with yes, no, or maybe. To a participant, the agent is just another attendee on the invite. This post walks through using that calendar from two angles: the HTTP API for your backend, and the Nylas CLI for the terminal. I work on the CLI, so the terminal commands below are the ones I reach for. The calendar an Agent Account comes with When Nylas provisions an Agent Account, it creates a primary calendar that belongs to the account. You reach it through the same Calendars and Events endpoints at /v3/grants/{grant_id}/... that any other grant uses, so calendar code you've written for a connected Google or Microsoft account works here unchanged. Each account gets: A primary calendar , provisioned automatically. It can't be deleted while other calendars exist on the account. Additional calendars , up to your plan's cap, for separating concerns — a sales-calls calendar and an internal one on the same agent. Free/busy queries , so the agent can check its own availability before proposing a time. Event webhooks — event.created , event.updated , and event.deleted fire on every change, whether it came from the agent or from someone responding to an invitation. List the calendars from the terminal with nylas calendar list , or over the API with GET /v3/grants/{grant_id}/calendars . Both return the primary calendar plus any you've added. List what'

"It runs on my own GPU, so it's basically free." I believed that until I put a meter on it. So I ran a controlled benchmark on one box — an openSUSE machine with a single RTX 3090 — driving three local models through ollama under an identical fixed workload (256-token generations in a loop for ~4 minutes each), while my open-source dashboard priced every run by the real GPU energy it burned : power sampled from nvidia-smi every 10 s, integrated over each run's exact window, multiplied by my actual day/night tariff. One number per model, in euros per million output tokens. Here's the part that made me re-run it. The tiny gemma3:1b came out at €0.118 / 1M tokens — about 5× cheaper than a hosted Flash-class API (~€0.55). But gemma3:27b 's electricity alone was €0.706 / 1M — more expensive per token than just paying the cloud, and that's before a single cent of the GPU's purchase price. "Local" didn't make it cheaper; it made it cost more and I own the depreciation. The mechanism is one line: each token costs watts ÷ throughput , and a big dense model is both slow and thirsty. A newer mid-size architecture ( gemma4:26b ) bought a lot of that back, landing at €0.272 . The full guide is methodology-first and reproducible end to end — minting an ingest key, the stdlib-only client, the exact ollama loop that reads eval_count / eval_duration for real tokens-per-second, reading each run back priced, and the honest caveats (this is marginal GPU energy only — not capex, idle, or cooling — and the absolute numbers round to fractions of a cent; the shape is the finding). Read the full guide on Medium → https://medium.com/@arsen.apostolov/how-much-does-it-actually-cost-to-run-a-local-llm-per-million-tokens-measured-4a90a7f31a48

The Ray-Ban Meta Optics lineup adds a premium fit and a few extra features, but comes with a higher price.